System and method for determining priority of data processing

Abstract

The present disclosure relates to systems and methods for prioritizing data processing. A vehicle data processing system includes a set of sensors and a controller. The set of sensors are arranged on a vehicle and are operable to detect and capture driving events. The controller is coupled to the set of sensors and is operable to receive one or more data streams indicative of driving event data from the set of sensors. The controller is further operable to (i) analyze the one or more data streams, (ii) determine a current operating state of the vehicle based on the one or more data streams, (iii) determine whether the current operating state of the vehicle triggers a prioritized processing of the one or more data streams, and (iv) apply the prioritized processing when it is determined that the prioritized processing is triggered.

Description

Technical Field

[0001] The embodiments described herein generally relate to systems and methods for determining the priority of data processing for use in vehicles, and more specifically, to systems and methods for determining the priority of data processing based on the current operating state of the vehicle. Background Technology

[0002] The amount of data generated and transmitted by vehicles is increasing dramatically. Some vehicles are already generating massive amounts of data that are being processed and transmitted to remote computer systems, such as via cloud networks. Such data processing and transmission are already very expensive, and these costs are expected to only increase.

[0003] Vehicles may encounter driving events that require largely real-time determination. To make real-time determinations regarding various driving events, vehicle computing systems may need to process large amounts of vehicle data rapidly. If large amounts of vehicle data are processed largely simultaneously, the processor may not return the results required for the vehicle's current operating state. For example, an autonomous vehicle may need to prioritize identifying moving pedestrians over detecting nearby buildings, potentially causing the processor to process vehicle data in a different order. Furthermore, the processor may be overloaded by concurrent data processing, leading to inefficiency and / or latency. Therefore, there is a need for systems and methods that prioritize vehicle data processing based on the vehicle's current operating state. Additionally, there is a need for systems and methods for determining the vehicle's current operating state in relation to processing priorities that may trigger or prescribe priority processing. Summary of the Invention

[0004] In one embodiment, the vehicle data processing system includes a set of sensors, a controller, and a communication interface. The set of sensors is disposed on the vehicle and is operable to detect and capture driving event data. The controller is coupled to the set of sensors and is operable to receive one or more data streams indicative of driving event data from the set of sensors. The controller is also operable to (i) analyze the one or more data streams, (ii) determine the current operating state of the vehicle based on the one or more data streams, (iii) determine whether the current operating state of the vehicle triggers priority processing of the one or more data streams, and (iv) apply the priority processing when it is determined that priority processing has been triggered.

[0005] In another embodiment, the vehicle data processing system includes multiple sensors, a processor, and a memory. The multiple sensors collect one or more data points related to driving events of the vehicle. The processor is coupled to the multiple sensors and is operable to receive the one or more data points. The processor is also operable to determine the current action of the vehicle based on the one or more data points. The memory is coupled to the processor and stores a state manager program and a sensor data queue for storing a set of sensor data prior to the current reference point. When executed by the processor, the state manager program causes the processor to determine the priority of the current sensor data collected by the multiple sensors at the current reference point and the set of sensor data queued in the sensor data queue based on the current action of the vehicle, and processes the current sensor data and the set of sensor data.

[0006] In another embodiment, the vehicle data processing method includes the following steps: (i) using multiple sensors to collect one or more data points related to driving events of the vehicle; (ii) storing a set of sensor data collected before the current reference point in a sensor data queue; (iii) receiving one or more data points from the multiple sensors at a processor; (iv) determining the current action of the vehicle based on the one or more data points; and (v) using the processor, based on the current action of the vehicle, determining the priority of the current sensor data collected by the multiple sensors at the current reference point and the set of sensor data queued in the sensor data queue, and processing the current sensor data and the set of sensor data.

[0007] These and other features provided by the embodiments of this disclosure will be more fully understood in conjunction with the accompanying drawings and in view of the following detailed description. Attached Figure Description

[0008] The embodiments illustrated in the accompanying drawings are illustrative and exemplary in nature and are not intended to limit this disclosure. The following detailed description of the illustrative embodiments will be understood when read in conjunction with the following drawings, in which the same structures are indicated by the same reference numerals, in which:

[0009] Figure 1 A schematic depiction of a connected vehicle system according to one or more embodiments shown and described herein;

[0010] Figure 2 A block diagram illustrating a vehicle data prioritization system according to one or more embodiments shown and described herein;

[0011] Figure 3 Description of execution in Figure 2 A flowchart of the priority processing algorithm used in the vehicle data priority processing system;

[0012] Figure 4A A flowchart describing the determination and prioritization of processes according to one or more embodiments shown and described herein;

[0013] Figure 4B Describes various factors that trigger or prescribe priority processing according to one or more embodiments shown and described herein;

[0014] Figure 5 A flowchart describing the execution of a first priority algorithm according to one or more embodiments shown and described herein;

[0015] Figure 6 A flowchart describing the execution of the second priority algorithm according to one or more embodiments shown and described herein;

[0016] Figure 7 Describe the hierarchical priority level of the examples in the sensor dataset. Detailed Implementation

[0017] Connected vehicles can be equipped to communicate with other devices using connectivity available via wireless and / or cellular networks. Connected vehicles can connect to and communicate with their surroundings. Connected vehicles can communicate through various communication models, including vehicle-to-infrastructure (“V2I”), vehicle-to-vehicle (“V2V”), vehicle-to-cloud (“V2C”), and vehicle-to-everything (“V2X”) communication models. The V2I communication model facilitates communication between vehicles and one or more infrastructure devices, enabling the exchange of vehicle-generated data and information about the infrastructure. The V2V communication model facilitates communication between vehicles and allows the exchange of data generated by surrounding vehicles, including speed and location information. The V2C communication model facilitates the exchange of information between vehicles and cloud systems. The V2X communication model connects all types of vehicles and infrastructure systems to each other.

[0018] As described above, a connected vehicle is operable to capture and generate large amounts of data about the vehicle, surrounding vehicles, the environment, etc. The connected vehicle can seamlessly transmit this data to surrounding vehicles, cloud servers, other infrastructure, etc., and communicate with them via a network. Embodiments disclosed herein include systems and methods for determining the priority of vehicle data processing based on the vehicle's current operating status. Embodiments of the systems and methods described herein for determining the priority of vehicle data processing can provide advantages for use by connected vehicles. For example, embodiments can efficiently process large amounts of vehicle data based on predetermined priorities, or predetermined hierarchical groupings of priorities. This can result in faster, more efficient processing of vehicle data and enable the vehicle to make decisions in response to driving events.

[0019] In the embodiments disclosed herein, prioritizing vehicle data can reduce the size or volume of vehicle data processed at a specific point in time. In the embodiments disclosed herein, vehicle data can be processed based on a predetermined priority, rather than simultaneously. In some embodiments, prioritization can be based on multiple factors representing the current operating state of the vehicle, such as the nature of the data, the purpose of the data, identified objects in the scene, the vehicle's location, and the programmed vehicle operations of the autonomous vehicle. These multiple factors may vary and can be modified based on the current operating state and various driving events.

[0020] In the embodiments disclosed herein, it can be determined whether vehicle data, such as sensor data and video data, indicates a predetermined operating state of the vehicle. In some embodiments, such determination is associated with a determination of whether priority processing is required. Depending on the current operating state of the vehicle, certain data may undergo priority processing. For example, a vehicle traveling at high speed (e.g., on a highway) may have priority applied to data streams from vehicle sensors related to vehicle speed. As another example, if a vehicle is traveling along a city street and plans to make a right turn, then data streams from vehicle sensors related to right turns (such as cameras capturing moving objects near the right side of the vehicle) may be processed according to a priority-determining system.

[0021] The embodiments described herein relate to prioritizing data processing based on the urgency of data processing relative to vehicle operation. In some embodiments, the order in which data from various sensors is processed and / or transmitted is determined based on data usage. For example, for a vehicle using image data for navigation, collision avoidance, or other related systems, image data related to pedestrian traffic (e.g., related to identified pedestrians) may be processed first, while data or images or portions of images containing background buildings may be processed later, as updating dynamic data (such as pedestrian traffic data) may be prioritized compared to relatively static images such as buildings. Various systems and methods for prioritizing vehicle data processing will now be described in more detail with specific reference to the accompanying drawings.

[0022] Figure 1This schematic depicts an automotive system 10 including a vehicle 100 and a cloud computing system 20. The vehicle 100 includes a head unit 120, a storage device 140, and a set of sensors including various sensors 150. The head unit 120 controls the operation of the vehicle 100 based on data points captured and transmitted from the sensors 150. The storage device 140 is coupled to the head unit 120 and stores a set of data points under the control of the head unit 120. The sensors 150 include various types of sensors used in the vehicle 100. In some embodiments, the sensors 150 include one or more cameras, LiDAR sensors, radar sensors, ultrasonic sensors, accelerometers, proximity sensors, brake sensors, motion sensors, etc. The sensors 150 used in the vehicle 100 are not limited to these and other sensors can be implemented.

[0023] In some embodiments, vehicle 100 also receives data points from a set of sensors 170 that may be disposed externally to vehicle 100. For example, sensors 170 may be disposed on or near buildings such as parking lots, municipal infrastructure, or the surrounding environment of vehicle 100. Vehicle 100 can receive data points from sensors 170 via network 200. In other embodiments, vehicle 100 can receive data points from surrounding vehicles 210 via a V2V communication channel. Sensors 170 may include various types of sensors, such as one or more cameras, LiDAR sensors, accelerometers, proximity sensors, brake sensors, motion sensors, etc.

[0024] like Figure 1 As shown, vehicle 100 includes a communication unit 180 for exchanging data and information between vehicle 100 and network 200. Figure 1 As shown, vehicle 100 can connect to and communicate with one or more edge servers 220, 240, and 260. Edge servers 220, 240, and 260 can connect to and communicate with central server 300. Central server 300 can communicate with receivers 280 and 285. Receivers 280 and 285 also communicate with vehicles 100 and 210.

[0025] See Figure 2This document details the structure and operation of a vehicle data prioritization system 400 according to one or more embodiments shown and described herein. The vehicle data prioritization system 400 includes a processor 410, a memory 420, and a set of sensors 460. A controller local area network (CAN) bus 440 is connected to the processor 410, the memory 420, and the set of sensors 460, and serves as a communication interface. The sensors 460 provide various data points to the processor 410, which in turn processes these data points for various purposes. The processor 410 analyzes the data points from the sensors 460 to make various decisions. In some embodiments, the processor 410 analyzes the data to determine whether to save or discard the data points, and if so, whether to save them on-board or externally. In other embodiments, the processor 410 analyzes the data to determine the necessary action or reaction to the data points. For example, the processor 410 receives data points from sensors such as accelerometers, analyzes the data points, and detects that the vehicle 100 may be at risk of a collision. The processor 410 may then determine whether to cause actions such as stopping the vehicle 100 or outputting a warning requiring the vehicle 100 to slow down.

[0026] Alternatively, the processor 410 analyzes data points from the sensor 460 to determine the operating status, speed, rotational speed, acceleration, deceleration, steering, and movement of objects of the vehicle 100. In other embodiments, the processor 410 may determine the operating status and maintenance needs of various components of the vehicle 100, such as whether there are any maintenance problems with the engine of the vehicle 100.

[0027] In some embodiments, the vehicle data priority processing system 400 may, for example, Figure 1 The system is implemented in the audio head unit 120 shown. In other embodiments, the vehicle data priority processing system 400 may be implemented independently of and separately from the audio head unit 120. In some embodiments, the vehicle data priority processing system 400 may be included in the vehicle 100.

[0028] In some embodiments, memory 420 includes a priority processing algorithm 430 and a predetermined operating state 435. The priority processing algorithm 430 is configured to determine the priority of data processing based on various factors. In some embodiments, the priority processing algorithm 430 is configured to determine the priority of data processing based on the urgency of data processing relative to the operation of vehicle 100. In other embodiments, the priority processing algorithm 430 is configured to determine the priority of data processing based on vehicle data usage, vehicle operating conditions, vehicle 100 location, required response time, data processing requirements, or a combination thereof.

[0029] In some embodiments, memory 420 includes a set of predetermined operating states 435, each of which may be associated with a priority assigned to it. For example, for autonomous vehicle driving, the processing and detection of dynamic features, such as pedestrian movement, may be prioritized over static features. As another example, when a vehicle is stopped at an intersection, the tracking of vehicles, pedestrians, and objects may be prioritized. As yet another example, vehicle data relating to completing or supplementing the vehicle's next sequence of movements may be prioritized. As yet another example, for vehicles using image data for navigation, collision avoidance, or other related systems, image data relating to pedestrian traffic (e.g., related to identified pedestrians) may be prioritized, while data or images, or portions thereof, containing background buildings, may be processed later, as their updating may not be as urgent as updating pedestrian traffic. In some embodiments, the set of predetermined operating states 435 defines vehicle conditions, operating conditions, driving event contexts, etc., that may trigger or prescribe priority-based processing.

[0030] In some embodiments, the priority processing algorithm 430 includes a machine language-based program for processing data points. Additionally or otherwise, the memory 420 may store one or more machine learning algorithms or programs, such as deep learning programs, neural network programs, pattern recognition programs, etc. Machine learning algorithms can enable the system to learn the driving patterns of vehicle 100, predict situations that may lead to a crash or collision of vehicle 100, and so on. In other embodiments, the memory 420 further stores programs for recognizing and modifying user profiles so that vehicle settings can be changed accordingly. The memory 420 further stores various programs related to the analysis of data points from sensors 150 and / or 170. The memory 420 also stores various applications, such as media players, navigation programs, programs used in vehicle settings, etc.

[0031] See Figure 3 The flowchart details the process 500 of executing the priority processing algorithm 430. Processor 410 receives data points from sensor 460 (step 510). As described above, sensor 460 includes a variety of different sensor sets that generate data streams indicating different operating conditions of vehicle 100. Based on the received data points, processor 410 determines the current operating state of vehicle 100 (step 520). In some embodiments, the current operating state of vehicle 100 includes driving speed, driving position, driving environment (e.g., road conditions, weather conditions), vehicle maintenance status (e.g., component failure, component replacement, etc.), or combinations thereof. Additionally, the current operating state may include any indicator that could lead to the determination of priority processing of vehicle data. Furthermore, the current operating state may further indicate the importance and / or use of vehicle data.

[0032] In some embodiments, the current operating state may indicate whether priority processing is required. For example, a current operating state indicating driving speed, driving position, driving environment, maintenance status, the importance of vehicle data, the use of vehicle data, or a combination thereof may trigger, prescribe, or invoke priority processing. The current operating state and priority processing will be discussed in conjunction below. Figure 4A and 4B Let's discuss this in more detail. See also: [link to previous page] Figure 3 Once it is determined that the data point corresponds to the current operating state (step 530), priority processing associated with the current operating state can be applied (step 540). Subsequently, the processor 410 generates output based on this priority processing (step 550).

[0033] Figure 4A The flowchart illustrating further details of step 530 describes the determination of whether a vehicle data point corresponds to the current operating state (step 610). The current operating state can trigger or predetermine priority processing by processor 410. In some embodiments, the current operating state may indicate a situation where a real-time response may be required, which may involve the next movement of the vehicle, the mitigation of damage and / or injury, the prevention of an emergency, etc. Alternatively, the current operating state may indicate a background or situation that is more relevant and urgent than the current driving event. Figure 4A As shown, the vehicle data points correspond to current operational states, such as those involving collision avoidance (step 620), pedestrian tracking (step 630), the next operation sequence (step 640), etc. These operational states are merely examples, and the vehicle data prioritization system 400 is not limited thereto. In some embodiments, when a vehicle data point corresponds to a current operational state indicating collision avoidance (step 620), prioritization is performed (step 660). If the current operational state indicates an approaching accident or accident situation, then the processor 410 prioritizes the vehicle data point, thus providing a faster and necessary response. A faster and necessary response may lead to collision avoidance.

[0034] When a vehicle data point corresponds to a current operating state indicating pedestrian tracking (step 630), priority processing is performed (step 660). For example, when a vehicle, such as an autonomous vehicle, is stopped at an intersection, pedestrian tracking may need to be prioritized. Similarly, when a vehicle is driving in a congested area, data associated with pedestrian tracking may need to be prioritized. When a vehicle is driving, tracking moving objects such as pedestrians may be more important than detecting static objects such as buildings or trees. Furthermore, the vehicle's current operating state may include determining the traffic conditions around the vehicle, the vehicle's location (e.g., highway, rural road, city street), the vehicle's speed, the frequency of stop-and-go traffic, or combinations thereof. By prioritizing data, the processor 410 can quickly determine the vehicle's situation and provide a rapid response to that situation.

[0035] In other embodiments, in the case of autonomous vehicles, processing resources can be prioritized for creating a three-dimensional (3D) landscape compared to other processing needs, since the autonomous vehicle depends on the current landscape. Alternatively, within a 3D landscape, regardless of whether it is an autonomous vehicle, the processing and detection of dynamic features such as pedestrian movement are prioritized over static features such as updating the positions of buildings, trees, etc.

[0036] like Figure 4A As shown, if a data point does not correspond to the current operating state, the processor 410 may process these data points according to a predetermined order of normal processing (step 650). In some embodiments, the predetermined order may be based on multiple factors, such as the time when the processor 410 receives the data points, the processing time required to process those data points, the current processing load of the processor 410, whether there are external requests for data points from the cloud or any other component of the vehicle, or a combination thereof. Processing is performed based on the predetermined order (step 670).

[0037] Figure 4B This describes various scenarios that may trigger or prescribe priority processing for 690. In some embodiments, these scenarios include current operating state, current action, driving event, driving condition, driving environment, nature and purpose of vehicle data, processing and / or response time, computing resources, etc. Figure 4B As shown, one example of various scenarios includes data importance 680. In some embodiments, the priority of information processing or transmission can be determined based on the importance of the information and / or its use. For example, information related to pedestrians may be immediately transmitted to vehicles based on their geographical location, while information related to weather conditions may be transmitted to a central server later.

[0038] Another factor may include accident avoidance 682. For example, for vehicles that use image data for navigation, collision avoidance or other related systems, image data related to pedestrian traffic (e.g., related to identified pedestrians) may be processed first, while data or images or portions of images containing background buildings may be processed later, because updating these data may not be given priority compared to pedestrian traffic.

[0039] In relevant situations, vehicle maintenance (684) may be given priority. For example, considering image data used for road navigation while the vehicle is in motion, engine status data such as fuel condition and level may not be given priority. When the vehicle is stationary, if computing resources are available, data associated with the maintenance status of the engine and / or the vehicle may be given priority and processed.

[0040] Another example of various scenarios includes a response time of 688. In some embodiments, determining the priority of processing data may be based on an estimate of how long it takes to process the data to obtain a result, and / or the response time to send information outside the vehicle (i.e., to another vehicle or server) for processing and receiving the result. In some cases, especially when large amounts of data need to be processed or complex images need to be obtained, it may be ideal to distribute the data to other local computing devices to achieve the desired priority for the results. Determining priority may include the order in which data is transmitted between local distribution networks, regional distribution networks, and central distribution networks.

[0041] Another example of various scenarios includes the vehicle's driving position 692. For instance, when a vehicle is driving on a highway or city street, processing may be subject to change. For example, when a vehicle is driving on a highway, processing image data for highway navigation may take precedence over processing data points that can be processed when the vehicle is stationary or driving at low speeds. As another example, when a vehicle is driving at an intersection on a city street, processing data points related to moving pedestrians, nearby vehicles, etc., may take precedence over detecting static objects such as nearby buildings.

[0042] For example, sensor data related to driving conditions and / or driving environment 686 can be processed according to priority. For example, priority can be given to processing the determination of road conditions (e.g., the identification of road anomalies such as potholes).

[0043] Another factor may be the urgency of data point 694. In some embodiments, urgency may be defined based on the vehicle’s current movement. For example, considering image data used for navigating the road while the vehicle is in motion, engine status data such as fuel condition and fuel level may not be given priority, as the image data may be of urgency for the vehicle’s movement.

[0044] although Figure 4BSeveral factors that may trigger priority processing 690 are described; however, these factors may overlap to some extent. For example, data importance 680 may also be associated with urgency 694. In other embodiments, location 692 may influence both data importance 680 and urgency 694. Driving conditions and driving environment 686 may influence vehicle maintenance 684 and accident avoidance 682.

[0045] In some embodiments, processing and / or transmission of data points from the vehicle's sensors 460 may be prioritized based on the vehicle's current condition, the need for results derived from the data points, and / or available processing resources. For example, if engine status data such as fuel condition and level are no longer prioritized, considering image data used for road navigation while the vehicle is in motion, then a re-prioritization and processing of engine status may occur when the vehicle stops. As another example, determining road conditions may take precedence over performing vehicle health assessments and determining whether maintenance is required.

[0046] In some embodiments, the priorities among different sensor datasets can be modified and updated as the vehicle's operating conditions change. In other words, because changing vehicle operating conditions may affect the established priorities, the prioritization of data points that were previously not prioritized or were previously prioritized may be re-determined. In some embodiments, the priorities among different data points can be relative priorities.

[0047] In some embodiments, the vehicle data prioritization system 400 may collect data from multiple sensors 460 and various components within the vehicle. The memory 420 may include, for example, a state manager program indicating the vehicle's current actions (e.g., cruising on a highway, navigating city streets, stopping at a red light, parking in a parking lot, etc.). The memory 420 also stores sensor data currently being collected by the sensors 460, and sensor data that was previously given lower priority and is currently queued. For example, considering the vehicle's current action of being in motion, engine state data such as fuel condition and fuel level are not prioritized and are queued. When the current action is that the vehicle is in motion, currently collected image data undergoes priority processing for road navigation. When the vehicle is stationary, engine state data may be processed from a state change that was not prioritized, if computing resources are available.

[0048] Figure 5This document describes a flowchart of the execution of a first priority algorithm 700 according to one or more embodiments shown and illustrated herein. Processor 410 receives data points from multiple sensors 460 and / or a cloud server (step 710). Based on the data points, processor 410 can identify the current operating state (step 720). Processor 410 then determines the importance of the data points based on the current operating state (step 730). The next step is to determine whether the data points meet the criteria for priority processing (step 740). If the data points meet the criteria for priority processing, then these data points undergo priority processing (step 760). If the criteria are not met, then a predetermined order of processing for the data points can be retrieved (step 750). The data points can then be processed later based on the retrieved predetermined order (step 755).

[0049] Figure 6 This document describes a flowchart of the execution of a second priority algorithm 800 according to one or more embodiments shown and illustrated herein. Processor 410 receives data points from multiple sensors 460 and / or a cloud server (step 810). Based on the data points, processor 410 may identify the current operating state (step 820). Processor 410 then accesses the sensor data currently being collected and sensor data previously queued with different priorities (step 830). Processor 410 determines the priority of sensor data processing based on the vehicle's current operation (step 840). The vehicle's current operation may change. In some embodiments, the vehicle's current operation may be in motion and then change to a stopped state. In other embodiments, the vehicle's current operation may also change from driving on a highway to driving on city streets. Because the vehicle's current operation may change, the priority of sensor data in the queue may be adjusted (step 850). Processor 410 generates processing-related output (step 860).

[0050] Figure 7 This is an example illustrating a hierarchical priority hierarchy of sensor data from multiple sensors 460 stored in data queue 900. The sensor data stored in data queue 900 may have different priority levels. In some embodiments, collision avoidance has a higher priority 910, and data 920 that could lead to a collision may be the next higher priority. Hardware failures 930 that result in a direct accident scenario may be the next lower priority. Component failures 940 that can be replaced by new components may be the lowest priority. Figure 7 The hierarchical relationships shown are examples, and the priority systems and methods described in this article are not limited to these.

[0051] In other embodiments, when a vehicle is stopped at an intersection, higher priority may include tracking vehicles, pedestrians, and moving objects. In this case, for example, object detection and defining the local environment may have lower priority than pedestrian tracking. As another example, higher priority may include processing sensor data related to completing or supplementing said "next action" if the route is known or the next action is anticipated, such as turning, rather than processing other sensor data, such as weather processing, inter-vehicle communication, and requests. As yet another example, in the case of autonomous vehicles, processing resources may be prioritized for the need to create a 3D landscape compared to other processing requirements, since autonomous vehicles rely on the current landscape. Alternatively, within a 3D landscape, regardless of whether it is an autonomous vehicle, the processing and detection of dynamic features such as pedestrian movement takes precedence over static features such as updating the positions of buildings, trees, etc.

[0052] In one embodiment, the vehicle data processing system includes a set of sensors, a controller, and a communication interface. The set of sensors is disposed on the vehicle and is operable to detect and capture driving events. The controller is coupled to the set of sensors and is operable to receive one or more data streams indicative of driving event data from the set of sensors. The communication interface is coupled to the set of sensors and the controller for data transmission. The controller is also operable to (i) analyze the one or more data streams, (ii) determine the current operating state of the vehicle based on the one or more data streams, (iii) determine whether the current operating state of the vehicle triggers priority processing of the one or more data streams, and (iv) apply the priority processing when it is determined that priority processing has been triggered.

[0053] In another embodiment, the current operating state indicates collision avoidance, moving object tracking, or both. In yet another embodiment, the current operating state indicates the vehicle's speed, vehicle position, driving environment, or a combination thereof. In yet another embodiment, the current operating state indicates the next sequence of operations for the autonomous vehicle.

[0054] In another embodiment, the controller may also be operable to (i) determine a first driving event of the vehicle based on a first set of the one or more data streams, (ii) determine a second driving event of the vehicle based on a second set of the one or more data streams, and (iii) determine the priority between the first driving event and the second driving event based on the current operating state of the vehicle.

[0055] In another embodiment, the controller may also be operable to adjust the priority between the first driving event and the second driving event as the current operating state of the vehicle changes.

[0056] In another embodiment, the controller may also be operable to process one or more data streams of driving events that are not prioritized when computing resources are available.

[0057] In another embodiment, the vehicle data processing system includes multiple sensors, a processor, and a memory. The multiple sensors collect one or more data points related to driving events of the vehicle. The processor is coupled to the multiple sensors and is operable to receive the one or more data points. The processor is also operable to determine the current action of the vehicle based on the one or more data points. The memory is coupled to the processor and stores a state manager program and a sensor data queue for storing a set of sensor data prior to the current reference point. When executed by the processor, the state manager program causes the processor to determine the priority of the current sensor data collected by the multiple sensors at the current reference point and the set of sensor data queued in the sensor data queue based on the current action of the vehicle, and processes the current sensor data and the set of sensor data.

[0058] In another embodiment, as the vehicle's current action changes, the processor is also operable to re-prioritize the current sensor data and the set of sensor data. In another embodiment, when the vehicle's current action becomes stationary, the processor is also operable to process the set of sensor data queued with lower priority. In another embodiment, when computing resources are available, the processor is also operable to process the set of sensor data queued with lower priority. In another embodiment, the processor is also operable to prioritize the processing of one or more data points required for the vehicle's current action.

[0059] In another embodiment, the processor may also be operable based on the vehicle’s current action, without prioritizing the current sensor data and one or more of the set of sensor data stored in a sensor data queue.

[0060] In another embodiment, the vehicle data processing method includes the following steps: (i) collecting one or more data points related to driving events of the vehicle using multiple sensors; (ii) storing a set of sensor data collected before the current reference point in a sensor data queue; (iii) receiving one or more data points from the multiple sensors at a processor; (iv) determining the current action of the vehicle based on the one or more data points; and (v) using the processor, based on the current action of the vehicle, determining the priority of the current sensor data collected by the multiple sensors at the current reference point and the set of sensor data queued in the sensor data queue, and processing the current sensor data and the set of sensor data.

[0061] In another embodiment, the vehicle data processing method further includes the step of re-prioritizing the current sensor data and the set of sensor data as the vehicle's current action changes. The vehicle data processing method also includes processing the set of sensor data queued in lower priority when the vehicle's current action becomes stationary. The vehicle data processing method further includes processing the set of sensor data queued in lower priority when computing resources are available.

[0062] In another embodiment, the vehicle data processing method further includes prioritizing the processing of one or more data points required for the vehicle's current action. The vehicle data processing method also includes, based on the vehicle's current action, not prioritizing the current sensor data and one or more of the set of sensor data stored in a sensor data queue.

[0063] In another embodiment, the vehicle data processing method further includes the following steps: (i) determining a first driving event of the vehicle based on a first set of the one or more data points, (ii) determining a second driving event of the vehicle based on a second set of the one or more data points, and (iii) determining the priority between the first driving event and the second driving event based on the current operating state of the vehicle.

[0064] Although specific embodiments have been illustrated and described herein, it should be understood that various other changes and modifications may be made without departing from the spirit and scope of the claimed subject matter. Furthermore, although various aspects of the claimed subject matter have been described herein, these aspects need not be used in combination. Thus, the appended claims are intended to cover all such changes and modifications within the scope of the claimed subject matter.

Claims

1. A vehicle data processing system, comprising: A set of sensors, which are arranged on the vehicle and operable to detect and capture driving event data; and A controller coupled to the set of sensors and operable to receive one or more data streams from the set of sensors that indicate driving event data; The controller can also be operated as follows: Analyze the one or more data streams; Based on the one or more data streams, determine the current operating status of the vehicle; Determine whether the vehicle’s current operating state triggers priority processing of one or more data streams, wherein the current operating state indicates any one of the autonomous vehicle’s next operation sequence, collision avoidance, and moving object tracking; and When priority processing is determined to be triggered, the priority processing is applied, wherein applying the priority processing includes processing the first data stream corresponding to the first driving event at the current reference point before processing the second data stream corresponding to the second driving event at the current reference point, and wherein the priority processing is based on the corresponding time for processing the first and second data streams to achieve the corresponding result; and The controller is also operable to determine and adjust the priority between the first driving event and the second driving event as the current operating state of the vehicle changes.

2. The vehicle data processing system according to claim 1, wherein the current operating state indicates the vehicle's speed, the vehicle's position, the driving environment, or a combination thereof.

3. The vehicle data processing system according to claim 1, wherein the controller is further operable as follows: When computing resources are available, process one or more data streams of driving events that are not prioritized.

4. The vehicle data processing system according to claim 1, wherein, The priority processing is also based on the corresponding response time between sending the first data stream and the second data stream outside the vehicle and receiving the corresponding result in response to sending the first data stream and the second data stream outside the vehicle.

5. A vehicle data processing system, comprising: Multiple sensors, which collect one or more data points related to driving events of the vehicle; A processor coupled to the plurality of sensors and operable to receive one or more data points, the processor also operable to determine the current action of the vehicle based on the one or more data points, wherein the current action indicates any one of the autonomous vehicle’s next sequence of actions, collision avoidance, and moving object tracking; and The memory, coupled to the processor, stores the state manager program and a sensor data queue for storing a set of sensor data prior to the current reference point. When executed by the processor, the state manager program causes the processor to determine, based on the vehicle's current action, the priority of the current sensor data collected by the plurality of sensors at the current reference point and the set of sensor data queued in the sensor data queue, and to process the current sensor data and the set of sensor data. The determination of priority and processing includes prioritizing and processing the current sensor data collected by the plurality of sensors at the current reference point before the set of sensor data queued in the sensor data queue, based on the vehicle's current action. The determination of priority and processing is based on the corresponding time for processing the first data stream and the second data stream to achieve the corresponding results. and The processor is also operable to determine and adjust the priority between the first data stream and the second data stream as the current operating state of the vehicle changes.

6. The vehicle data processing system according to claim 5, wherein the processor is further operable to re-determine the priority of the current sensor data and the set of sensor data as the current action of the vehicle changes.

7. The vehicle data processing system according to claim 6, wherein the processor is further operable to process the set of sensor data queued in a lower priority manner when the current action of the vehicle becomes a stationary action.

8. The vehicle data processing system according to claim 6, wherein the processor is also operable to process the set of sensor data queued in a lower priority manner when computing resources are available.

9. The vehicle data processing system according to claim 5, wherein the processor is further operable to prioritize processing one or more data points required for the current action of the vehicle.

10. The vehicle data processing system according to claim 5, wherein the processor is also operable to be based on the current action of the vehicle, without giving priority to one or more of the current sensor data and the set of sensor data stored in the sensor data queue.

11. The vehicle data processing system according to claim 5, wherein the determination of priority and processing is further based on the corresponding response time between sending the first data stream and the second data stream outside the vehicle and receiving the corresponding result in response to sending the first data stream and the second data stream outside the vehicle.

12. A vehicle data processing method, comprising: Collect one or more data points related to vehicle driving events using multiple sensors; Store a set of sensor data collected before the current reference point in the sensor data queue; The processor receives one or more data points from the plurality of sensors; Based on the one or more data points, the current action of the vehicle is determined, wherein the current action indicates any one of the autonomous vehicle's next sequence of operations, collision avoidance, and moving object tracking; and Using the processor, based on the vehicle's current action, the processor determines the priority of current sensor data collected by the plurality of sensors at the current reference point and the set of sensor data queued in the sensor data queue, and processes the current sensor data and the set of sensor data. The determination of priority and processing include prioritizing and processing the current sensor data collected by the plurality of sensors at the current reference point before the set of sensor data queued in the sensor data queue, based on the vehicle's current action. The determination of priority and processing is based on the corresponding time for processing the first data stream and the second data stream to achieve the corresponding results. The method further includes determining and adjusting the priority between the first data stream and the second data stream as the current operating state of the vehicle changes.

13. The vehicle data processing method according to claim 12 further includes: As the vehicle's current actions change, the priority of the current sensor data and the set of sensor data is re-determined; and The current sensor data and the additional priority sorting and processing of the set of sensor data are based on the amount of time spent processing the current sensor data, compared to the different amounts of time spent processing the set of sensor data queued in the sensor data queue.

14. The vehicle data processing method according to claim 12 further includes: When the vehicle's current action changes to a stationary action, the set of sensor data queued in a lower priority manner is processed.

15. The vehicle data processing method according to claim 13 further includes: When computing resources are available, process the set of sensor data that is queued in a lower priority order.

16. The vehicle data processing method according to claim 12 further includes: Prioritize processing one or more data points required for the vehicle's current action.

17. The vehicle data processing method according to claim 12 further includes: Based on the vehicle's current action, one or more of the current sensor data and the set of sensor data stored in the sensor data queue are not given priority.

18. The vehicle data processing method according to claim 12 further includes: Based on a first set of one or more data points, determine the first driving event of the vehicle; Based on the second set of one or more data points, a second driving event of the vehicle is determined; and Adjust the priority between the first driving event and the second driving event based on the vehicle's current operating status.

19. The vehicle data processing method according to claim 12, wherein the determination of priority and processing is further based on the corresponding response time between sending the first data stream and the second data stream outside the vehicle and receiving the corresponding result in response to sending the first data stream and the second data stream outside the vehicle.

Images