Systems and methods for adaptive traffic shaping for in-vehicle networks

Abstract

Systems, methods, and devices are described for network adaptive traffic shaping. The system includes a communication data network and a network device. The communication data network is configured to transmit data in the form of data packets. A network device communicates with a communication data network for controlling a plurality of data packets transmitted over the communication data network. Further, the network device is configured to execute the program code to monitor data traffic metrics for data packets transmitted over the communication data network, compare the data traffic metrics to data network requirements, determine whether the data traffic metrics differ from the data network requirements, and transmit the data traffic metrics to the communication data network. And adjusting the data traffic shaper based on determining that the data traffic metric differs from the data network requirement.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle networking, and more particularly, to a system and method for traffic shaping of network traffic on a vehicle network. Background Art

[0002] As technology advances, vehicles include more and more components that communicate messages using, for example, serial data traffic within one or more controller area networks. These components send and / or receive messages with different categories or priorities to the network. In addition, based on various operating modes or conditions of the vehicle and its components, the components may send these messages at uneven or unpredictable times and quantities. As a result, network traffic is highly variable and changes dynamically during vehicle operation.

[0003] Network switches may include traffic shapers that may delay message forwarding or drop messages entirely when the network is congested. Traffic shapers can facilitate the transmission of higher priority data classes while de-rate the transmission of lower priority data classes when congestion prevents the switch from forwarding all received data in an attempt to mitigate the impact on vehicle performance. While traditional traffic shapers achieve their intended purpose, they still need to address the changing data patterns and resulting prioritization of data transmission on in-vehicle networks. Summary of the invention

[0004] Even the loss or delay of packets in the "low priority" category may adversely affect the performance of the vehicle or its components. Furthermore, the categories of data that are considered "higher priority" and those that are considered "lower priority" may change based on various combinations of the operating mode, condition, and age of the vehicle. For example, adding, changing, and / or replacing components of the vehicle may change the optimal traffic shaping configuration. Similarly, updates to the software, instructions, or controls of vehicle components may change the optimal traffic shaping configuration.

[0005] Thus, a communication system having a network adaptive traffic shaper is provided. The network adaptive traffic shaper changes over time to meet the changing needs of the communication system based at least in part on network traffic conditions that occurred during previous vehicle operation. The network adaptive traffic shaper can change automatically without direct intervention (e.g., without software updates), and can continuously optimize compliance with service requirements for each message class.

[0006] The network adaptive traffic shaper may also extend the life of the communication system and / or mitigate the adverse effects of unexpected events within the communication system.

[0007] According to one aspect of the present disclosure, a vehicle is provided, which has a communication data network, a plurality of electronic control units, a plurality of sensors, and a network device. The communication data network transmits a plurality of data. The data is in the form of a plurality of data packets. A plurality of electronic control units ECU communicate with the communication data network for sending a plurality of data packets to the communication data network and receiving a plurality of data packets from the communication data network. A plurality of sensors communicate electrically with the ECU and each sensor provides a sensor signal to the ECU. The network device communicates with the communication data network for controlling a plurality of data packets transmitted on the communication data network. The network device is configured to execute program code to perform the following operations: monitor a plurality of data traffic metrics of a plurality of data packets on the communication data network, compare the plurality of data traffic metrics with a plurality of data network requirements, determine whether the plurality of data traffic metrics are different from the plurality of data network requirements, and adjust a data traffic shaper based on determining that the plurality of traffic metrics are different from the plurality of data network requirements.

[0008] According to another aspect of the present disclosure, a network switch has a plurality of states adjustable by a data traffic shaper to control data traffic on a communications data network.

[0009] According to another aspect of the present disclosure, a network switch has a first operating mode defined by a first plurality of states among a plurality of states.

[0010] According to another aspect of the present disclosure, the network switch has a second operating mode defined by a second plurality of states of the plurality of states of the network switch.

[0011] According to another aspect of the present disclosure, the network switch has a third operating mode defined by a third plurality of states of the plurality of states of the network switch.

[0012] According to another aspect of the present disclosure, adjusting the data traffic shaper further includes placing the network switch in a first operating mode based at least in part on the classification of the plurality of data packets.

[0013] According to another aspect of the present disclosure, adjusting the data traffic shaper further includes placing the network switch in a second operating mode based at least in part on a plurality of data packets originating from a first plurality of sensors among the plurality of sensors.

[0014] According to another aspect of the present disclosure, adjusting the data traffic shaper further includes placing the network switch in a third operating mode based at least in part on a packet density of the plurality of packets.

[0015] According to another aspect of the present disclosure, the data traffic shaper also includes a reinforcement learning algorithm for determining a first plurality of states among a plurality of states defining a first operating mode, a second plurality of states among a plurality of states defining a second operating mode, and a third plurality of states among a plurality of states defining a third operating mode.

[0016] According to another aspect of the present disclosure, a system having a communication data network and a network device is provided. The communication data network is configured to transmit a plurality of data. The plurality of data includes a plurality of data packets. The network device communicates with the communication data network for controlling the plurality of data packets transmitted on the communication data network. The network device is configured to execute program code to perform the following operations: monitor a plurality of data traffic metrics of the plurality of data packets transmitted on the communication data network, compare the plurality of data traffic metrics with a plurality of data network requirements, determine whether the plurality of data traffic metrics differ from the plurality of data network requirements, and adjust a data traffic shaper based on determining that the plurality of traffic metrics differ from the plurality of data network requirements.

[0017] According to yet another aspect of the present disclosure, the network device is a network switch having a plurality of states adjustable by a data traffic shaper to control data traffic on a communication data network.

[0018] According to yet another aspect of the present disclosure, a network switch has a first operating mode defined by a first plurality of states among a plurality of states.

[0019] According to yet another aspect of the present disclosure, a network switch has a second operating mode defined by a second plurality of states of the plurality of states of the network switch.

[0020] According to yet another aspect of the present disclosure, a network switch has a third operating mode defined by a third plurality of states of the plurality of states of the network switch.

[0021] According to yet another aspect of the present disclosure, adjusting the data traffic shaper further includes placing the network switch in a first operating mode based at least in part on the classification of the plurality of data packets.

[0022] According to yet another aspect of the present disclosure, adjusting the data traffic shaper further includes placing the network switch in a second operating mode based at least in part on a plurality of data packets originating from a first plurality of sensors among the plurality of sensors.

[0023] According to yet another aspect of the present disclosure, adjusting the data traffic shaper further includes placing the network switch in a third operating mode based at least in part on a packet density of the plurality of packets.

[0024] According to another aspect of the present disclosure, the data traffic shaper also includes a reinforcement learning algorithm for determining a first plurality of states among a plurality of states defining a first operating mode, a second plurality of states among a plurality of states defining a second operating mode, and a third plurality of states among a plurality of states defining a third operating mode.

[0025] According to another aspect of the present disclosure, a method is provided. The method includes transmitting a plurality of data on a communication data network, wherein the plurality of data includes a plurality of data packets, and controlling the plurality of data packets transmitted on the communication data network using a network device in communication with the communication data network. The network device has a plurality of states and is configured to execute program code to perform the following operations: monitor a plurality of data traffic metrics of the plurality of data packets transmitted on the communication data network, compare the plurality of data traffic metrics with a plurality of data network requirements, determine whether the plurality of data traffic metrics differ from the plurality of data network requirements, and adjust a data traffic shaper based on determining that the plurality of traffic metrics differ from the plurality of data network requirements.

[0026] According to yet another aspect of the present disclosure, adjusting the data traffic shaper further includes changing a plurality of states of the network device in response to determining that the plurality of data traffic metrics differ from the plurality of data network requirements.

[0027] The above features and advantages and other features and advantages of the present disclosure are readily apparent from the following detailed description of the best modes for carrying out the disclosure when taken in connection with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings are illustrative and are not intended to limit the subject matter defined by the claims. Exemplary aspects are discussed in the following detailed description and are shown in the accompanying drawings, in which:

[0029] Figure 1 is a schematic diagram of a vehicle according to aspects of the present disclosure;

[0030] Figure 2 is a schematic topology of a portion of a communication network according to aspects of the present disclosure;

[0031] Figure 3 is a schematic block diagram of a portion of a communication network according to aspects of the present disclosure; and

[0032] Figure 4 is a flow chart of a method for training an RL algorithm in a pre-production environment according to aspects of the present disclosure. DETAILED DESCRIPTION

[0033] Figure 1 A vehicle 10 is shown in accordance with aspects of the present disclosure. The vehicle 10 generally includes a body 12, a vehicle control system 13, a power plant 14, a braking system 15, a steering system 16, a sensor system 17, an actuator system 18, a communication network 19, and one or more human-machine interfaces 20. The body 12 is disposed on a chassis and substantially surrounds the components of the vehicle 10. The body 12 and the chassis may together form a vehicle frame.

[0034] The control system 13 is configured to control at least one vehicle function by receiving, processing and sending signals between vehicle components. The vehicle control system 13 includes a controller 32 having at least one processor 34, at least one computer-readable storage medium 36, and at least one signal input and output interface 38. The control system 13 is or may include one or more power system, engine, transmission, brake, steering and / or suspension control modules, central control modules, etc. For example, the control system 13 is or may include one or more electronic control units (ECUs) (e.g., "brake ECUs" or "brake control units") responsible for controlling one or more systems or subsystems of the vehicle 10.

[0035] As indicated by the context, the terms "controller", "control module", "control means", "control unit", "processor" and similar terms may refer to application specific integrated circuits (ASICs), electronic circuits, central processing units (preferably microprocessors) and associated memory and storage devices (read-only memory, programmable read-only memory, random access memory, hard disk drives, etc.) that execute one or more software or firmware programs or routines, combinational logic circuits, sequential logic circuits, input / output circuits and devices, appropriate signal conditioning and buffer circuits, other components and combinations thereof, etc. to provide the described functionality. "Software", "firmware", "programs", "instructions", "routines", "code", "algorithms" and similar terms mean controller executable instruction sets including calibration and look-up tables.

[0036] To properly control the operation of the vehicle 10 and its components (e.g., components of the communication network 19), the controller 32 may include a processor (e.g., a microprocessor) and at least one memory, at least some of which is tangible and non-transitory. The memory may store a controller-executable instruction set, and the processor may execute the controller-executable instruction set stored in the memory. The memory may be a recordable medium or a recordable medium that participates in providing computer-readable data or processing instructions.

[0037] The recordable media can take many forms, including but not limited to non-volatile media and volatile media. Non-volatile media for the controller 32 can include, for example, optical or magnetic disks and other permanent storage. Volatile media can include, for example, dynamic random access memory (DRAM), which can constitute the main memory. The memory of the controller 32 can also include solid-state media, floppy disks, floppy disks, hard disks, magnetic tapes, another magnetic medium, CD-ROMs, DVDs, another optical medium, combinations thereof, and the like.

[0038] The controller executable instruction set may be transmitted via one or more transmission media, including coaxial cables, copper wires, conductive traces, optical fibers, combinations thereof, and the like. For example, the transmission medium may include a dedicated connection between two or more components or a shared connection (e.g., a bus) that couples multiple components together. Signals from or representing the controller executable instruction set may be transmitted in parallel, serially, or a combination thereof.

[0039] Processor 34 may be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), a secondary processor among multiple processors associated with controller 32, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally a device for executing instructions.

[0040] The computer readable storage medium 36 may include volatile and non-volatile storage devices such as in read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the processor 34 is powered off. The computer readable storage medium 36 can be implemented using a variety of suitable storage devices, such as a programmable read-only memory (PROM), an electrical PROM (EPROM), an electrically erasable PROM (EEPROM), flash memory, or other electrical, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable instructions, used by the controller 32 in controlling the vehicle 10.

[0041] The executable instructions may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. When the executable instructions are executed by the processor 34, the processor receives and processes signals from the sensor system 17, executes logic, calculations, methods and / or algorithms for automatically controlling components of the vehicle 10, and generates control signals to the actuator system 18 to automatically control components of the vehicle 10 based on the logic, calculations, methods and / or algorithms. Although Figure 1 1 , a single controller 32 is shown, but embodiments of the vehicle 10 may include more than one controller 32 that communicate via a suitable communication medium or combination of communication mediums and cooperate to route and / or process sensor signals to execute logic, calculations, methods and / or algorithms and generate control signals to automatically control features of the vehicle 10. In some aspects, one or more executable instructions of the controller 32 are embodied in the vehicle control system 13.

[0042] The vehicle 10 may be an autonomous vehicle having a vehicle control system 13 partially or fully incorporated therein. The vehicle control system 13 may be referred to simply as a system. For example, the vehicle 10 may be a vehicle that is automatically controlled to transport passengers from one location to another. In some aspects, the vehicle 10 is a so-called Level 4 or Level 5 automation system. A Level 4 system represents "high automation" and refers to the driving mode-specific performance of the autonomous driving system for various aspects of dynamic driving tasks even when a human driver does not respond appropriately to a request to intervene. A Level 5 system represents "full automation" and refers to the full-time performance of the autonomous driving system for various aspects of dynamic driving tasks under a variety of road and environmental conditions managed by a human operator.

[0043] The sensor system 17 includes one or more sensors 17a, 17b, 17n (i.e., sensing devices) that sense observable conditions external and / or internal to the vehicle 10. The sensors 17a, 17b, 17n communicate with the controller 32 and may include, but are not limited to, one or more accelerometers (e.g., gyroscopes), one or more radio detection and ranging (RADAR) devices, one or more light sensor detection and ranging (LIDAR) devices, one or more global positioning system (GPS) devices, one or more image capture devices 17a (e.g., optical cameras and / or thermal cameras), ultrasonic devices, one or more inertial measurement units 17b, speed sensing devices, pressure sensing devices, temperature sensing devices 17n, humidity sensing devices, voltage sensing devices, current sensing devices, resistance sensing devices, airflow sensing devices, gas sensing devices, actuation sensing devices, and / or other sensing devices.

[0044] The communication network 19 is configured to communicate information with vehicle components and / or other entities 91. Information can be communicated via wireless connections, wired connections, or a combination thereof. The communication system 19 may include an internal communication management module 92, an external communication management module 93, and a bus 94.

[0045] The bus 94 is configured to communicate signals and / or data between the various components of the vehicle 10. In some aspects, the bus 94 is a controller area network (CAN) bus 94. The internal communication management module 92 is configured to manage and / or coordinate signaling between vehicle components via the bus 94.

[0046] The external communication management module 93 is configured to manage and / or coordinate signaling between the vehicle 10 or components on the vehicle 10 and other entities 91. The other entities 91 can be, for example, other vehicles ("V2V" communication), infrastructure ("V2I" communication), remote systems (e.g., Internet-based communication), and / or personal devices (e.g., mobile phones). In some aspects, the external communication management module 93 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communications. However, additional or alternative communication methods, such as dedicated short-range communication (DSRC) channels are also considered to be within the scope of the present disclosure. A DSRC channel refers to a unidirectional or bidirectional short-range to medium-range wireless communication channel designed specifically for automotive use, as well as a set of corresponding protocols and standards. Therefore, the communication system 19 may include one or more antennas and / or transceivers for receiving and / or transmitting signals (e.g., collaborative sensing messages (CSM)). In some aspects, the external communications management module 93 is directly connected to the bus 94 , while in other aspects, the external communications management module 93 is coupled to the bus 94 via, for example, the internal communications management module 92 .

[0047] Figure 2 A schematic topology of a portion 100 of a communication network 19 is shown. The portion 100 depicts a switch cascade 102, including a central switch 104, a plurality of edge switches 106, and a plurality of sensors 17a to 17n. Each of the switches 102, 104 is configured to provide low-latency forwarding and / or routing of data between connected devices. In addition, the switches are configured to buffer and frame queues of data for transmission over the communication network 19 along one or more data paths 110.

[0048] The data paths 110 are independent routes for transmitting data through the network topology 100 and may be multi-mode. For example, one line of the data path 110 may be wired while another line may be wireless. Furthermore, while the data paths 110 are shown as one-to-one connections in the illustrated figure, it is contemplated that one or more data paths 110 may be implemented as shared connections. For example, the data paths 110 may include the CAN bus 94 or an automotive Ethernet communication bus.

[0049] The sensors 17a to 17n are configured to detect one or more physical characteristics. The detected characteristics are encoded for transmission via the communication network 19 for storage and / or processing. Although the illustrated topology 100 includes only sensors 17a to 17n at the network edge switches, it is contemplated that other devices may be coupled or connected to one or more edge switches 106.

[0050] Figure 3A schematic block diagram of a portion 200 of the communication network 19 is shown. The portion 200 shown includes a central switch 104, a plurality of edge switches 106, and a network adaptive traffic shaper 210.

[0051] Each switch 104, 106 includes a plurality of ports 212, a buffer 214, and a scheduler 216. Each port 212 provides a communication connection with a corresponding connected device. The buffer 214 is configured to store received data for retransmission at a predetermined time. The scheduler 216 is configured to manage the flow of packets through the switches 104, 106. The scheduler 216 can modify or reorder the flow of packets to optimize the performance of the communication network 19 and / or the consistency with the desired traffic metric or requirement. For example, the scheduler 216 can discard, delay, or prioritize packets for retransmission based on one or more criteria or network requirements.

[0052] Each switch 104, 106 includes a set of qualities or states that can be changed to affect performance. Switch qualities or states can include, for example, gating time, queue length, priority queues, and packet routing. For example, if an incoming packet is dropped because it cannot be buffered or a received packet is dropped because it cannot be retransmitted by the switch 104, 106, the gating time and / or available time slots for retransmission can be changed. If a class of packets is excessively dropped, the queue length can be increased for the class of packets and / or the queue length can be reduced for one or more other classes of packets. The number of priority queues can be changed to manage the bandwidth available for each queue by, for example, reducing the number of available priority queues so that each remaining queue has more available bandwidth.

[0053] For example, packet routing may be changed if a desired data path becomes too congested such that higher priority packet classes continue to be retransmitted onto the congested data path while lower priority packet classes are retransmitted to another switch 104, 106 in the network topology 200. Once a packet is received by the switch, the packet may be stored, retransmitted toward a destination device, or sent to yet another switch in, for example, a ring topology.

[0054] The switches 104, 106 may include multiple operating modes. Each operating mode may include a set of predetermined quality values ​​or states, a set of well-defined relationships between quality values ​​or states based on known or measurable characteristics, combinations thereof, etc. Advantageously, multiple operating modes may be selected to reduce the configuration space available for the switches 104, 106 and the network 19, thereby optimizing network requirements.

[0055] Each operating mode is configured to provide a desired behavior for the switch and network traffic. In some aspects, the operating modes may include a high traffic mode, an advanced driver assistance system ("ADAS") mode, and an over-the-air ("OTA") mode. The high traffic mode may be configured to adjust network traffic based at least in part on a class of packets. The ADAS mode may be configured to adjust network traffic based at least in part on packets originating from a subset of sensors and / or packets destined for a subset of connected devices. The OTA mode may be configured to adjust network traffic based at least in part on packet density or concentration.

[0056] The switches 104, 106 may change the operating mode or quality of the switches 104, 106 in response to receiving a control signal (e.g., a value, flag, or instruction) received from a management device (i.e., a network adaptive traffic shaper 210, described below), detecting predetermined attributes of received data, determining the state of the data and the switch's cache, a combination thereof, and the like.

[0057] The central switch 104 is coupled to each of the plurality of edge switches 106. In some aspects, the central switch 104 is also configured to manage aspects of the edge switches 106. For example, the central switch 104 can supplement or fully control the setting of the operating mode of the edge switches 106.

[0058] Each edge switch 106 is disposed between the central switch 104 and one or more connected sensors 17. Each edge switch 106 may also be connected to one or more other edge switches 106 in a suitable configuration, such as a ring topology. The edge switches 106 are configured to receive data packets from the corresponding connected sensors 17 using one or more data paths 110 and retransmit the received data packets to at least one connected switch 104, 106 or other device.

[0059] The network adaptive traffic shaper 210 is configured to monitor and manage various aspects of the network 19 and optimize network traffic (i.e., traffic shaping) within the network 19 to meet the time-varying needs or requirements of the communication network 19. The network adaptive traffic shaper 210 includes a global health monitor 220, a network device monitor 222, a switch status monitor 224, and a switch status updater 226.

[0060] The global health monitor 220 is configured to monitor network conditions and compliance with network requirements based on a plurality of switch metrics 230 .

[0061] Each of the plurality of switch metrics 230 (i.e., switch 1 metric through switch n metric) includes information related to a packet class for the corresponding switch 104, 106. The information includes appropriate metrics to determine whether network requirements are being met. In some aspects, the information provides information for each packet class C1 through C2. n jitter, latency, and dropped packets measurements.

[0062] The switch metrics 230 may be obtained from each switch 104 , 106 , from other devices in the network 19 , or determined by the health monitor 220 .

[0063] The network device monitor 222 is configured to monitor information related to devices connected to the network 19. The information may include network topology and / or device identity, location, operating status, configuration, etc. For example, the network device monitor 222 may obtain operating information for each switch 104, 106 within the communication network 19, including, for example, available switch operating modes, employed operating modes, settings of individual switch qualities or statuses, flags, errors, and associated times.

[0064] The switch state module 224 is configured to correlate the effects of changes to the configuration of the communication network 19 on network performance given known vehicle operating conditions, and provide instructions to the switch state updater 226 to change one or more switch states.

[0065] The switch state module 224 includes a suitable reinforcement learning (RL) algorithm, such as a state-action-reward-state-action model. In some aspects, the inputs to the learning model include a set of operating modes for each switch 104, 106, switch metrics 230, and system requirements. Advantageously, this smaller set of well-defined inputs reduces the resources required to operate the reinforcement or switch state module.

[0066] Additionally or alternatively, the inputs may include switch qualities or status of each of the switches 104, 106. Advantageously, this granularity provides robust control over network traffic shaping to optimize handling of a larger set of environmental events.

[0067] The switch state updater 226 is configured to receive action instructions from the switch state module 224 and send control signals to the switches 104, 106 to implement actions, such as transitioning the switches from a first operating mode to a second operating mode, thereby adjusting or changing switch state or quality.

[0068] Although the illustrated network adaptive traffic shaper 110 is depicted as a stand-alone block, it is contemplated that the network adaptive traffic shaper or components thereof may be implemented on the central switch 106 or other suitable device.

[0069] Reference now Figure 4 , a flow chart of a method 400 for training an RL algorithm in a pre-production environment is provided. It should be understood that to train the RL algorithm, a computer system other than the vehicle 12 and the controller 32 may be used, including, for example, a server computer, a cloud-based server farm, etc. Once the RL algorithm has been trained, the RL algorithm is transmitted to the controller 32 of the vehicle 12 so that the RL algorithm can be retrieved by the controller 32 of the vehicle 12 through the network adaptive traffic shaper 210 to implement changes to network performance.

[0070] In addition, the RL algorithm provides an RL agent that can take actions in the environment (i.e., the communication network 19). The RL algorithm takes into account a variety of factors, such as packet priority, dropped packets, latency, jitter, throughput, and other network traffic parameters. The group of possible actions that the switch state module 224 can take is called an action space. The action space can include, for example, changes: gating time, queue length, routing information, priority queues, etc. As will be discussed in further detail below, for each action taken, the RL agent receives an appropriate reward. Based on the observation space, the actions taken in the action space, and the calculated rewards, the RL algorithm is trained to maximize the rewards received. It should be understood that various additional training methods, such as supervised learning, semi-supervised learning, and unsupervised learning, are within the scope of the present disclosure. In addition, RL algorithms that use policy functions rather than machine learning models are also within the scope of the present disclosure.

[0071] The method 400 starts at block 402 and proceeds to blocks 404 and 406. At block 404, a traffic metric is determined for each of a plurality of changes in the state of a switch 104, 106 in the environment. This step is repeated so that the traffic metric for each of the plurality of changes in the switch state is continuously updated to account for changes in network traffic.

[0072] At block 406, a unique environment is generated. The environment includes multiple changes in ECUs, main switches and edge switches, a communication network, and switch states. In an exemplary embodiment, the environment also includes multiple traffic metrics, each of the multiple traffic metrics showing different traffic parameters. In an exemplary embodiment, multiple network parameters of network traffic in the environment are changed, such as an operating mode. After block 406, method 400 proceeds to block 408.

[0073] At block 408, the network adaptive traffic shaper 210 takes an action within the action space. In a non-limiting example, the action space includes changing the state of the switch: gating time, queue length, routing information, priority queue, etc. In an exemplary embodiment, the action is determined by the RL algorithm based on the traffic metrics of each of the multiple switch states and various additional factors such as traffic demand. After block 408, the method 400 proceeds to block 410.

[0074] At box 410, a total reward is calculated based on the action taken at box 408. In an exemplary embodiment, the total reward is calculated based on other factors (e.g., packet priority, dropped packets, latency, jitter, throughput, etc.) so that the traffic shaping algorithm is trained to effectively address various network traffic environments. For example, if the action taken at box 408 causes the packet to not meet the requirements, a large negative reward is provided. It should be understood that the purpose of calculating the first reward and the second reward is to train the RL algorithm to handle packets that do not meet the network requirements. In order to train the RL algorithm to effectively address various network traffic environments or patterns, additional rewards may be included in the total reward. After box 410, method 400 proceeds to box 412.

[0075] At block 412, to train the RL algorithm, the decision function of the RL algorithm is updated based on the calculated reward. In an exemplary embodiment, the parameters of the traffic shaping algorithm (e.g., switch state, etc.) are modified based on the reward calculated at block 410 so that the RL algorithm learns to maximize the total reward. In an exemplary embodiment where the RL algorithm uses a policy function, the RL policy function is updated. After block 412, method 400 proceeds to block 414.

[0076] At block 414, a predetermined stopping point is evaluated. In an exemplary embodiment, the predetermined stopping point is evaluated based on at least one of a plurality of factors, including, for example, the average total reward collected over time and the convergence of the RL agent. If the predetermined stopping point is reached, the method 400 proceeds to block 416. If the predetermined stopping point is not reached, the method 400 returns to block 408, whereby further actions are taken and the decision function of the RL algorithm is further updated.

[0077] At block 416, a predetermined training stopping point is evaluated. In an exemplary embodiment, the predetermined training stopping point is evaluated based on at least one of a plurality of factors, including, for example, the number of iterations performed and the convergence of the total reward received during each iteration. If the predetermined training stopping point is reached, the method 400 proceeds to enter a standby state at block 418. If the predetermined stopping point is not reached, the method 400 returns to block 404, thereby generating a new, unique environment.

[0078] The foregoing detailed description is merely exemplary in nature and is not intended to limit the present application and uses. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding section or the foregoing detailed description.

[0079] As used herein, the term "module" refers to one or a combination of hardware, software, firmware, electronic control components, processing logic and / or processor devices, including but not limited to: application-specific integrated circuits (ASICs), electronic circuits, processors (shared processors, dedicated processors, or processor groups) and memory that executes one or more software or firmware programs, combinational logic circuits, and / or other components that provide the described functionality.

[0080] Embodiments of the present disclosure may be described herein according to functions and / or logic block components and various processing steps. It should be understood that such block components may be implemented by one or more hardware, software and / or firmware components configured to perform a specified function. For example, embodiments of the present disclosure may employ various integrated circuit components (e.g., memory elements, digital signal processing elements, logic elements, lookup tables, etc.), which may perform various functions under the control of one or more microprocessors or other control devices. In addition, those skilled in the art will appreciate that embodiments of the present disclosure may be practiced in conjunction with one or more systems, and the systems described herein are merely exemplary embodiments of the present disclosure.

[0081] For the sake of brevity, technologies related to signal processing, data fusion, signaling, control, and other functional aspects of the system (as well as the individual operating components of the system) may not be described in detail. In addition, the connecting lines shown in the various figures are intended to represent example functional relationships and / or physical connections between the various elements. It should be noted that alternative or additional functional relationships or physical connections may exist in embodiments of the present disclosure.

[0082] As will be appreciated by those skilled in the art, the present disclosure is susceptible to various modifications and alternative forms, and some representative embodiments have been illustrated by way of example in the accompanying drawings and described in detail above. However, it should be understood that the novel aspects of the present disclosure are not limited to the specific forms shown in the accompanying drawings. On the contrary, the present disclosure encompasses modifications, equivalents, combinations, sub-combinations, permutations, groupings, and substitutions that fall within the scope and spirit of the present disclosure and are defined by the appended claims. For example, the present disclosure is not limited to automotive applications, and in fact, one of ordinary skill in the art will appreciate that the teachings disclosed herein may be applied to applications other than automotive.

[0083] As used herein, unless the context clearly dictates otherwise: the words “and” and “or” shall be considered both conjunctions and transitional conjunctions, unless the context clearly dictates otherwise; the word “all” means “any and all”; the word “any” means “any and all”; the word “include” means “including but not limited to”; the singular forms “a”, “an” and “the” include plural referents and vice versa.

[0084] Numerical values ​​of parameters (such as quantities or conditions) in this specification, unless otherwise explicitly or clearly indicated by the context (including the appended claims), should be understood as being modified by the term "about", whether or not "about" actually appears before the numerical value. The numerical parameters set forth herein and in the appended claims are approximate values, which may vary depending on the desired properties sought to be obtained by the present disclosure. At the very least, and without attempting to limit the application of the doctrine of equivalents to the scope of the claims, each numerical parameter should at least be interpreted in light of the number of reported significant digits and by applying ordinary rounding techniques.

[0085] Approximate words such as "approximately," "about," "substantially," etc. may be used herein in a sense such as "at, close to, or nearly at," "within 0-10% of," or "within an acceptable manufacturing tolerance," or logical combinations thereof.

[0086] While the best modes for carrying out the disclosure have been described in detail, those familiar with the art to which this disclosure relates will recognize various alternative designs and embodiments for practicing the disclosure within the scope of the appended claims.

Claims

1. A vehicle comprising: a communication data network for transmitting a plurality of data, wherein the data comprises a plurality of data packets; a plurality of electronic control units ECU, wherein each of the plurality of ECUs is in communication with the communication data network for sending the plurality of data packets to the communication data network and receiving the plurality of data packets from the communication data network; a plurality of sensors in electrical communication with the ECU, each sensor providing a sensor signal to the ECU; and A network device, the network device communicating with the communication data network, for controlling a plurality of data packets on the communication data network, wherein the network device is configured to execute program code to perform the following operations: monitoring a plurality of data flow metrics for a plurality of data packets on the communications data network; comparing the plurality of data traffic metrics to a plurality of data network requirements; determining whether the plurality of data traffic metrics differ from the plurality of data network requirements; and A data traffic shaper is adjusted based on determining that the plurality of traffic metrics differ from the plurality of data network requirements.

2. The vehicle according to claim 1, wherein: The network device is a network switch, wherein the network switch has a plurality of states adjustable by the data traffic shaper to control data traffic on the communication data network.

3. The vehicle according to claim 2, wherein: The network switch has a first operating mode defined by a first plurality of states of the plurality of states.

4. The vehicle according to claim 3, wherein: The network switch has a second operating mode defined by a second plurality of states of the plurality of states of the network switch.

5. The vehicle according to claim 4, wherein: The network switch has a third operating mode defined by a third plurality of states of the plurality of states of the network switch.

6. The vehicle according to claim 3, wherein: Adjusting the data traffic shaper also includes placing the network switch in the first operating mode based at least in part on the classification of the plurality of data packets.

7. The vehicle according to claim 4, wherein: Adjusting the data traffic shaper also includes placing the network switch in the second operating mode based at least in part on a plurality of data packets originating from a first plurality of sensors among the plurality of sensors.

8. The vehicle according to claim 5, wherein: Adjusting the data traffic shaper further includes placing the network switch in the third operating mode based at least in part on a packet density of the plurality of packets.

9. The vehicle according to claim 5, wherein: The data traffic shaper also includes a reinforcement learning algorithm for determining a first plurality of states among the plurality of states defining the first operating mode, a second plurality of states among the plurality of states defining the second operating mode, and a third plurality of states among the plurality of states defining the third operating mode.

10. A system comprising: a communication data network for transmitting a plurality of data, wherein the plurality of data comprises a plurality of data packets; and A network device, the network device communicating with the communication data network, for controlling a plurality of data packets transmitted on the communication data network, wherein the network device is configured to execute program code to perform the following operations: monitoring a plurality of data flow metrics for a plurality of data packets transmitted over the communications data network; comparing the plurality of data traffic metrics to a plurality of data network requirements; determining whether the plurality of data traffic metrics differ from the plurality of data network requirements; and A data traffic shaper is adjusted based on determining that the plurality of traffic metrics differ from the plurality of data network requirements.

Images