Edge-enhanced incremental learning for autonomous vehicles

Abstract

A system includes a computer including a processor and a memory. The memory includes instructions causing the processor to be programmed to determine, via a trained neural network model, a route to be traversed by a vehicle based on vehicle sensor data and update the trained neural network model based on data received from at least one of an edge computing device or an infrastructure (V2I) device.

Description

Technical Field

[0001] This disclosure relates to incrementally updating a trained neural network at an autonomous vehicle based on data received from edge computing devices, infrastructure devices, and / or other vehicles. Background Technology

[0002] Cloud computing and edge computing can be used to support various V2X use cases, such as vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-pedestrian (V2P). For example, edge computing devices can employ sensors, such as cameras, to monitor the environment in which vehicles pass. These edge computing devices can broadcast data, representing detected objects, to vehicles nearby. Summary of the Invention

[0003] A system includes a computer, which includes a processor and a memory. The memory includes instructions that program the processor to: determine a route to be taken by a vehicle based on vehicle sensor data via a trained neural network model, and update the trained neural network model based on data received from at least one of edge computing devices or vehicle-to-infrastructure (V2I) devices.

[0004] Among other features, the processor is also programmed to update the trained neural network model based on data received from at least one other vehicle.

[0005] Among other features, the processor is also programmed to receive the trained neural network model via over-the-air (OTA) updates.

[0006] Among other features, the processor is also programmed to determine an alternative route via the updated, trained neural network model.

[0007] Among other features, the processor is also programmed to cause the vehicle to travel along the route.

[0008] Among other features, the infrastructure equipment includes roadside equipment.

[0009] Among other features, the edge computing device includes an edge server.

[0010] Among other features, the processor is also programmed to store a data structure representing the weight changes between the trained neural network model and the updated trained neural network model.

[0011] Among other features, the processor is also programmed to upload the data structure to the specification platform.

[0012] Among other features, the processor is also programmed to receive a second trained neural network model from the specification platform, wherein the second trained neural model includes updated weights based on data stored within the data structure.

[0013] One approach includes: determining a route to be taken by a vehicle based on vehicle sensor data via a trained neural network model, and updating the trained neural network model based on data received from at least one of an edge computing device or a vehicle-to-infrastructure (V2I) device.

[0014] Among other features, the method also includes updating the trained neural network model based on data received from at least one other vehicle.

[0015] Among other features, the method also includes receiving the trained neural network model via over-the-air (OTA) updates.

[0016] Among other features, the method also includes determining an alternative route via the updated, trained neural network model.

[0017] Among other features, the method also includes directing the vehicle along the route.

[0018] Among other features, the infrastructure equipment includes roadside equipment.

[0019] Among other features, the edge computing device includes an edge server.

[0020] Among other features, the method also includes a stored data structure that represents the weight changes between the trained neural network model and the updated trained neural network model.

[0021] Among other features, the method also includes uploading the data structure to a standardized platform.

[0022] Among other features, the method also includes receiving a second trained neural network model from the specification platform, wherein the second trained neural model includes updated weights based on data stored within the data structure.

[0023] Further applicability will become apparent from the description provided herein. It should be understood that the specification and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0024] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0025] Figure 1 It is a block diagram of an example system including a vehicle;

[0026] Figure 2 This is a block diagram of an example computing device;

[0027] Figure 3 It is a block diagram of an example environment that includes autonomous vehicles, edge computing devices, infrastructure equipment, and standardized platforms;

[0028] Figure 4A It is a block diagram of an example environment that includes autonomous vehicles, other vehicles, edge computing devices, and infrastructure devices;

[0029] Figure 4B It is a block diagram of an example environment including autonomous vehicles, another vehicle, edge computing devices, and infrastructure equipment; and

[0030] Figure 5 This is a flowchart illustrating an example process for incrementally updating a trained neural network. Detailed Implementation

[0031] The following description is merely exemplary in nature and is not intended to limit this disclosure, its application, or its uses.

[0032] Some vehicles, such as autonomous vehicles, utilize cloud and edge computing services to communicate with other vehicles, infrastructure, and pedestrians. Cloud and edge computing can enhance vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-pedestrian (V2P) use cases (collectively, V2X), thereby improving vehicle and pedestrian safety. Cloud and edge computing allow vehicles to communicate with other vehicles, infrastructure, and / or pedestrians using wireless communication technologies, including, but not limited to, cellular, wireless communication technologies. IEEE 802.11, Dedicated Short Range Communication (DSRC), Ultra-Wideband (UWB), and / or Wide Area Network (WAN).

[0033] In intelligent transportation systems (ITS), roadside devices can be located along one or more roads to capture traffic data generated by vehicles and provide information (such as traffic advisories) from infrastructure (e.g., roadside devices) to the cloud and edge, and subsequently to vehicles, to inform drivers and / or vehicles about safety, mobility, and / or environmental conditions. In some instances, roadside devices are located within signalized intersections to provide information to vehicles traveling near the signalized intersections.

[0034] Figure 1 This is a block diagram of an example vehicle system 100. System 100 includes a vehicle 105, which may include land vehicles such as cars and trucks, air vehicles, and / or water vehicles. Vehicle 105 includes a computer 110, vehicle sensors 115, actuators 120 for actuating various vehicle components 125, and a vehicle communication module 130. Communication module 130 allows computer 110 to communicate with server 145 via network 135.

[0035] Computer 110 can operate vehicle 105 in automatic, semi-automatic, or non-automatic (manual) mode. For the purposes of this disclosure, automatic mode is defined as a mode in which each of the propulsion, braking, and steering operations of vehicle 105 is controlled by computer 110; in semi-automatic mode, computer 110 controls one or two of the propulsion, braking, and steering operations of vehicle 105; and in non-automatic mode, a human operator controls each of the propulsion, braking, and steering operations of vehicle 105.

[0036] Computer 110 may include programming to operate one or more of the following operations of vehicle 105: braking, propulsion (e.g., controlling the vehicle's acceleration by controlling one or more of an internal combustion engine, electric motor, hybrid engine, etc.), steering, climate control, interior and / or exterior lights, etc., and programming to determine whether and when computer 110 (as opposed to a human operator) controls these operations. Additionally, computer 110 may be programmed to determine whether and when a human operator controls these operations.

[0037] Computer 110 may include one or more processors (e.g., included in an electronic control unit (ECU) or similar unit within vehicle 105 for monitoring and / or controlling various vehicle components 125, such as power controllers, brake controllers, steering controllers, etc.) or be communicatively connected to one or more processors via vehicle 105 communication module 130 as further described below. Further, computer 110 may communicate with a navigation system using a Global Positioning System (GPS) via vehicle 105 communication module 130. As an example, computer 110 may request and receive location data from vehicle 105. The location data may be in a known form, such as geographic coordinates (latitude and longitude coordinates).

[0038] Computer 110 is typically configured to communicate on communication module 130 of vehicle 105, and also to wired and / or wireless networks (such as buses in vehicle 105, such as controller area network (CAN)) and / or other wired and / or wireless mechanisms within vehicle 105.

[0039] Via the vehicle 105 communication network, the computer 110 can send messages to and / or receive messages from various devices within the vehicle 105 (e.g., vehicle sensors 115, actuators 120, vehicle components 125, human-machine interface (HMI), etc.). Alternatively or additionally, where the computer 110 actually comprises multiple devices, the vehicle 105 communication network can be used for communication between devices represented herein as computer 110. Further, as mentioned below, various controllers and / or vehicle sensors 115 can provide data to the computer 110. The vehicle 105 communication network may include one or more gateway modules that provide interoperability between various networks and devices (such as protocol converters, impedance matching devices, rate converters, etc.) within the vehicle 105.

[0040] Vehicle sensors 115 may include various devices, such as those known to provide data to computer 110. For example, vehicle sensors 115 may include one or more light-detecting and ranging (LiDAR) sensors 115 disposed on top of vehicle 105, behind the windshield of vehicle 105, or around vehicle 105, providing information on the relative position, size, and shape of objects and / or the conditions around vehicle 105. As another example, one or more radar sensors 115 fixed to the bumper of vehicle 105 may provide data to provide and measure distance, speed, etc., of objects (potentially including a second vehicle 106) relative to vehicle 105. Vehicle sensors 115 may also include one or more camera sensors 115, such as forward-looking, side-looking, and rear-looking sensors, to provide images of the field of view from inside and / or outside vehicle 105.

[0041] The actuator 120 of the vehicle 105 is implemented via circuits, chips, motors, or other electronic and / or mechanical components that can actuate various vehicle subsystems according to known and appropriate control signals. The actuator 120 can be used in the control unit 125, including braking, accelerating, and steering the vehicle 105.

[0042] In the context of this disclosure, vehicle component 125 is one or more hardware components adapted to perform mechanical or electromechanical functions or operations (e.g., moving vehicle 105, slowing or stopping vehicle 105, steering vehicle 105, etc.). Non-limiting examples of component 125 include propulsion components (which include, for example, internal combustion engines and / or electric motors), transmission components, steering components (which may include, for example, one or more steering components such as a steering wheel, steering gears, etc.), braking components (as described below), parking assist components, adaptive cruise control components, adaptive steering components, movable seats, etc.

[0043] Additionally, computer 110 can be configured to communicate with devices external to vehicle 105 via vehicle-to-vehicle communication module or interface 130, for example, via vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2I) wireless communication to another vehicle, to a remote server 145 (such as an edge server) (typically via network 135). Module 130 may include one or more mechanisms that computer 110 can use for communication, including wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or multiple topologies when using multiple communication mechanisms) and any desired combination thereof. Exemplary communications provided via module 130 include cellular, IEEE 802.11, Dedicated Short Range Communications (DSRC), and / or Wide Area Networks (WAN) (including the Internet) are used to provide data communication services.

[0044] Network 135 can be one or more of various wired or wireless communication mechanisms, including wired (e.g., cable and fiber optic) and / or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or multiple topologies when using multiple communication mechanisms) and any desired combination thereof. Exemplary communication networks include wireless communication networks (e.g., using Bluetooth, Bluetooth Low Energy (BLE), IEEE 802.11, vehicle-to-vehicle (V2V) communication such as Dedicated Short Range Communication (DSRC), etc.), local area networks (LANs), and / or wide area networks (WANs) (including the Internet) to provide data communication services.

[0045] System 100 also includes infrastructure equipment 150, which can communicate with server 145 and vehicle 105 via communication network 135. Although only a single infrastructure equipment 150 is shown, it should be understood that system 100 may include multiple infrastructure equipment 150 deployed throughout the traffic environment accessible by vehicle 105. Infrastructure equipment 150 may include roadside devices, traffic lights, cameras attached to structures, or any other vehicle-to-infrastructure (V2I) equipment.

[0046] System 100 also includes a specification platform 155 that provides distributed learning capabilities to system 100. In an example implementation, specification platform 155 communicates with server 145 (e.g., an edge computing device) via communication network 135. The components and functions of the specification platform will be described in more detail below.

[0047] Figure 2 An example computing device 200 configured to perform one or more processes described herein is illustrated. As shown, the computing device may include a processor 205, a memory 210, a storage device 215, an I / O interface 220, and a communication interface 225. Furthermore, the computing device 200 may include input devices such as a touchscreen, a mouse, a keyboard, etc. In some embodiments, the computing device 200 may include a... Figure 2 The number of components shown is less or more.

[0048] In a particular implementation, processor(s) 205 includes hardware for executing instructions, such as those that constitute a computer program. By way of example and not limitation, in order to execute instructions, processor(s) 205 may retrieve (or fetch) instructions from internal registers, internal caches, memory 210, or storage device 215, and decode and execute those instructions.

[0049] Computing device 200 includes memory 210 connected to one or more processors 205. Memory 210 can be used to store data, metadata, and programs executed by the processors. Memory 210 may include one or more volatile and non-volatile memories, such as random-access memory (RAM), read-only memory (ROM), solid-state disk (SSD), flash memory, phase-change memory (PCM), or other types of data storage devices. Memory 210 may be internal memory or distributed memory.

[0050] Computing device 200 includes storage device 215, which includes storage means for storing data or instructions. By way of example and not limitation, storage device 215 may include the non-transitory storage media described above. Storage device 215 may include a hard disk drive (HDD), flash memory, a universal serial bus (USB) drive, or a combination of these or other storage devices.

[0051] The computing device 200 also includes one or more input or output (“I / O”) devices / interfaces 220, providing the one or more I / O devices / interfaces to allow a user to provide input (such as user strokes) to the computing device 200, receive output from the computing device 200, and otherwise transmit data to and from the computing device 200. These I / O devices / interfaces 220 may include a mouse, a small keyboard or keypad, a touchscreen, a camera, an optical scanner, a network interface, a modem, other known I / O devices, or combinations of such I / O devices / interfaces 220. The touchscreen can be activated with a writing device or a finger.

[0052] I / O device / interface 220 may include one or more devices for presenting output to a user, including but not limited to a graphics engine, a display (e.g., a screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In some embodiments, device / interface 220 is configured to provide graphical data to the display for presentation to the user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content that may serve a particular embodiment.

[0053] The computing device 200 may also include a communication interface 225. The communication interface 225 may include hardware, software, or both. The communication interface 225 provides one or more interfaces for communication (such as, for example, packet-based communication) between the computing device and one or more other computing devices 200 or one or more networks. By way of example and not limitation, the communication interface 225 may include a network interface controller (NIC) or network adapter for communicating with Ethernet or other wired networks, or a wireless NIC (WNIC) or wireless adapter for communicating with wireless networks such as Wi-Fi. The computing device 200 may also include a bus 230. The bus 230 may include hardware, software, or both for connecting components of the computing device 200 to each other.

[0054] Figure 3 An example environment 300 is illustrated, which includes a vehicle 105 communicating with edge computing device 145 and / or infrastructure device 150. Edge computing device 145 and / or infrastructure device 150 may also communicate with a specification platform 155. As discussed below, specification platform 155 may retrain one or more neural network models based on records uploaded from vehicle 105 to specification platform 155. Records may include data representing incremental changes to the neural network models during operation of vehicle 105.

[0055] Referring to specification platform 155, specification platform 155 can be designed to be modular, allowing certain software components to be swapped in or out as needed. This allows specification platform 155 to be easily and / or quickly reconfigured for different applications. In some implementations, specification platform 155 can receive and / or transmit information to one or more client edge servers 145 and / or one or more infrastructure devices 150.

[0056] In some implementations, as shown in the figures, the specification platform 155 may be hosted in a cloud computing environment 305, a local computing environment, a hybrid (e.g., local and cloud-based) computing environment, etc. It is worth noting that although the implementations described herein describe the specification platform 155 as hosted in a cloud computing environment, in some implementations, the specification platform 155 may not be cloud-based (i.e., it may be implemented outside of a cloud computing environment) or may be a partially cloud-based computing environment.

[0057] The cloud computing environment 305 includes the environment of the managed specification platform 155. The cloud computing environment 305 can provide computing, software, data access, storage, and other services without requiring end users to know the physical location and configuration of one or more systems and / or one or more devices of the managed specification platform 155. As shown in the figure, the cloud computing environment may include a set of computing resources 310 (collectively referred to as "computing resources 124" and individually referred to as "computing resources 310").

[0058] Computing resource 310 includes one or more personal computers, workstations, mainframes, or other types of computing and / or communication devices. In some embodiments, computing resource 310 may host specification platform 155. Cloud resources may include computing instances running in computing resource 310, storage devices provided in computing resource 310, data transmission devices provided by computing resource 310, etc. In some embodiments, computing resource 310 may communicate with other computing resources 310 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0059] Figure 4A The illustration depicts an example environment 400, which includes multiple vehicles 105-1 to 105-5 traveling along road 405. Environment 400 also includes an edge computing device 145 and / or infrastructure device 150 that broadcasts data indicating information about environment 400. For example, the edge computing device 145 and / or infrastructure device 150 may broadcast data indicating information about other vehicles 105-2 to 105-5 that also travel along road 405.

[0060] Within environment 400, vehicle 105-1 includes an autonomous vehicle. Each of vehicles 105-1 to 105-5 may include at least a semi-autonomous vehicle. In other words, each of vehicles 105-1 to 105-5 may include one or more neural network models to assist vehicles 105-1 to 105-5 in performing semi-autonomous vehicle actions. The computer 110 of each vehicle 105-1 to 105-5 uses the neural network model to perform the semi-autonomous vehicle actions. In an example implementation, the neural network model may provide at least perception and / or route planning functions based on sensor data.

[0061] In this context, vehicle 105-1 (and vehicles 105-2 to 105-5) may initially receive the neural network model offline (i.e., by the vehicle manufacturer) via a suitable over-the-air (OTA) service through a secure communication channel, such as during vehicle maintenance. Reference Figure 4A The vehicle 105-1 can use a neural network model to perceive the surrounding environment and plan a route based on the perceived environment.

[0062] The computer 110 of the vehicle 105-1 can also receive data broadcast from the edge computing device 145 and / or infrastructure device 150, and a neural network model running on the computer 110 can incorporate this data for planning. For example, the computer 110 can incorporate appropriate data fusion techniques when running the neural network model.

[0063] Figure 4B The illustration shows environment 400, in which the neural network model of vehicle 105-1 perceives vehicle 105-2 solely based on sensor data generated by sensor 115 of vehicle 105-1. Thus, as discussed below, the neural network model of vehicle 105-1 can perform incremental learning based on data broadcast from edge computing device 145 and / or infrastructure device 150, as well as sensor data transmitted from other vehicles 105-2 to 105-5.

[0064] Figure 5 An example process 500 for incrementally updating a trained neural network is illustrated. The boxes of process 500 can be run by a computer 110 of an autonomous vehicle (i.e., vehicle 105-1). Within process 500, computer 110 receives a trained neural network model from specification platform 155 via over-the-air (OTA) upgrade. At box 505, the trained neural network model generates a route to be taken based on sensor data received from sensor 115.

[0065] At block 510, computer 110 receives data from edge computing device 145 and / or infrastructure device 150. The data may include information representing sensed objects (such as vehicles and / or obstacles) detected by sensors associated with edge computing device 145 and / or infrastructure device 150. The data may also include suggested routes generated by neural network models native to edge computing device 145 and / or infrastructure device 150. In some embodiments, computer 110 may receive data transmitted from other vehicles 105-2 to 105-5 near the autonomous vehicle. This data may include sensed objects detected by the other vehicles 105-2 to 105-5. In some instances, computer 110 integrates the data received at blocks 510, 515 for comparison purposes.

[0066] At box 520, computer 110 determines whether the output from the neural network model (i.e., the planned route) differs from data provided by other vehicles 105-2 to 105-5 and / or edge computing devices 145 and / or infrastructure devices 150. Computer 110 can determine whether the neural network model differs by comparing the accuracy of the neural network model output using sensor data from sensor 115 with the accuracy of the neural network model output using sensor data from sensor 115 and data received at boxes 510, 515.

[0067] If the accuracy difference exceeds a predetermined accuracy amount, computer 110 can locally update the neural network model at box 525. During the local update process, computer 110 can use data from boxes 510 and 515 to locally update one or more weights of the neural network model. The weights can be updated via appropriate backpropagation techniques, etc. Computer 110 can also generate and store records (i.e., data structures) that track incremental neural network model updates.

[0068] At box 530, computer 110 can upload the record to specification platform 155. Specification platform 155 can retrain or merge neural network models using the data stored in the record according to the following equation:

[0069]

[0070] Where N represents the updated neural network model, and M' represents the neural network model retained on the standard platform 155. The weight update operator is represented by Σ, which represents the sum of weight operations applied to the differential weights between the neural network models from i to n. Weight differences can be maintained in records, which include the weight differences for each incrementally updated neural network model. At box 535, computer 110 receives neural network model N from specification platform 155. Then, process 500 ends.

[0071] It should be understood that the vehicle 105-1 can operate in different traffic environments. For example, the vehicle 105-1 can operate between a first urban environment and a second urban environment, and the vehicle 105-1 may include a neural network model corresponding to the first urban environment. In these instances, the traffic environment and / or traffic behavior may differ significantly. However, the computer 110 may determine that when traversing the second urban environment, the perception accuracy of the neural network model is greater than a predetermined perception accuracy. In these instances, the computer 110 may choose not to update or incorporate the neural network model associated with the second urban environment.

[0072] Conversely, computer 110 can determine that the neural network model used for perception purposes may not be suitable for a particular environment. In these instances, computer 110 can switch vehicle 105 from automatic operation mode to manual operation mode when the difference in accuracy of the neural network model exceeds imprecise parameters. Imprecise parameters may include predetermined values ​​set through empirical analysis based on various environments encountered by the automated vehicle.

[0073] The descriptions in this disclosure are merely exemplary in nature, and any changes that do not depart from the spirit and scope of this disclosure are intended to fall within its scope. Such changes should not be considered as departing from the spirit and scope of this disclosure.

[0074] Typically, the described computing system and / or device may employ any of a variety of computer operating systems, including, but not limited to, the following versions and / or variants: Microsoft Operating system, Microsoft Operating systems, Unix operating systems (for example, those released by Oracle Corporation, located on the Redwood Coast of California). Operating systems include: AIX UNIX (published by International Business Machines in Armonk, New York), Linux; Mac OSX and iOS (published by Apple Inc. in Cupertino, California); BlackBerry OS (published by BlackBerry, Ltd. in Waterloo, Canada); Android (developed by Google and the Open Handset Alliance); and infotainment systems provided by QNX software systems. CAR platform. Examples of computing devices include, but are not limited to, in-vehicle computers, computer workstations, servers, desktop computers, laptops, handheld computers, or other computing systems and / or devices.

[0075] Computers and computing devices typically include computer-executable instructions, which can be executed by one or more computing devices (such as those listed above). Computer-executable instructions can be compiled or interpreted from computer programs created using various programming languages ​​and / or technologies, including but not limited to Java, alone or in combination. TM Languages ​​such as C, C++, Matlab, Simulink, Stateflow, Visual Basic, JavaScript, Perl, and HTML are used. Some of these applications can be compiled and run on virtual machines such as the Java Virtual Machine and the Dalvik Virtual Machine. Typically, a processor (e.g., a microprocessor) receives instructions from memory, computer-readable media, etc., and executes those instructions to perform one or more processes, including one or more processes described herein. Such instructions and other data can be stored and transferred using various computer-readable media. Files in computing devices are typically collections of data stored on computer-readable media such as storage media and random access memory.

[0076] Memory can include computer-readable media (also known as processor-readable media), which includes any non-transitory (e.g., tangible) medium involved in providing data (e.g., instructions) that can be read by a computer (e.g., by the computer's processor). Such media can take many forms, including but not limited to non-volatile and volatile media. Non-volatile media can include, for example, optical discs or magnetic disks, and other permanent storage devices. Volatile media can include, for example, dynamic random access memory (DRAM), which typically constitutes main memory. Such instructions can be transmitted via one or more transmission media, including coaxial cables, copper wires, and optical fibers, including wires containing the system bus of the processor connected to the ECU. Common forms of computer-readable media include, for example, floppy disks, hard disks, magnetic tape, any other magnetic media, CD-ROMs, DVDs, any other optical media, punched cards, paper tape, any other physical media with a perforated pattern, RAM, PROMs, EPROMs, flash EEPROMs, any other memory chips or cassette tapes, or any other media from which a computer can read them.

[0077] The databases, data repositories, or other data stores described in this document can include various mechanisms for storing, accessing, and retrieving a wide range of data, including hierarchical databases, a set of files in a file system, application databases in proprietary formats, relational database management systems (RDBMS), and so on. Each such data store is typically contained on a computing device employing a computer operating system such as one of the computer operating systems mentioned above, and is accessed via a network in any one or more of various ways. File systems can be accessed from the computer operating system and can include files stored in various formats. In addition to the languages ​​used to create, store, edit, and run the stored programs, RDBMS typically employs a structured query language (SQL), such as PL / SQL mentioned above.

[0078] In some examples, system elements may be implemented as computer-readable instructions (e.g., software) on one or more computing devices (e.g., servers, personal computers, etc.) stored on an associated computer-readable medium (e.g., disks, storage, etc.). A computer program product may include such instructions stored on a computer-readable medium for performing the functions described herein.

[0079] In this application, the term "module" or "controller" may be replaced by the term "circuit" as defined below. The term "module" may refer to, be part of, or include the following components: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuits; digital, analog, or mixed-signal analog / digital integrated circuits; combinational logic circuits; field-programmable gate arrays (FPGAs); processor circuitry (shared, dedicated, or grouped) that executes code; memory circuitry (shared, dedicated, or grouped) that stores code executed by the processor circuitry; other suitable hardware components that provide the described functionality; or combinations of some or all of the above, such as in a system-on-a-chip.

[0080] A module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that connect to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module disclosed herein may be distributed across multiple modules connected via the interface circuits. For example, multiple modules may allow for load balancing. In another example, a server (also referred to as a remote or cloud) module may perform some functions on behalf of a client module.

[0081] Regarding the media, processes, systems, methods, heuristics, etc., described herein, it should be understood that although the steps of such processes are described as occurring according to an ordered sequence, such processes can be practiced using steps performed in a different order than those described herein. It should also be understood that some steps may be performed simultaneously, other steps may be added, or some steps described herein may be omitted. In other words, the process descriptions herein are provided for the purpose of illustrating certain embodiments and should not be construed as limiting the claims in any way.

[0082] Therefore, it should be understood that the above description is illustrative and not restrictive. Many embodiments and applications beyond the examples provided will be apparent to those skilled in the art upon reading the above description. The scope of the invention should not be determined by reference to the above description, but rather by reference to the appended claims and the full scope of their equivalents. It is foreseeable and anticipated that future developments will occur in the art discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In conclusion, it should be understood that the invention is capable of modifications and variations and is limited only by the appended claims.

[0083] All terms used in the claims are intended to be given their generic and ordinary meaning as understood by those skilled in the art, unless expressly indicated otherwise herein. In particular, the use of singular articles such as “a,” “the,” “the,” etc., should be interpreted as referring to one or more of the indicated elements, unless the claims set forth an express limitation to the contrary.

Claims

1. An edge-enhanced incremental learning system for an autonomous vehicle, comprising a computer, the computer including a processor and a memory, the memory including instructions that program the processor to: The route to be taken by the vehicle is determined based on sensor data from the vehicle using a trained neural network model; and The trained neural network model is updated based on data received from at least one of the edge computing devices or infrastructure V2I devices; The processor is also programmed to store a data structure representing the weight changes between the trained neural network model and the updated trained neural network model. The processor is also programmed to upload the data structure to a standard platform; the standard platform retrains the trained neural network model using the stored data according to the following formula: ， in, N represents the updated neural network model, and M' represents the neural network model retained by the standard platform. Σ represents the weight update operator, where Σ represents the sum of weight operations applied to the weight differences between the neural network models from i to n, and the weight differences are stored in a record containing the weight differences of the neural network models for each incremental update; The actuators are driven by control signals output from the computer to selectively execute automatic operating modes, including braking, propulsion, and steering of the vehicle. as well as If the accuracy difference of the neural network model is greater than the non-precise parameter, the system switches the vehicle from automatic operation mode to manual operation mode. The non-precise parameter is a preset value set through empirical analysis based on various environments encountered by the automatic vehicle. The accuracy difference of the neural network model is determined by comparing the accuracy of the neural network model output using sensor data from sensors with the accuracy of the neural network model output using sensor data from sensors and data received from edge computing devices and / or infrastructure devices.

2. The system according to claim 1, wherein, The processor is also programmed to update the trained neural network model based on data received from at least one other vehicle.

3. The system according to claim 1, wherein, The processor is also programmed to receive the trained neural network model via over-the-air (OTA) updates.

4. The system according to claim 1, wherein, The processor is also programmed to determine an alternative route via an updated, trained neural network model.

5. The system according to claim 1, wherein, The processor is also programmed to cause the vehicle to travel along the route.

6. The system according to claim 1, wherein, The infrastructure equipment includes roadside equipment.

7. The system according to claim 1, wherein, The edge computing device includes an edge server.

8. The system according to claim 1, wherein, The processor is also programmed to receive a second trained neural network model from the specification platform, wherein the second trained neural network model includes updated weights based on data stored within the data structure.

Images