Apparatus and method for automated vehicle control distributed network

Abstract

an automated vehicle control distributed network node including: at least two modems for communicating with two nearby roadside nodes on the same side of a roadway; at least one antenna for communicating with vehicles via a wireless connection; a pattern recognition process operative to detect patterns using image data from a plurality of high-speed, high-resolution video cameras including night vision capabilities; a vehicle prediction process operatively coupled to the pattern recognition process and operative to predict vehicle position, speed, and direction using the pattern recognition process; and a vehicle controller operatively coupled to the vehicle prediction process to receive vehicle prediction data, operatively coupled to the at least one antenna, and operative to transmit acceleration, deceleration, and steering control signals to a plurality of vehicles in response to the vehicle prediction data received from the vehicle prediction process.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 029,542, filed May 24, 2020, entitled "AUTOMATED VEHICLE CONTROL DISTRIBUTED NETWORK APPARATUSES AND METHODS," which is incorporated herein by reference in its entirety and is assigned to the same assignee as this application.

[0002] Field of the Disclosure The present disclosure relates generally to the Internet of Things (IoT), and more particularly to methods and apparatus for automated vehicle control. [Background technology]

[0003] background The Society of Automotive Engineers (SAE) defines automation levels for automated vehicle systems, including functions such as steering, accelerating and decelerating, monitoring the driving environment, degraded driving performance of the dynamic driving task, and system functions defined by driving modes such as conditional automation, high automation, and full automation (SAE Level 5). At SAE Level 5 automation, the automated driving system performs all aspects of the dynamic driving task under all road and environmental conditions that a human driver can handle. Existing automated driving systems are based on on-board artificial intelligence (AI) systems. However, SAE Level 5 cannot be achieved with existing on-board AI systems. [Brief explanation of the drawings]

[0004] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] FIG. 1 is a diagram of a roadway having an automated vehicle control distribution network, in accordance with various embodiments. [Figure 2]2 is a diagram of the road shown in FIG. 1 illustrating automated vehicles communicating with an automated vehicle control distribution network, in accordance with various embodiments. [Figure 3] FIG. 1 illustrates various automated vehicles communicating with an automated vehicle control distribution network. [Figure 4] FIG. 2 is a diagram of a node in accordance with various embodiments. [Figure 5] 1 is a diagram of a pavement marker according to various embodiments. [Figure 6] 1 is a diagram of a road illustrating automated vehicles communicating with an automated vehicle control distribution network and among themselves, according to some embodiments. [Figure 7] 1 is a diagram of a road illustrating an automated vehicle performing a handover to various nodes of an automated vehicle control distribution network, according to some embodiments. [Figure 8] FIG. 1 is a diagram of a node processing unit that operates to enhance pattern recognition of objects on a road by 100 times, according to one embodiment. [Figure 9] 1 is a flowchart illustrating a method of operating an automated vehicle control distribution network, in accordance with various embodiments. [Figure 10] 1 is a flowchart illustrating a method of operating an automated vehicle control distribution network, in accordance with various embodiments. [Figure 11] 1 is a flowchart illustrating a method of operating an automated vehicle control distribution network, in accordance with various embodiments. [Figure 12] 1 is a flowchart illustrating a method of operation for an automated vehicle to hand over between roadside nodes, according to various embodiments. [Figure 13] 1 is a flowchart illustrating a method of operating an automated vehicle control distribution network, in accordance with various embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0005] Detailed Description In brief, the present disclosure provides an automated vehicle control distributed network that enables a fully automated self-driving system that performs all aspects of the dynamic driving task under all road and environmental conditions without any interaction or control by a human driver. The automated vehicle control distributed network of the present disclosure enables the evolution from SAE Level 2 automation to SAE Level 5 automation.

[0006] The automated vehicle control distributed network of the present disclosure enables detection of the vehicle's surrounding environment, including, but not limited to, the speed, position, and direction of surrounding objects, such as, but not limited to, humans, animals, construction vehicles, other vehicles, etc. All road conditions, including, but not limited to, potholes, ice, other objects, etc., are detected in three dimensions (3D).

[0007] The present disclosure provides an automated vehicle control distributed network node, including at least two modems for communicating with two neighboring roadside nodes on the same side of a roadway, at least one antenna for communicating with vehicles via a wireless connection, a pattern recognition process operative to detect patterns using image data from a plurality of high-speed, high-resolution video cameras including night vision capabilities, a vehicle prediction process operatively coupled to the pattern recognition process and operative to predict vehicle position, speed, and direction using the pattern recognition process, and a vehicle controller operatively coupled to the vehicle prediction process to receive vehicle prediction data, operatively coupled to the at least one antenna, and operative to transmit acceleration, deceleration, and steering control signals to a plurality of vehicles in response to the vehicle prediction data received from the vehicle prediction process. The automated vehicle control distributed network of the present disclosure enables evolution from SAE Level 2 automation to SAE Level 5 automation.

[0008] In some embodiments, the automated vehicle control distributed network node may include at least one high-speed, high-resolution video camera, including night vision capabilities, operatively coupled with pattern recognition processing. The automated vehicle control distributed network node may further include at least a third modem for communicating with a third neighboring node across the road. The automated vehicle control distributed network node may further include at least two modems, at least one antenna, and radios operatively coupled with the vehicle controller, distributed core network, and vehicle processing. The vehicle prediction processing may be implemented using machine learning algorithms.

[0009] The present disclosure also provides an automated vehicle control distribution network including a plurality of automated vehicle control distribution network nodes operatively coupled to one another.

[0010] In some embodiments, the pattern recognition process further operates to detect missing points from the node image using image data from cameras of neighboring nodes. The wireless, distributed core network, and vehicular processes may include fourth generation (4G) and fifth generation (5G) radio access components and associated distributed 4G and / or 5G core networks.

[0011] The present disclosure provides a method of operation that includes acquiring high-speed, high-resolution video data from a plurality of roadway cameras, using the high-speed, high-resolution video data to determine a vehicle position, direction, and speed for at least one vehicle, predicting a position of the at least one vehicle, and transmitting acceleration, deceleration, and steering commands to the vehicle based on the predicted position.

[0012] The method may further include acquiring high-speed, high-resolution video data from at least one camera mounted on a plurality of roadside poles to acquire a time-stamped three-dimensional image. The method may further include performing image correction on the three-dimensional image to generate a corrected image and using the corrected image to determine vehicle position, direction, and speed for at least one vehicle. The method may further include transmitting acceleration, deceleration, and steering commands to the plurality of vehicles as unicast Internet Protocol (IP) packets. The method may further include transmitting acceleration, deceleration, and steering commands to the plurality of vehicles as multicast Internet Protocol (IP) packets. The method may further include acquiring environmental data from a plurality of environmental sensors. The method may further include acquiring environmental data from the plurality of environmental sensors via a pavement marker having the plurality of environmental sensors and a transponder and communicating with the transponder.

[0013] The present disclosure provides a method of operation that includes monitoring a road using a plurality of high-speed, high-resolution road cameras to detect vehicles, animals, pedestrians, road anomalies, and obstacles, creating a predictive model for each of the detected vehicles, animals, pedestrians, road anomalies, and obstacles, determining a control action for at least one vehicle based on the at least one predictive model, and transmitting acceleration, deceleration, and steering commands to the at least one vehicle based on the determined control action. The method may further include receiving control feedback for the at least one vehicle via the plurality of high-speed, high-resolution road cameras, and transmitting acceleration, deceleration, and steering commands to the at least one vehicle that are adjusted based on the control feedback.

[0014] The method may further include maintaining multiple wireless connections between the at least one vehicle and the automated vehicle control distributed network via multiple roadside nodes, and continuously performing make-before-break wireless handoffs with at least one additional roadside node while the at least one vehicle travels along a roadway such that there are no communication delays between the at least one vehicle and the automated vehicle control distributed network. The method may further include redundantly transmitting coordinated acceleration, deceleration, and steering commands to the at least one vehicle using the multiple roadside nodes.

[0015] Reference is now made to the drawings. Like reference numbers represent like components between the drawings. FIG. 1 illustrates an automated vehicle control distribution network 100 according to various embodiments. The automated vehicle control distribution network 100 is one type of disclosed apparatus according to various embodiments. A roadway 101 includes pavement markers 103 (or pavement studs) that house transponder components, thereby enabling the transponder components to communicate information to various other components of the automated vehicle control distribution network. The transponders may be, for example, radio frequency identification (RFID) or equivalent transponder communication capabilities.

[0016] The automated vehicle control distribution network includes various poles 105 or towers positioned at various roadside points on both sides of a road 101. Each pole 105 is mounted with a set of high-speed (i.e., at least 60 fps), high-resolution video cameras 107, including night vision capabilities, and further includes a node 110. The cameras 107 on each pole are operatively coupled to and communicate with the node 110. The viewing angles of the cameras 107 are positioned to overlap with each other with respect to the field of view of each camera along the road. For example, the field of view of the right-most camera on any pole 105 is positioned to overlap with the field of view of the left-most camera on that pole 105, and so on.

[0017] In one embodiment, the fields of view of all cameras 107 on the pole 105 are stitched together into one frame (e.g., one long, high-resolution rectangle or trapezoid). The fields of view of the various cameras 107 cover both sides of the road 101. In one embodiment, the images from the multi-camera 107 may be corrected using keystone (i.e., applying a keystone correction algorithm), such as optical keystone correction, digital keystone correction, or a combination thereof. The image processing used is 3D image processing, and a 4D image is generated using frame timestamps. The entire field of view of the cameras 107 extends across the road 101 to capture objects other than vehicles, such as animals, pedestrians, road deformations (e.g., potholes / holes, pavement cracks, pavement buckling), etc. The position of the pole 105 on the other side of the road is offset to the midpoint, as shown in FIG. 1 .

[0018] FIG. 2 is a block diagram of the road shown in FIG. 1 , illustrating automated vehicles 109 communicating with the automated vehicle control distributed network 100 via multiple wireless links 108, in accordance with various embodiments. The automated vehicle control distributed network 100 operates to recognize each vehicle 109 based on a specific singularity (e.g., 16, 32, 48, etc.). In some embodiments, if the make, model, color, and license plate number of the vehicle 109 can be detected, the automated vehicle control distributed network 100 can perform a database lookup to obtain information such as the vehicle's size, year, or other information. If not, the automated vehicle control distributed network 100 operates to obtain information such as the vehicle's make, model, color, size, and year using pattern recognition and a database (e.g., using the license plate number as the vehicle's ID). Non-automated vehicles can also be detected by the automated vehicle control distributed network 100, and information can also be retrieved from the database under the same conditions. The transponder-enabled pavement markers 103 can be passive or battery-powered transponders.

[0019] The transponder-enabled pavement markers 103 operate to communicate with the nodes 110 and are used by the nodes 110 to calibrate their locations and provide roadway 101 condition updates (e.g., temperature, humidity, etc.). Accordingly, each pavement marker 103 may also include various environmental sensors (e.g., but not limited to, temperature, humidity, pressure, etc.). Because the roadway 101 expands and contracts with temperature, may buckle, or may develop potholes, cracks, and other deformations, the exact locations of the pavement markers 103 periodically change over time. Based on the geographic information of the pavement markers 103 detected by the various nodes 110, the automated vehicle control distributed network 100 operates to ensure accuracy by periodically recalibrating each location.

[0020] The direction and speed of a vehicle 109 is calculated by the nodes 110 for each vehicle 109 using the ID of the vehicle 109 detected in adjacent video images and corresponding timestamps. In an example of operation on an east-west road, all vehicle IDs traveling east will pass and be identified by all adjacent and oncoming nodes 110 on both sides of the road in the eastward direction along the road 101. The same is done for all westward-bound vehicles with adjacent and oncoming nodes 110 in the westward direction along the road 101.

[0021] Each automated vehicle 109 simultaneously communicates with at least five nodes 110 and performs wireless handoff of at least one of the wireless links 108 from one node 110 to another node 110 while moving, ensuring that communication with at least five nodes 110 is maintained at all times. In other words, 4+1 redundancy of the wireless links 108 is maintained. Vehicle control commands (e.g., but not limited to, acceleration, deceleration, and steering commands) may be redundantly transmitted to the vehicle via each of the redundant wireless links, thereby increasing the reliability of the control commands. In one example of a wireless handoff operation, the vehicle 109 may initially communicate with node A-2, which is located at the far right of the road 101, and then initiate a wireless handoff with node A+1 while moving in the direction of the movement arrow shown in FIG. 2 . In other words, the vehicle 109 maintains multiple wireless connections with multiple nodes and continuously performs make-before-break wireless handoffs while moving along the road. Each node 110 performs its own prediction of the vehicle's 109 location and can share this information with each of the other nodes 110. In other words, each node 110 operates independently of the other nodes 110 in the distributed network to perform pattern recognition and apply artificial intelligence or machine learning to create predictive models for vehicle location, direction, and speed (as well as for non-autonomous vehicles, unregistered vehicles, other objects, pedestrians, animals, and roadway deformations) and to send control signals, including acceleration, deceleration, and steering. Nodes 110 communicate with each other as needed to share data, models, and processing capabilities, thereby providing increased redundancy for all automated vehicles. Each node 110 in the distributed network also operates to collect training data used to train machine learning / AI algorithms (e.g., but not limited to, pattern recognition and vehicle prediction processing), so that these and other machine / AI algorithms can be initially trained and further enhanced with additional collected big data.

[0022] 3 is a diagram illustrating a road and the interaction of various automated vehicles with the automated vehicle control distributed network 100, according to various embodiments. Each node 110 in the automated vehicle control distributed network 100 performs pattern recognition on its video images, operative to identify animals in the area, humans in the area, road variations such as potholes or buckles, motorcycles between lanes, oversized truck cargo, etc. Detected objects are modeled in the node 110 with at least 16 characteristics, including size, weight, maximum speed, hardness, etc., and stored in a database. Each modeled object is assigned a unique ID by the creating node 110, and the assigned ID is passed from the creating node 110 to adjacent nodes 110 via wireless or wired communication links 112 between the nodes 110.

[0023] Each node 110 realizes fully distributed network functions (Network Function Virtualization (NFV)) and accommodates 4G / 5G wireless and core network functions in a 1:1 ratio. Each pole 105 in the automated vehicle control distributed network 100 includes at least one node 110. Each node 110 has its own handover neighbor list, but the neighbor list does not include its adjacent node 110. Instead, the neighbor list includes the second-neighbor node 110. In an example of handoff groups between nodes 110, handoff group 1 is node (2n) (n=1, 2, 3, ..., M), and handoff group 2 is node (2n+1) (n=1, 2, 3, ..., M). Handoff group 1 is configured to use the same frequency of a first wireless channel, and handoff group 2 is configured to use the same frequency of a second wireless channel.

[0024] Node-to-node communication links include communication links 112 on the same side of the road and communication links 114 that cross the road, thereby forming a grid or mesh. The communication links 112 and 114 between nodes 110 may be wired communication links, wireless communication links, or a combination of both wired and wireless communication links. The wireless or wired communication links 112 and 114 are set up in a mesh configuration as shown to allow for redundancy. Adjacent and opposite nodes 110 are linked. Each node 110 collects all vehicle and object IDs within its visual detection area, as well as the vehicle IDs and object IDs of each neighboring node 110.

[0025] For example, in FIG. 3, node A would have all IDs of vehicles and objects it has identified, as well as all IDs for nodes A+1, A-1, B, and B-1. All IDs and their associated information are compiled into a single package, including the object's size, weight, speed, direction, current location, type, timestamp, score, etc. Appropriate objects are also assigned a danger score. Detected road anomalies, such as potholes, are assessed for their size and depth. For vehicles, recent vehicle driving history is evaluated. For example, drunk driving or reckless driving detected through pattern recognition is flagged. Information including all vehicle and object IDs and vehicle driving direction is passed to adjacent nodes on both sides of the road. All vehicles on the road are tracked, including vehicles not receiving control signals (i.e., vehicles not registered with the automated vehicle control distributed network 100) and non-automated vehicles.

[0026] Communication from node 110 to node 110 is performed using Internet Protocol (IP) packets and is performed to all automated vehicles registered in the automated vehicle control distribution network 100. The IP packets may be broadcast, unicast, or multicast, depending on the situation. For example, unicast Internet Protocol (IP) packet delivery is used for direct vehicle control. Driving instructions are based on identified hazards, road conditions, and the speed and location of vehicles around the registered vehicle. Braking, acceleration, and steering control signals may be based on this identified hazard information and may be sent to multiple vehicles using broadcast packets.

[0027] The functionality of multicast IP packets may be used for fleet vehicle control. For example, weather conditions may be the basis for sending a message specifying the maximum speed to a truck fleet. Multicast users also receive broadcast packets. Broadcast IP packets are used to provide information to all registered vehicles, and aid information such as 3D size, direction of movement, and speed information of all objects (vehicles, animals, humans) is provided to the current pole node 110 and its neighboring nodes. Data updates within the system are as fast as 20 milliseconds.

[0028] In one embodiment, efficient pattern recognition is performed using least squares methods. In one example where a vehicle is captured at 8 xyz points and compared to a model of 8 XYZ points, the score is sqrt[(x1-X1) 2 +(y1-Y1) 2 +(z1-Z1) 2 ]+sqrt[(x2-X2) 2 +(y2-Y2) 2 +(z1-Z1) 2 ]+...+sqrt[(x8-X8) 2 +(y8-Y8) 2 +(z8-Z8) 2], and the smallest score becomes the model. The node 110 processor is specially designed for least-squares fitting up to 64 points, using a logarithmic algorithm to significantly reduce multiplication, division, squaring, and square root operations.

[0029] FIG. 4 is a diagram of a node 110 according to various embodiments. The automated vehicle control distributed network node 110 is one type of device disclosed according to various embodiments. Each node 110 includes a cellular antenna 113 for communicating with vehicles via wireless link 108 and transmitting control signals to the vehicles. The cellular antenna 113 may be an antenna array, which may be a multiple-input multiple-output (MIMO) antenna array. At least three modems 111 facilitate communication with nearby neighboring nodes via wireless communication link 115, wired communication link, or a combination of both. Like the wireless link 108, in some embodiments, the wireless communication link 115 may also be facilitated using a MIMO antenna array. Cameras C1-C4 are operatively coupled to the node 110 and to 3D image processing 407. The 3D image processing 407 feeds processing results to pattern recognition 409, which provides pattern recognition data to vehicle prediction processing 405.

[0030] A transponder reader 411 operates to communicate with the transponders of the pavement markers 103 via wireless link 413 to obtain environmental sensor data. The environmental sensor data is provided to the vehicle prediction process 405 via operative coupling.

[0031] Node 110 may include any number of modems 111, and Figure 4 shows an example where there are three modems 111 in node 110. The three modems 111 are operatively coupled to wireless (e.g., 4G / 5G) and 4G / 5G distributed core networks and vehicle processing 401, which is further operatively coupled to vehicle controller 403.

[0032] Vehicle controller 403 is operatively coupled to vehicle prediction process 405. Object identification data and prediction data generated by pattern recognition 409 and vehicle prediction process 405 is shared with neighboring nodes via wireless communication link 115 using modem 111. Vehicle prediction process 405 is operatively coupled to vehicle controller 403 and operates to communicate road condition and object information. Vehicle prediction process 405 and vehicle controller 403 are configured as a feedback system in which vehicle prediction process 405 detects changes in vehicle position that occur in response to vehicle control signals transmitted from vehicle controller 403.

[0033] Vehicle controller 403 operates to control the vehicle by sending acceleration, deceleration, and steering control signals over wireless link 108 using 4G / 5G radio and 4G / 5G distributed core network and vehicle processing 401. 4G / 5G radio and 4G / 5G distributed core network and vehicle processing 401 is operatively coupled to cellular antenna 113 for sending vehicle control signals over wireless link 108. 4G / 5G radio and 4G / 5G distributed core network and vehicle processing 401 includes 4G / 5G radio and embedded distributed core network functionality that allows node 110 to operate as an independent entity within the distributed network, thereby maintaining full control of the automated vehicle even if other nodes 110 are disabled or otherwise unavailable.

[0034] The various processing / processing units within node 110 may be implemented as a system-on-chip (SoC) system and may include hardware, firmware, and software for performing the various functions of node 110.

[0035] 5 is a diagram of an example pavement marker 103 according to various embodiments. The pavement marker 103 is one type of device disclosed according to various embodiments. The pavement marker 103 includes a transponder (which may be an RFID transponder) and an environmental sensor operatively coupled to the transponder 501. The node 110 is capable of communicating with the transponder 501 to extract data from the environmental sensor 503. The environmental sensor may include, but is not limited to, a temperature sensor, a humidity sensor, a pressure sensor, an inertial sensor, etc.

[0036] 6 is a block diagram of a roadway 101 illustrating automated vehicles communicating with an automated vehicle control distribution network 100 and among themselves, according to some embodiments. Vehicle-to-vehicle communications 601 may be facilitated by the same pole 105 and node 110 or by adjacent poles / nodes. In some embodiments, each vehicle has 4+1 redundant links for reliability.

[0037] FIG. 7 illustrates an automated vehicle 701 performing handovers along a road to various nodes 110 of the automated vehicle control distribution network 100, according to some embodiments. The automated vehicle 701 maintains multiple wireless connections 703 with at least four nodes 110 at any given time and performs make-before-break handover operations with each upcoming new node 110 as it travels along its travel path. In the example of FIG. 7 , the automated vehicle 701 initially communicates with five nodes: A+0, A+1, A+2, B+0, and B-1 (not shown). At its current location, the automated vehicle 701 establishes a new connection 705 with node B+2 and drops its previous connection with node B-1. As the automated vehicle 701 travels along the road to position 707, a make-before-break connection 705 is established with node A+3. After the connection with node A+3 is established, the connection with node A+0 is dropped. Upon traveling to position 709, the automated vehicle 701 establishes a connection with node B+3 and drops the connection with node B+0. Shortly before position 711, the automated vehicle adds a connection with node A+4 and drops the connection with node A+1. Shortly before position 713, the automated vehicle adds a connection with node B+4 and drops the connection with node B+1. Shortly before position 715, the automated vehicle adds a connection with node A+5 and drops the connection with node A+2, and so on.

[0038] 8 is a diagram of a node processor 800 according to one embodiment. The node 110 processor 800 may perform 3D image processing 407 and pattern recognition 409, and is specially designed for up to 64-point least squares and uses a logarithmic algorithm to significantly reduce multiplication, division, squaring, and square root operations. In some embodiments, the node processor 800 may be used to implement vehicle predictive processing 405, vehicle controller 403, and 4G / 5G+ core network and vehicle processing 401, or any combination thereof. The 4G / 5G+ core network and vehicle processing 401, vehicle controller 403, vehicle predictive processing 405, 3D image processor 407, and / or pattern recognition 409 may each be implemented as one or more microprocessors, ASICs, FPGAs, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or devices that manipulate signals based on operational instructions. One or more of the processors used to implement node 110 are configured and operative to, among other functions, fetch and execute computer-readable instructions (i.e., executable instructions) stored in memory (not shown), which may be a separate, non-volatile, non-transitory memory within node 110 and / or on-board memory that is part of an SoC configuration, or a combination of both. Regardless of the specific implementation of 4G / 5G+ Core Network and Vehicle Processing 401, Vehicle Controller 403, Vehicle Prediction Processing 405, 3D Image Processor 407, and Pattern Recognition 409, each component is operatively coupled with the communication inputs and outputs shown in FIG. 4 and is operative to execute all associated software and / or firmware, including all required APIs (Application Programming Interfaces) between such components. 4G / 5G+ Core Network and Vehicle Processing 401 includes all required wireless baseband hardware and software and is operative to run the Internet Protocol (IP) stack and form multiple wireless IP connections with vehicles and other nodes for sharing information, sending control commands, and receiving feedback information.4G / 5G+ Core Network and Vehicle Processing 401 is a complete network infrastructure / architecture that includes all 4G / 5G radio and core network components / entities required to implement 4G / 5G operational functions, including maintaining redundant radio links with controlled vehicles and performing make-before-break radio handoff for multiple vehicles.

[0039] 9 is a flowchart illustrating a method of operation of the automated vehicle control distributed network 100, according to various embodiments. The method begins at operation block 901, when a node 110 of the automated vehicle control distributed network 100 acquires visual image data of a road. The visual image data captures vehicles and, among other things, pedestrians, animals, road objects, and pavement deformations. At operation block 903, the node 110 acquires environmental sensor data from pavement markers 103. At operation block 905, the node 110 determines vehicle position, direction, and speed for multiple vehicles on the road. At operation block 907, the node 110 predicts vehicle positions for all vehicles registered with the automated vehicle control distributed network 100. At operation block 909, the node 110 transmits acceleration, deceleration, and steering control signals to each registered vehicle.

[0040] 10 is a flowchart illustrating a method of operation of an automated vehicle control distributed network in accordance with various embodiments. The method begins in operation block 1001, in which a node 110 of the automated vehicle control distributed network 110 acquires visual image data of a vehicle from at least five cameras to obtain a three-dimensional image with a timestamp for each frame. In operation block 1003, the node 110 performs keystone correction on the image to generate a corrected image. In operation block 1005, the node 110 uses the corrected image to determine the position, direction, and speed of the vehicle. In operation block 1007, the node 110 predicts the vehicle position. In operation block 1009, the node 110 transmits acceleration, deceleration, and steering control signals to the vehicle.

[0041] 11 is a flowchart illustrating a method for operating an automated vehicle control distributed network in accordance with various embodiments. The method begins in operation block 1101 with visual image data of a vehicle acquired from multiple road nodes 110 using at least four cameras at each node, obtaining three-dimensional image data with a timestamp for each frame. Each node 110 then shares its image data with its neighboring nodes in operation block 1103. Each node 110 determines the position, direction, and speed of all vehicles and objects in its image data in operation block 1105. Each node 110 then predicts the vehicle's position and identifies potential road hazards in operation block 1107. Each node 110 then transmits acceleration, deceleration, and steering control signals from the node 110 to the vehicle in operation block 1109.

[0042] FIG. 12 is a flowchart illustrating a method for an automated vehicle to perform a handover between roadside nodes as shown in FIG. 7 , according to various embodiments. The process begins with an automated vehicle traveling along a roadway, establishing and maintaining wireless communication links with at least four nodes, as in operation block 1201. As the automated vehicle moves past a new roadside node 110, an increase in RSSI (Received Signal Strength Indicator) occurs and is observed by the automated vehicle. The increase in the automated vehicle's RSSI is also sensed by the radio transceiver of the node 110 that the automated vehicle is approaching. Accordingly, in decision block 1203, the automated vehicle may detect the next node, for example, using a threshold RSSI value or some other communication link metric. In other words, RSSI is merely one example of a metric, and any other suitable communication link metric may be used to detect the next node, such as, but not limited to, a bit error rate, a frame error rate, a frame erasure rate, or some other metric. The existing at least four wireless communication links are maintained in operation block 1201 until a candidate next node meets the threshold metric requirement. If a next node candidate appears at decision block 1203, the automated vehicle establishes a new make-before-break communication link with the candidate node at operation block 1205. Once that communication link is established, the automated vehicle may delete one of the at least four previous communication links at operation block 1207. Typically, the communication link with the lowest metric is deleted; however, in some embodiments, the node farthest from the vehicle may be deleted by default. If the vehicle is shut down at decision block 1209 (e.g., because the vehicle has stopped or is otherwise stuck at its destination), the process ends. If the vehicle is not shut down, the process continues in a loop from operation block 1201 until the vehicle stops moving.

[0043] 13 is a flowchart illustrating a method of operation of an automated vehicle control distributed network according to various embodiments. The method begins with the automated vehicle control distributed network monitoring all activity on a roadway at operational block 1301. The automated vehicle control distributed network acquires sensor data from pavement sensors via a wireless connection at operational block 1303 and uses the sensor data to calibrate pavement marker positions at operational block 1305. The sensor data may include, but is not limited to, temperature data, humidity data, pressure data, etc.

[0044] The automated vehicle control distribution network then monitors all road activity, including vehicle monitoring at decision block 1307, animal monitoring at decision block 1309, pedestrian monitoring at decision block 1311, road anomaly monitoring at decision block 1313, and obstacle monitoring at decision block 1315. The process at each decision block continues indefinitely, continuously tracking all items on the road.

[0045] If a vehicle is detected at decision block 1307, the automated vehicle control distributed network may detect the vehicle's license plate number at action block 1317 and check the license plate number against a database at decision block 1319. All vehicle information in the database is retrieved at action block 1321. On the other hand, if the vehicle does not have a license plate, or if no information is available in the database at decision block 1319, the automated vehicle control distributed network uses a visual detection system to detect the make, model, color, and weight of the vehicle at action block 1323. At action block 1325, the automated vehicle control distributed network uses any database information and information from the visual detection system to create a predictive model.

[0046] If an animal is detected at decision block 1309, the automated vehicle control distributed network creates an animal movement prediction model at action block 1327. Similarly, if a pedestrian is detected at decision block 1311, the automated vehicle control distributed network creates a pedestrian movement prediction model at action block 1329. If any road anomalies are detected at decision block 1313, a model of the anomaly is created at action block 1331, including features such as, but not limited to, location, size, depth of potholes, etc. If an obstacle is detected at decision block 1315, a model of the obstacle is created at action block 1333, including features such as, but not limited to, object size, material, weight, etc., to the extent that it can be detected by a combination of information from the visual detection system and pavement sensors.

[0047] Based on all of the generated predictive models, the automated vehicle control distributed network determines appropriate avoidance actions for each automated vehicle, as also modeled in operation block 1325, in operation block 1335. In operation block 1337, the automated vehicle control distributed network cooperatively sends appropriate control commands to each automated vehicle so that all collisions are avoided. In operation block 1339, feedback is obtained to make further trajectory corrections for each automated vehicle.

[0048] While various embodiments have been shown and described, it should be understood that the invention is not so limited. Those skilled in the art will recognize various modifications, changes, variations, substitutions, and equivalents that do not depart from the scope of the invention, as defined in the appended claims.

Claims

1. 1. An automated vehicle control distributed network node, comprising: at least two modems for communicating with two neighboring roadside nodes on the same side of the road; at least one antenna for communicating with the vehicle via a wireless connection; a wireless distributed core network and vehicular processing operatively coupled to the at least two modems and the at least one antenna; a pattern recognition process operative to detect patterns using image data from a plurality of high speed, high resolution video cameras including night vision capabilities; a vehicle prediction process operatively coupled with said pattern recognition process and operable to predict vehicle position, velocity, and orientation using said pattern recognition process; a vehicle control device operatively coupled to the wireless distributed core network and to vehicle processing, operatively coupled to the vehicle prediction processing to receive vehicle prediction data, operatively coupled to the at least one antenna, and operable to transmit acceleration, deceleration, and steering control signals to a plurality of vehicles in response to the vehicle prediction data received from the vehicle prediction processing; An automated vehicle control distributed network node including:

2. at least one high speed, high resolution video camera, including night vision capabilities, operatively coupled with said pattern recognition processing; 2. The automated vehicle control distributed network node of claim 1, comprising:

3. 10. The automated vehicle control distributed network node of claim 1, further comprising: at least a third modem for communicating with a third neighboring node across the road.

4. The automated vehicle control distributed network node of claim 1, wherein the wireless distributed core network and vehicle processing includes fully distributed network functionality and wireless and core network functionality operatively coupled with the at least two modems, the at least one antenna, and the vehicle control device.

5. The automated vehicle control distributed network node of claim 1 , wherein the vehicle prediction process is performed using a machine learning algorithm.

6. An automated vehicle control distributed network comprising a plurality of operatively coupled automated vehicle control distributed network nodes according to claim 1.

7. The pattern recognition process further comprises: Detect missing points from the node image using image data from cameras of neighboring nodes It works like this, The automated vehicle control distributed network node of claim 1 .

8. 5. The automated vehicle control distributed network node of claim 4, wherein the wireless distributed core network and vehicle processing includes fourth generation (4G) and fifth generation (5G) radio access components and associated distributed core networks.

9. Acquiring high-speed, high-resolution video data from multiple road cameras by an automated vehicle control distributed network node including a wireless distributed core network and vehicle processing; using the high-speed, high-resolution video data to determine, for at least one vehicle, a vehicle position, direction, and speed; predicting a position of the at least one vehicle; transmitting acceleration, deceleration, and steering commands to the vehicle from the automated vehicle control distributed network node based on the predicted position; A method comprising:

10. acquiring the high-speed, high-resolution video data from at least one camera mounted on a plurality of roadside poles to obtain a time-stamped three-dimensional image; The method of claim 9 further comprising:

11. performing image correction on the three-dimensional image to generate a corrected image; using the corrected image to determine, for at least one vehicle, a vehicle position, orientation, and velocity; The method of claim 10 further comprising:

12. Sending acceleration, deceleration, and steering commands to multiple vehicles as unicast Internet Protocol (IP) packets The method of claim 9 further comprising:

13. Sending acceleration, deceleration, and steering commands to multiple vehicles as multicast Internet Protocol (IP) packets The method of claim 12 further comprising:

14. Acquiring environmental data from multiple environmental sensors The method of claim 9 further comprising:

15. acquiring environmental data from the plurality of environmental sensors via a pavement marker including the plurality of environmental sensors and the transponder by communicating with the transponder; The method of claim 9 further comprising:

16. acquiring geographic location data from the plurality of sensors via a pavement marker including the plurality of sensors and the transponder by communicating with the transponder; calibrating the three-dimensional image using the geographic location data; The method of claim 10 further comprising:

17. Monitoring roads using multiple road high-speed, high-resolution cameras to detect vehicles, animals, pedestrians, road anomalies, and obstacles; creating a predictive model for each of the detected vehicles, animals, pedestrians, road anomalies, and obstacles, each of the predictive models being created in an automated vehicle control distributed network node including a wireless distributed core network and vehicle processing; determining a control action for the at least one vehicle based on the at least one predictive model; sending acceleration, deceleration, and steering commands to the at least one vehicle based on the determined control action; A method comprising:

18. receiving control feedback for the at least one vehicle via the plurality of road high speed high resolution cameras; transmitting acceleration, deceleration, and steering commands to the at least one vehicle that are adjusted based on the control feedback; 20. The method of claim 17, further comprising:

19. maintaining a plurality of wireless connections between the at least one vehicle and an automated vehicle control distribution network via a plurality of roadside nodes; successively performing make-before-break wireless handoffs by the at least one vehicle to at least one additional roadside node as the at least one vehicle moves along a roadway such that there is no communication delay between the at least one vehicle and the automated vehicle control distributed network; 20. The method of claim 17, further comprising:

20. using the plurality of roadside nodes to redundantly transmit acceleration, deceleration, and steering commands to the at least one vehicle; 20. The method of claim 19 further comprising:

Images