Traffic camera-based determination of traffic signal and lane association for automated vehicle operation

Abstract

An example method includes obtaining information about motion of a vehicle using video or image frames of the vehicle passing through a traffic intersection during a time window obtained by a traffic camera; obtaining road definition information indicative of a traffic lane at the traffic intersection, wherein the road definition information corresponds to a motion of the vehicle indicated in a video frame or image frame of the vehicle as obtained by the traffic camera; obtaining traffic signal light information of traffic signal lights at the traffic intersection, the traffic signal light information corresponding to a time window of the vehicle passing through the traffic intersection, the traffic signal light information indicating timing of the traffic signal lights; and associating the traffic lane with the traffic light by determining that the traffic signal is associated with the lane using the information about the movement of the vehicle, the road definition information, and the traffic light information.

Description

[0001] Related Applications

[0002] This application claims the benefit of U.S. Application No. 18 / 320,935, entitled “TRAFFIC CAMERA-BASED DETERMINATION OF TRAFFIC SIGNAL-TO-LANE ASSOCIATION FOR AUTOMATED VEHICLE OPERATION” and filed on May 19, 2023, which is assigned to the assignee of the present application and is hereby incorporated by reference in its entirety. BACKGROUND TECHNICAL FIELD

[0004] The present disclosure relates generally to the field of traffic management, and more specifically, to a method for determining traffic signal-to-lane associations using traffic camera data in order to facilitate operation of automated vehicles.

[0005] Related Art

[0006] As the development of automated vehicles (AVs) continues to gain momentum, successfully navigating through complex multi-lane intersections with multiple traffic signal lights can be challenging as determining the proper traffic signal light to obey when approaching or stopping at an intersection can be a challenge for AVs, and even for human drivers. AVs that typically rely on forward-facing fixed directional cameras for detecting features, objects, road users, etc. ahead of the vehicle face difficulties in accurately interpreting the traffic signals governing a particular lane. SUMMARY

[0007] An example method for traffic signal and lane association based determination for traffic intersections for automated vehicle operation, the method performed by a server and can include obtaining information about motion of vehicles using video frames or image frames of vehicles crossing a traffic intersection during a time window obtained by a traffic camera; and obtaining road delineation information indicative of traffic lanes at the traffic intersection, where the road delineation information corresponds to the motion of vehicles as indicated in the video frames or image frames of vehicles obtained by the traffic camera. The method can further include obtaining traffic signal light information of traffic signal lights at the traffic intersection, the traffic signal light information corresponding to the time window of vehicles crossing the traffic intersection, where the traffic signal light information is indicative of timing of the traffic signal lights; and determining traffic signal and lane association to associate the traffic lanes with the traffic signal lights by using the information about motion of vehicles, the road delineation information, and the traffic signal light information. The method can further include propagating the traffic signal and lane association to an automated vehicle (AV) for the AV to navigate through the traffic intersection.

[0008] An example server for traffic signal and lane association based determination for traffic intersections for automated vehicle operation, the radar unit includes a transceiver; a memory; and one or more processors communicatively coupled with the transceiver and the memory. The one or more processors can be configured to obtain information about motion of vehicles using video frames or image frames of vehicles crossing a traffic intersection during a time window obtained by a traffic camera; and obtain road delineation information indicative of traffic lanes at the traffic intersection, where the road delineation information corresponds to the motion of vehicles as indicated in the video frames or image frames of vehicles obtained by the traffic camera. The one or more processors can be further configured to obtain traffic signal light information of traffic signal lights at the traffic intersection, the traffic signal light information corresponding to the time window of vehicles crossing the traffic intersection, where the traffic signal light information is indicative of timing of the traffic signal lights; and determine traffic signal and lane association to associate the traffic lanes with the traffic signal lights by using the information about motion of vehicles, the road delineation information, and the traffic signal light information. The one or more processors can be further configured to propagate the traffic signal and lane association to an automated vehicle (AV) for the AV to navigate through the traffic intersection.

[0009] An example apparatus for traffic signal and lane association determination based on traffic cameras for traffic intersections for automated vehicle operation, the apparatus can include means for obtaining information about motion of vehicles using video or image frames of vehicles crossing a traffic intersection during a time window obtained by a traffic camera; and means for obtaining road delineation information indicative of traffic lanes at the traffic intersection, where the road delineation information corresponds to the motion of vehicles as indicated in the video or image frames of vehicles obtained by the traffic camera. The apparatus can also include means for obtaining traffic signal light information of traffic signal lights at the traffic intersection, the traffic signal light information corresponding to the time window of vehicles crossing the traffic intersection, where the traffic signal light information is indicative of timing of the traffic signal lights; and means for determining traffic signal and lane association to associate the traffic lanes with the traffic signal lights by using the information about motion of vehicles, the road delineation information, and the traffic signal light information. The apparatus can also include means for propagating the traffic signal and lane association to an automated vehicle (AV) for the AV to navigate through the traffic intersection.

[0010] An example non-transitory computer-readable medium storing instructions for traffic signal and lane association determination based on traffic cameras for traffic intersections for automated vehicle operation, the instructions can include code for obtaining information about motion of vehicles using video or image frames of vehicles crossing a traffic intersection during a time window obtained by a traffic camera; and obtaining road delineation information indicative of traffic lanes at the traffic intersection, where the road delineation information corresponds to the motion of vehicles as indicated in the video or image frames of vehicles obtained by the traffic camera. The instructions can also include code for obtaining traffic signal light information of traffic signal lights at the traffic intersection, the traffic signal light information corresponding to the time window of vehicles crossing the traffic intersection, where the traffic signal light information is indicative of timing of the traffic signal lights; and determining traffic signal and lane association to associate the traffic lanes with the traffic signal lights by using the information about motion of vehicles, the road delineation information, and the traffic signal light information. The instructions can also include code for propagating the traffic signal and lane association to an automated vehicle (AV) for the AV to navigate through the traffic intersection.

[0011] This summary is neither intended nor should it be construed to identify any key or essential features, nor is it intended to be used in determining the scope of the subject matter. The subject matter should be understood from reading the entire specification, including any claims, and from cooperating the appropriate portions of the specification, any or all drawings, and the entire claims. The foregoing and other features and examples are described in greater detail below. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a diagram of a system according to embodiments in which vehicles can be on various networks and communicate with various devices, vehicles, and servers.

[0013] Figure 2 An example scenario of a traffic intersection 200 in which traffic signals need to be associated with lanes is illustrated.

[0014] Figure 3 is a high level block diagram of a method of determining traffic signal and lane association according to embodiments.

[0015] Figure 4 is an example of information element (IE) definitions according to embodiments.

[0016] Figure 5 is a flowchart of a method of traffic camera based determination of traffic signal and lane association for automated vehicle operation at a traffic intersection according to some embodiments.

[0017] Figure 6 is a block diagram of an embodiment of a computer system.

[0018] Figure 7 is a block diagram of an embodiment of a V2X device.

[0019] Like reference symbols in the various drawings indicate like elements. In addition, multiple instances of an element can be indicated by adding a dash and a second number to the first number, e.g., 110-1, 110-2, 110-3, etc., or 110a, 110b, 110c, etc. When only the first number is used to refer to such an element, it is understood that any instance of that element (e.g., element 110 would refer to elements 110-1, 110-2, and 110-3 or to elements 110a, 110b, and 110c in the previous example) is intended. DETAILED DESCRIPTION

[0020] Several illustrative embodiments will now be described with reference to the drawings, which form a part of this disclosure. Although specific embodiments can be described in detail below, other embodiments can be utilized and various modifications can be made without departing from the scope of the disclosure.

[0021] As referred to herein, "V2X devices," "V2X vehicles," and "V2X entities" refer to devices, vehicles, and entities, respectively, that are capable of transmitting and receiving V2X messages. Similarly, non- "V2X vehicles" and non-V2X entities" refer to vehicles and entities that do not participate in or are not capable of participating in V2X communications. Further, "V2X devices" described in greater detail herein refer to devices, systems, components, etc. that can be incorporated into and / or used by V2X entities to enable V2X communications. While many embodiments describe "V2X vehicles" and "non-V2X vehicles," it will be appreciated that many embodiments can be extended to include non-vehicle entities, such as pedestrians, bicyclists, road hazards, obstacles, and / or other traffic-related objects, etc. Further, it can be noted that embodiments can apply to vehicles and / or RSUs that are capable of traffic-related communications, and not necessarily V2X-capable vehicles / RSUs. Further, while embodiments provided herein can be performed by autonomous and / or semi-autonomous vehicles, embodiments are not so limited. For example, embodiments can include traditional (non-autonomous) vehicles that have the ability to determine and communicate intended maneuvers (e.g., within an onboard navigation computer, capable of communicating instructions to a human driver). One of ordinary skill in the art will appreciate such variations.

[0022] Various aspects generally relate to the field of traffic management, and more particularly, to a method for determining traffic signal to lane associations using traffic camera data to facilitate operation of AVs. In some examples, data obtained by traffic cameras can be used to determine associations of traffic signal lights to corresponding traffic lanes at an intersection, ultimately to assist AV navigation. For example, a cloud-based server can obtain movements of vehicles at an intersection based on traffic cameras. The cloud-based server can also obtain road definition information (e.g., road markings, traffic lanes, etc.) at an intersection of interest corresponding to the movements of the vehicles, and can obtain traffic signal light information indicative of timing of traffic signal lights at the traffic intersection, the traffic signal light information corresponding to a time window in which the vehicles cross the traffic intersection. Thus, traffic signal to lane associations of traffic lanes and traffic signal lights at the intersection of interest can be determined by using information about the movements of the vehicles, the road definition information, and the traffic signal light information to associate the traffic lanes with the traffic signal lights.

[0023] In some implementations, traffic signal and lane association information can be propagated to the AV (Active Access Controller) through various methods, including prior provisioning, location-based knowledge provisioning, or via request-response communication mechanisms (e.g., embedded in Onboard Unit (OBU) application layer messages or OBU subscription services). Traffic signal and lane association information can be sent to the AV via various channels, such as wired connections, over-the-air methods (e.g., Wi-Fi download facilitated by Original Equipment Manufacturers (OEMs) or commercial cloud-based services), or over-the-air communication via Uu or V2X, using application layer or lower-layer messages (e.g., RRC) from cloud-based services.

[0024] Specific aspects of the subject matter described in this disclosure can be implemented to achieve one or more of the following potential advantages. In some examples, by utilizing the techniques discussed herein, an AV can effectively acquire and / or update traffic signal and lane association information, thereby enhancing its ability to navigate safely and efficiently through complex traffic scenarios.

[0025] Figure 1 This is an example of a system according to one embodiment in which vehicles can communicate on various networks and with various devices, vehicles, and servers. In one embodiment, V2X vehicle A 180 can communicate with V2X or other communication-enabled vehicle B 190 on link 123 using a V2X or other wireless communication transceiver. Some embodiments may, for example, communicate to perform relative positioning between vehicles, negotiation for lane changes, crossing intersections, and / or exchanging V2X data elements (such as GNSS measurements; vehicle status, vehicle position, and vehicle capabilities; measurement data; and / or calculated status). Such communication may additionally or alternatively be used to exchange other V2X vehicle status steps that may not be included in the V2X capability data elements.

[0026] In some implementations, vehicle A 180 can also communicate with vehicle B 190 via a network. This can be accomplished using radio signals 122 / 124 to / from base station 120 and / or via radio signals 132 to / from access point 130. Additionally or alternatively, such communication can be accomplished via one or more communication-enabled RSUs 125, any of which can relay communications and information and / or convert protocols for use by other vehicles, such as vehicle B 190. For example, in implementations where vehicle B 190 cannot communicate directly with vehicle A 180 using a common protocol, this functionality can be completed. In one implementation, RSU 125 may include various types of roadside beacons, traffic and / or vehicle monitors, traffic control devices, and location beacons.

[0027] In one embodiment, the RSU 125 may have a processor 125A configured to operate a radio transceiver 125E to transmit and receive radio messages (e.g., BSM, CAM, or other V2X messages) to / from vehicle A 180 and / or vehicle B 190, from base station 120 and / or access point 130. For example, the radio transceiver 125E may transmit and / or receive radio messages (such as V2X communication with vehicles) (e.g., using sidelink communication) according to various protocols, and / or communicate over wireless communication networks using various wide area network (WAN), wireless local area network (WLAN), and / or personal area network (PAN) protocols. In one embodiment, the RSU 125 may include one or more processors 125A communicatively coupled to the radio transceiver 125E and memory, and may include instructions and / or hardware to execute as a traffic control unit 125C, and / or provide and / or process environmental and roadside sensor information 125D or act as a position reference for its GNSS relative position with respect to vehicles. In one embodiment, the RSU 125 may include a network interface 125B (and / or a wireless transceiver 125E), which in one embodiment can communicate with an external server, such as a traffic optimization server 165, a vehicle information server 155, and / or an environmental data server 140. In one embodiment, the wireless transceiver 125E can communicate on a wireless communication network by transmitting or receiving wireless signals from a wireless base transceiver subsystem (BTS), a node B, an evolved Node B (eNodeB), or a next-generation node B (gNodeB) over a wireless communication link. In one embodiment, the wireless transceiver 125E may include various combinations of WAN, WLAN, and / or PAN transceivers. In one embodiment, the local transceiver may also be Bluetooth. ® Transceivers, ZigBee transceivers, or other PAN transceivers. Local transceivers, WAN wireless transceivers, and / or mobile wireless transceivers may include WAN transceivers, access points (APs), femtocell base stations, home base stations, small cell base stations, home node Bs (HNBs), home eNodeBs (HeNBs), or next-generation node Bs (gNodeBs) and may provide support for WLANs (e.g., IEEE 1102.11 networks), wireless PANs (e.g., Bluetooth), and more. ® Access to a wireless network (such as an LTE network or other wireless WAN, as discussed in the next paragraph) or a cellular network. It should be understood that these are merely examples of networks that can communicate with the RSU 125 over a wireless link, and the subject matter claimed is not limited in this respect.

[0028] RSU 125 can receive position, status, GNSS and other sensor measurements and capability information from vehicle A 180 and / or vehicle B 190, such as GNSS measurements, sensor measurements, speed, heading, position, stopping distance, priority or emergency status, and other vehicle-related information. In one embodiment, environmental information (such as road surface information / status, weather conditions, and camera information) can be collected and shared with vehicles via point-to-point or broadcast message transmission. RSU 125 can use information received from vehicle A 180 and / or vehicle B 190 via wireless transceiver 125E, environmental and roadside sensors 125D, and network information and control messages from, for example, a traffic control and optimization server 165 to coordinate and guide traffic flow and provide environmental, vehicle, safety, and notification messages to vehicle A 180 and vehicle B 190.

[0029] Processor 125A may be configured to operate network interface 125B, which in one embodiment may be connected to network 170 via backhaul, and in another embodiment may be used to communicate and coordinate with various centralized servers, such as centralized traffic control and optimization server 165, which monitors and optimizes traffic flow in an area (such as a city or urban area or region). Network interface 125B may also be used to remotely access RSU 125 for crowdsourcing vehicle data, maintenance of RSU 125 and / or coordination with other RSU 125s, or other purposes. RSU 125 may have processor 125A configured to operate traffic control unit 125C, which may be configured to process data received from vehicles (such as vehicle A 180 and vehicle B 190), such as location data, parking distance data, road condition data, signage data, and other information related to the status and location of nearby vehicles and the environment. The RSU 125 may have a processor 125A configured to acquire data from environmental and roadside sensors 125D, which may include temperature, weather, camera, pressure sensor, road sensor (e.g., for vehicle detection), accident detection, motion detection, speed detection, and other vehicle and environmental monitoring sensors.

[0030] In one implementation, the vehicle A 180 can also use short-range communications and personal networks (such as Bluetooth). ®Vehicle A 180 can communicate with mobile device 100 via WAN-related protocols, such as Wi-Fi or Zigbee, or via V2X (e.g., CV2X / sidelink communication) or other vehicle-related communication protocols, for example, in one embodiment to access a WAN and / or Wi-Fi network and / or in one embodiment to obtain sensor measurements and / or location measurements from mobile device 100. In one embodiment, vehicle A 180 can communicate with mobile device 100 via a WAN network using WAN-related protocols, such as via WAN base station 120 or using Wi-Fi directly peer-to-peer or via a Wi-Fi access point. Vehicle A 180 and / or vehicle B 190 can communicate using various communication protocols. In one embodiment, vehicle A 180 and / or vehicle B 190 can support various and multiple wireless communication modes (such as, for example, using V2X, Global System for Mobile Communications (GSM), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), High Rate Packet Data (HRPD), Wi-Fi, Bluetooth). ® (Communication protocols for WiMAX, LTE, and 5G NR access technologies).

[0031] In one implementation, vehicle A 180 can communicate via WAN base station 120 using WAN protocols or with WLAN access point 130 using WLAN protocols (such as Wi-Fi). The vehicle may also support communication using, for example, WLAN or PAN (such as Bluetooth). ® Wireless communication (such as ZigBee).

[0032] In one implementation, vehicle A 180 and / or vehicle B 190 may include one or more GNSS receivers (such as GNSS receivers) for receiving GNSS signals 112 from GNSS satellite 110 for location determination, time acquisition, and time maintenance. Various GNSS systems may be supported individually or in combination, thereby using GNSS receivers or other receivers to receive signals from BeiDou, Galileo, GLONASS, and / or GPS, and various regional navigation systems (such as Quasi-Zenith Satellite System (QZSS), Indian Regional Navigation Satellite System (IRNSS), or NavIC). Other wireless systems may be utilized, such as those dependent on beacons (such as, in one example, one or more RSUs 125, one or more WLAN access points 130, or one or more base stations 120). Various GNSS signals 112 may be combined with vehicle sensors to determine location, speed, and proximity to other vehicles (such as between vehicle A 180 and vehicle B 190).

[0033] In one embodiment, vehicle A and / or vehicle B may access GNSS measurements and / or location determined by GNSS provided by mobile device 100, which in one embodiment also has GNSS, WAN, Wi-Fi, and other communication receivers and / or transceivers. In one embodiment, vehicle A 180 and / or vehicle B 190 may access GNSS measurements (such as pseudorange measurements, Doppler measurements, and satellite IDs) and / or location determined by GNSS provided by mobile device 100 as a fallback mechanism in the event of GNSS receiver failure or providing location accuracy below a threshold level.

[0034] Vehicle A 180 and / or vehicle B 190 can access various servers on the network (such as vehicle information server 155, route server 145, location server 160, map server 150 and environmental data server 140).

[0035] The vehicle information server 155 can provide information describing various vehicles (such as antenna location, vehicle size, and vehicle capabilities), which can be used to make decisions about movement relative to nearby vehicles (such as whether they can stop or accelerate in time, whether they are in autonomous driving mode, whether they have autonomous driving capabilities, and whether they have communication capabilities). In one embodiment, the vehicle information server 155 can also provide information about vehicle size, shape, capabilities, identification, ownership, occupancy, and / or a defined location point (e.g., the location of a GNSS receiver) and the position of vehicle boundaries relative to the defined location point.

[0036] The route server 145 can receive current location and destination information and provide route planning information, map data, alternative route data and / or traffic and street condition data for vehicles.

[0037] In one implementation, location server 160 may provide location determination capabilities, transmitter signal acquisition aids (such as GNSS satellite orbit prediction information, approximate location information and / or approximate time information), transceiver almanacs (such as those containing the identifiers and locations of Wi-Fi access points and base stations), and in some implementations, additional route-related information (such as speed limits, traffic and road / building conditions).

[0038] Map server 150 can provide map data such as road location, points of interest along the road, address location along the road, road size, road speed limit, traffic conditions and / or road conditions (wet, slippery, snow / icy, etc.), and road conditions (open, under construction, accident, etc.).

[0039] In one implementation, the environmental data server 140 can provide information related to weather and / or roads, traffic information, terrain information, road quality information, speed information, and / or other relevant environmental data.

[0040] Although described separately, it is conceivable that in some implementations, the functionality of one or more of the following elements may be integrated into a single cloud-based server or collectively referred to as a single cloud-based server: transportation information server 155, route server 145, location server 160, map server 150, and environmental data server 140.

[0041] In one implementation scheme Figure 1 Vehicles 180 and 190 and mobile device 100 can communicate over network 170 via various network access points, such as WLAN access point 130 or wireless WAN base station 120 on network 170. In some implementations, vehicles 180 and 190 and mobile device 100 can also use various short-range communication mechanisms to communicate directly between devices, between vehicles, between devices and vehicles, and between vehicles and devices, without going through network 170 (such as via Bluetooth). ® Zigbee and 5G NR electrical standards).

[0042] Figure 2 An example scenario is illustrated for a traffic intersection 200 with multiple lanes and multiple traffic lights. For example... Figure 2 As shown, traffic intersection 200 may include multiple traffic lanes 201, 202, and 203 and multiple traffic lights 211, 212, and 213. In some embodiments, traffic lanes 201, 202, and 203 may be defined by road markings 210 and may each correspond to one of the traffic lights 211, 212, and 213. As described above, when vehicle 220 approaches / appears at traffic intersection 200, a cloud-based server (not shown) can use, for example, traffic camera data (e.g., video frames and / or image frames spanning multiple traffic light cycles) collected by traffic camera 230 during a predetermined time window to obtain the movement of vehicle 220 at traffic intersection 200. In some embodiments, the traffic camera data may be obtained from sources such as official traffic management agencies, city databases, third-party services, etc. In some embodiments, the predetermined time window may include multiple traffic light cycles (e.g., including several instances of a change from one red light to another).

[0043] The cloud-based server can also obtain road definition information at the traffic intersection 200 corresponding to the movement of the vehicle 220 (e.g., road markings 210, traffic lanes 201, 202 and 203, ground arrow markings 205, etc.), and can obtain traffic light information indicating the timing of traffic lights at the traffic intersection 200 corresponding to the time window in which the vehicle 220 passes through the traffic intersection 200 (e.g., information indicating the timing of traffic lights 211, 212 and 213).

[0044] For example, based on traffic camera data, if the movement of vehicle 220 remains within the boundary of traffic lane 202 when crossing traffic intersection 200, then traffic lane 202 can be linked to or associated with the movement / movement of vehicle 220 (assuming vehicle 220 is traveling straight through traffic intersection 200, such as...). Figure 2 (As shown). On the other hand, traffic light information of the traffic lights (e.g., traffic light 212) at the traffic intersection 200 corresponding to the time window in which the vehicle 220 passes through the traffic intersection 200 can be obtained (assuming that during the time window, the movement of the vehicle 220 occurs when the traffic light 212 turns green, and / or the vehicle 220 stops when the traffic light 212 turns red).

[0045] Therefore, traffic signals and lane associations for traffic lanes (e.g., traffic lane 202) and traffic lights (e.g., traffic light 212) at traffic intersection 200 can be determined by using information about the movement of vehicle 220, road boundary information, and traffic light information.

[0046] It is conceivable that, despite Figure 2 Only one vehicle 220 is shown, but the movement / movement of more than one vehicle can be obtained during one or more traffic light cycles. Furthermore, signal-lane associations can be determined at various times of day (e.g., at different times of day) to accurately capture the actual traffic conditions. For example, if the signal-lane association changes during the day (e.g., traffic intersection 200 includes tidal flow lanes), the signal-lane association can be determined at least twice a day (e.g., before and after the tidal flow change) to better represent the actual traffic situation.

[0047] As described above, in some implementations, traffic signal and lane association information can be propagated to the AV using various methods. For example, a priori provisioning methods can be used to propagate traffic signal and lane association information. Therefore, the traffic signal and lane association information can be part of a database provided within the AV. In some implementations, the traffic signal and lane association information can be pre-loaded into the AV's onboard system before the AV begins its journey. For example, the AV can download the latest traffic signal and lane association data for a specific area during routine software updates or when the vehicle is being serviced at a maintenance center.

[0048] In some implementations, location-based knowledge provisioning methods can be used to propagate traffic signal and lane association information to the vehicle (AV). For example, traffic signal and lane association information can be provided to the AV based on its current location. In some implementations, traffic signal and lane association information can be broadcast at the intersection, such as via application-layer messages. Additionally or alternatively, traffic signal and lane association information can be distributed via Dedicated Short Range Communication (DSRC) systems, cellular networks, or other wireless communication technologies. For example, when the AV approaches an intersection, a responding cloud-based server can use geofencing or a proximity-based system (e.g., via nearby RSUs) to send relevant signal and lane association data to the vehicle.

[0049] In some implementations, a request-response communication mechanism can be used to propagate traffic signal-lane association information to the AV. For example, the AV can request traffic signal-lane association information as needed, and a cloud-based server can respond with the requested data. Lane-signal association can be sent to the AV based on its location, which can be determined from Cooperative Awareness (CAM) / Basic Safety (BSM) messages (or other application layer messages) to the RSU or from location data in the vehicle via the Uu to the cloud service. For example, when the AV approaches an intersection or encounters a situation requiring signal-lane association data, it can transmit a request to the cloud-based service. Depending on the request and available data, the response can include the necessary information for that specific intersection or a broader area. In some implementations, traffic signal-lane association can be embedded in OBU application layer messages or OBU subscription services. For example, embedded information can be exchanged between the OBU and a cloud-based server or other connected devices via a communication protocol involving the real-time or near-real-time transmission of requests and reception of responses.

[0050] In some implementations, an AV capable of determining its current lane can use pre-supplied information (e.g., information obtained using a priori supply methods) or information provided on the fly (e.g., information obtained using location-based knowledge supply methods and / or request-response communication mechanisms). Conversely, an AV unable to determine its current lane can request an RSU (e.g., an RSU equipped with a camera) to help determine its current lane.

[0051] As described above, traffic signal and lane association information can be transmitted to the AV via various channels, such as wired connections, over-the-air methods (e.g., Wi-Fi downloads facilitated by OEMs or commercial cloud-based services), or over-the-air communication via Uu or V2X. In some implementations, traffic signal and lane association information can be transmitted using application-layer or lower-layer messages (e.g., RRC) from cloud-based services. In some implementations, traffic signal and lane association information can be enhanced by the detection of turn signs or turn arrows painted on the road (e.g., ground arrow sign 205) by traffic cameras.

[0052] Figure 3 This is a high-level block diagram of a traffic camera-based method for determining traffic signal and lane association for AV operation, according to the implementation scheme. The functionality illustrated in blocks 301, 302, and 303 can be performed by a cloud-based server as discussed above. Figure 3 The signal and lane association determination and propagation functionality illustrated in the examples (e.g., illustrated in boxes 301, 302, and 303) can be combined Figure 2 The functionality described herein shall be performed. Furthermore, the functionality in boxes 301, 302, and 303 may be provided by one or more computer systems (such as...). Figure 6 The computer system 600 illustrated herein executes the operation. This can be facilitated via wired connections and / or wireless communication networks (e.g., cellular / mobile communication networks and / or similar wireless wide area networks (WWAN)). Figure 3 The communication between different entities shown in the diagram (e.g., between traffic camera 310, cloud-based server, and any of the V2C devices associated with AV 330).

[0053] In box 301, the method can begin with the functionality shown in box 312, where a cloud-based server can identify intersections of interest (e.g., Figure 2 The presence of vehicles at a traffic intersection (200) in the intersection of interest. In some implementations, the presence of a vehicle at the intersection of interest can be determined when the vehicle is in or about to be in the vicinity of the intersection of interest (e.g., within a predetermined range). This can be achieved using any suitable positioning mechanism, which may be based on static positioning, proximity, or a combination of the proximity and heading of the vehicle, or the speed of the vehicle.

[0054] At box 314, a cloud-based server can use, for example, traffic camera data (such as video frames and / or image frames spanning multiple traffic light cycles) to obtain vehicle movement. Traffic camera data can be collected by traffic camera 310 during a predetermined time window and can be transmitted as shown in arrow 313. In some embodiments, traffic camera data can be obtained from sources such as official traffic management agencies, city databases, third-party services, etc. In some embodiments, the predetermined time window may include multiple traffic light cycles (e.g., including several instances of a transition from one red light to another).

[0055] At box 316, a cloud-based server can obtain road demarcation information at the intersection of interest corresponding to the movement of vehicles indicated in traffic data such as that obtained by traffic cameras (e.g., Figure 2 (Road markings 210, traffic lanes 201, 202, and 203, and ground arrow markings 205). For example, based on traffic camera data, if the movement of a vehicle remains within the boundaries of a traffic lane while crossing an intersection of interest, the traffic lane can be linked to or associated with the movement / movement of the vehicle.

[0056] At box 318, a cloud-based server can obtain traffic light information at the intersection of interest corresponding to the time window in which a vehicle crosses the intersection. For example, if a vehicle moves when the traffic light turns green during the time window, and / or stops when the traffic light turns red, the traffic light can be linked to or associated with the vehicle's movement.

[0057] At box 320, a cloud-based server can determine traffic lanes and traffic signal lane associations at an intersection of interest by using information about vehicle movement, road definition information, and traffic light information, to associate traffic lights with traffic lanes. For example, traffic lanes at an intersection of interest can be determined by associating traffic lights indicated in traffic light information with traffic lanes indicated in road definition information (as obtained in boxes 316 and 318). Figure 2 Traffic lanes 201, 202 and 203) and traffic lights (e.g., Figure 2 Traffic signals (211, 212 and 213) are associated with lanes.

[0058] At box 322, the functionality performed in boxes 312-320 can be repeated on different vehicles at different predetermined time periods and / or at different times of the day. For example, as described above, the movement / movement of more than one vehicle can be obtained during one or more traffic light cycles at various time periods of the day (e.g., at different times of the day) to accurately capture the actual traffic situation.

[0059] In box 302, a cloud-based server can determine the relevant vehicle-specific intersection / traffic light relative to the AV based on traffic signal and lane association. In some implementations, location-based knowledge can be used to determine the relevant vehicle-specific intersection / traffic light. For example, traffic signal and lane association information can be provided to the AV based on its current location. Additionally or alternatively, in some implementations, the relevant vehicle-specific intersection / traffic light can be determined in response to a request from a specific AV (e.g., in response to determining that the AV is located in a traffic lane associated with the traffic signal and lane). For example, the AV can request the relevant vehicle-specific intersection / traffic light relative to the AV as needed, and the cloud-based server can thus determine the relevant vehicle-specific intersection / traffic light.

[0060] In box 303, the cloud-based server can propagate vehicle-specific intersection / traffic light information to AV 330 based on the determination made in box 302. For example, the cloud-based server can define a Signal Distribution Information Element (IE) according to the V2X protocol, such as encapsulating the IE within a V2X payload in an application-layer message. An example of an IE definition is provided in... Figure 4 The cloud-based server can also determine whether there is more than one AV at or near the intersection of interest. In some implementations, depending on network load, the cloud-based server can propagate vehicle-specific intersection / traffic light information based on public or private signaling distribution (e.g., using PC5 or Uu interfaces).

[0061] Figure 5 This is a flowchart of a traffic camera-based method 500 for determining the association between traffic signals and lanes at a traffic intersection for automated vehicle operation, according to some implementation schemes. In some implementation schemes, a cloud-based server may correspond to... Figure 2 and Figure 3 The discussion focuses on cloud-based servers used for execution. Figure 5 The functional components / structures illustrated in one or more of the boxes shown can be implemented by hardware and / or software components of a computer system as described herein. Figure 6The example components of the computer system are illustrated below, and these example components are described in more detail below.

[0062] At box 510, functionality includes obtaining information about the movement of vehicles using video or image frames acquired by traffic cameras that cross the traffic intersection during a time window. In some embodiments, traffic camera data may be obtained from sources such as official traffic management agencies, city databases, third-party services, etc. In some embodiments, the predetermined time window may include multiple traffic light cycles (e.g., including several instances of changing from one red light to another).

[0063] Components used to perform functionality at block 510 may include bus 605, processor 610, communication subsystem 630, memory 635, and / or other components of computer system 600, such as... Figure 6 exemplified.

[0064] At box 520, functionality includes: obtaining road definition information indicating traffic lanes at a traffic intersection, wherein the road definition information corresponds to the movement of a vehicle as indicated in a video frame or image frame of the vehicle obtained by a traffic camera. For example, based on traffic camera data (e.g., video frames or image frames of a vehicle), if the movement of the vehicle remains within the boundaries of the traffic lane while crossing the intersection of interest, the traffic lane may be attached to or associated with the movement / movement of the vehicle.

[0065] Components used to perform functionality at block 520 may include bus 605, processor 610, communication subsystem 630, memory 635, and / or other components of computer system 600, such as... Figure 6 exemplified.

[0066] At box 530, the functionality includes: obtaining traffic light information for traffic lights at a traffic intersection, the traffic light information corresponding to a time window in which a vehicle crosses the traffic intersection, wherein the traffic light information indicates the timing of the traffic lights. For example, if during the time window, a vehicle moves when the traffic light turns green, and / or the vehicle stops when the traffic light turns red, then the traffic lights may be linked to or associated with the movement / movement of the vehicle.

[0067] Components for performing functionality at block 530 may include bus 605, processor 610, communication subsystem 630, memory 635, and / or other components of computer system 600, such as... Figure 6 exemplified.

[0068] At box 540, functionality includes: determining traffic signal-lane association by using information about vehicle movement, road definition information, and traffic light information to associate traffic lanes with traffic lights. For example, traffic lanes at an intersection of interest can be determined based on associating traffic lights indicated in traffic light information with traffic lanes indicated in road definition information (as obtained in boxes 520 and 530). Figure 2 Traffic lanes 201, 202 and 203) and traffic lights (e.g., Figure 2 Traffic signals (211, 212 and 213) are associated with lanes.

[0069] The functional components used to execute the block 540 may include bus 605, processor 610, communication subsystem 630, memory 635 and / or other components of computer system 600, such as Figure 6 exemplified.

[0070] At box 550, functionality includes: propagating traffic signals and lane associations to automated vehicles (AVs) for AV navigation through traffic intersections. For example, a cloud-based server could, for instance, define the signal allocation interface (IE) according to V2X protocols, such as encapsulating the IE within a V2X payload in an application-layer message. An example of the IE definition is... Figure 4 The cloud-based server can also determine whether there is more than one AV at or near the intersection of interest. In some implementations, depending on network load, the cloud-based server can propagate vehicle-specific intersection / traffic light information based on public or private signaling distribution (e.g., using PC5 or Uu interfaces).

[0071] As described above, in some implementations, traffic signal and lane association information can be propagated to the AV using various methods. For example, a priori provisioning methods can be used to propagate traffic signal and lane association information. Therefore, the traffic signal and lane association information can be part of a database provided within the AV. In some implementations, the traffic signal and lane association information can be pre-loaded into the AV's onboard system before the AV begins its journey. For example, the AV can download the latest traffic signal and lane association data for a specific area during routine software updates or when the vehicle is being serviced at a maintenance center.

[0072] In some implementations, location-based knowledge provisioning methods can be used to propagate traffic signal-lane association information to the AV. For example, traffic signal-lane association information can be provided to the AV based on its current location. In some implementations, traffic signal-lane association information can be broadcast at the intersection, such as via application-layer messages. Additionally or alternatively, traffic signal-lane association information can be distributed via DSRC systems, cellular networks, or other wireless communication technologies. For example, when the AV approaches an intersection, a responding cloud-based server can use geofencing or a proximity-based system (e.g., via nearby RSUs) to send relevant signal-lane association data to the vehicle.

[0073] In some implementations, a request-response communication mechanism can be used to propagate traffic signal-lane association information to the AV. For example, the AV can request traffic signal-lane association information as needed, and a cloud-based server can respond with the requested data. Lane-signal association can be sent to the AV based on its location, which can be determined from CAM / BSM (or other application layer messages) to the RSU or from location data in vehicles connected to the cloud service via Uu. For example, when the AV approaches an intersection or encounters a situation requiring signal-lane association data, the AV can transmit a request to the cloud-based service. Depending on the request and available data, the response can include the necessary information for that specific intersection or a broader area. In some implementations, traffic signal-lane association is embedded in OBU application layer messages or OBU subscription services. For example, embedded information can be exchanged between the OBU and a cloud-based server or other connected devices via a communication protocol involving the real-time or near-real-time transmission of requests and reception of responses.

[0074] In some implementations, an AV capable of determining its current lane can use pre-supplied information (e.g., information obtained using a priori supply methods) or information provided on the fly (e.g., information obtained using location-based knowledge supply methods and / or request-response communication mechanisms). Conversely, an AV unable to determine its current lane can request an RSU (e.g., an RSU equipped with a camera) to help determine its current lane.

[0075] As described above, traffic signal and lane association information can be transmitted to the AV via various channels, such as wired connections, over-the-air methods (e.g., Wi-Fi downloads facilitated by OEMs or commercial cloud-based services), or over-the-air communication via Uu or V2X. In some implementations, traffic signal and lane association information can be transmitted using application-layer or lower-layer messages (e.g., RRC) from cloud-based services. In some implementations, traffic signal and lane association information can be enhanced by the detection of turn signs or turn arrows painted on the road (e.g., ground arrow sign 205) by traffic cameras.

[0076] The functional components used to execute the block 550 may include bus 605, processor 610, communication subsystem 630, memory 635 and / or other components of computer system 600, such as Figure 6 exemplified.

[0077] In some implementations, as described above, information about the movement of a vehicle across a traffic intersection can be obtained in response to determining that the vehicle is present at the traffic intersection. In some implementations, the presence of a vehicle at an intersection of interest can be determined when the vehicle is in or about to be in the vicinity of the intersection of interest (e.g., within a predetermined range). This can be achieved using any suitable positioning mechanism, which may be based on static positioning, proximity, or a combination of the vehicle's proximity and heading, or the vehicle's speed.

[0078] In some implementations, propagating traffic signal-lane association may be performed in response to determining that the AV is located in the traffic lane associated with the traffic signal-lane association. In some implementations, propagating traffic signal-lane association may be performed by a cellular communication interface using a safety message. In some implementations, propagating traffic signal-lane association may be embedded in onboard unit (OBU) application layer messages or OBU subscription services.

[0079] Figure 6 This is a block diagram of an embodiment of computer system 600, which can be used, wholly or partially, to provide the functionality of one or more components and / or devices as described in the embodiments herein, including a server (e.g., a sensing server / SMF, a location server / LMF, etc.) communicating with one or more base stations and / or one or more sensing nodes to coordinate RF sensing as described in the embodiments herein. This can include, for example, computer servers, personal computers, personal electronic devices, etc. It should be noted that... Figure 6 This is intended only to provide generalized examples of various components, any or all of which may be utilized as appropriate. Therefore, Figure 6This broadly illustrates how individual system components can be implemented in a relatively separate or relatively more integrated manner. Furthermore, it can be noted that... Figure 6 The illustrated components can be localized as a single device and / or distributed among various networked devices that can be located in different geographical locations.

[0080] Computer system 600 is shown as including hardware elements electrically coupled (or otherwise communicable) via bus 605. The hardware elements may include processor 610, which may include, but is not limited to, one or more general-purpose processors, one or more special-purpose processors (such as digital signal processing chips, graphics accelerators, etc.), and / or other processing architectures, configured to perform one or more of the methods described herein. Computer system 600 may also include one or more input devices 615, which may include, but is not limited to, a mouse, keyboard, camera (e.g., a traffic camera for acquiring image / traffic camera data), microphone, etc.; and one or more output devices 620, which may include, but is not limited to, display devices, printers, etc.

[0081] The computer system 600 may also include one or more non-transitory storage devices 625 (and / or communicate with said one or more non-transitory storage devices), which may include, but are not limited to, local and / or network-accessible storage devices, and / or may include, but are not limited to, disk drives, drive arrays, optical storage devices, solid-state storage devices (such as random access memory (RAM) and / or read-only memory (ROM)), which may be programmable, flash-updatable, etc. Such storage devices may be configured to implement any suitable data storage, including but not limited to various file systems, database structures, etc. Such data storage may include databases and / or other data structures for storing and managing messages and / or other information to be transmitted via a central hub to one or more devices, as described herein.

[0082] Computer system 600 may also include a communication subsystem 630, which may include wireless communication technologies managed and controlled by wireless communication interface 633, as well as wired technologies such as Ethernet, coaxial communication, universal serial bus (USB), etc. In some embodiments, traffic camera data may be received via communication subsystem 630 (e.g., where traffic camera input is not directly fed to computer system 600). Wireless communication interface 633 may include one or more wireless transceivers that can transmit and receive wireless signals 655 (e.g., signals according to 5G NR or LTE) via wireless antenna 650. Therefore, communication subsystem 630 may include modems, network interface cards (wireless or wired), infrared communication devices, wireless communication devices, and / or chipsets, etc., which enable computer system 600 to communicate with any device (including user equipment (UE), base station and / or other transmit / receive point (TRP), and / or any other electronic device described herein) on any or all of the communication networks described herein. Therefore, communication subsystem 630 can be used to receive and transmit data as described in the embodiments herein.

[0083] In many embodiments, computer system 600 will also include working memory 635, which may include RAM or ROM devices as described above. Software elements shown residing within working memory 635 may include operating system 640, device drivers, executable libraries, and / or other code (such as one or more applications 645), which may include computer programs provided by various embodiments and / or may be designed to implement methods provided by other embodiments and / or configure systems provided by other embodiments, as described herein. By way of example only, one or more processes described with respect to the methods discussed above may be implemented as code and / or instructions executable by a computer (and / or a processor within a computer); then, in one aspect, such code and / or instructions may be used to configure and / or adapt a general-purpose computer (or other device) to perform one or more operations according to the described methods.

[0084] These sets of instructions and / or code may be stored on a non-transitory computer-readable storage medium (such as storage device 625 described above). In some cases, the storage medium may be incorporated into a computer system such as computer system 600. In other embodiments, the storage medium may be separate from the computer system (e.g., a removable medium, such as an optical disc), and / or may be provided in an installation package so that the storage medium can be used to program, configure, and / or adapt a general-purpose computer containing the instructions / code. These instructions may take the form of executable code that can be executed by computer system 600, and / or may take the form of source and / or installable code that, when compiled and / or installed on computer system 600 (e.g., using any of a variety of generally available compilers, installers, compression / decompression utilities, etc.), takes the form of executable code.

[0085] Figure 7 This is a block diagram of an embodiment of V2X device 700, which can be utilized and / or integrated into a vehicle, RSU, or any other system or device to wirelessly communicate with the vehicle and / or RSU, as previously described. When utilized by a vehicle, V2X device 700 may include or be integrated into a vehicle computer system for managing one or more systems related to the vehicle's navigation and / or automated driving, and for communicating with other onboard systems and / or other traffic entities. Furthermore, V2X device 700 may be integrated into an RSU computer system, which may include additional components and may perform additional RSU-related functionality. With this in mind, according to some embodiments, V2X device 700 may include a standalone device or component of the vehicle or RSU that can be communicatively coupled to other components / devices of the vehicle or RSU. It should also be noted that V2X entities other than vehicles or RSUs may utilize V2X device 700 in a similar manner. Additionally, embodiments may not be limited to V2X communication. Thus, alternative implementations may include equipment similar to V2X device 700, having the same... Figure 7 The components shown are similar to those in the previous discussion and are capable of performing the functions of the vehicles and / or RSUs described in the previously discussed embodiments, but do not have V2X functionality.

[0086] It should also be pointed out that, Figure 7 This is intended only to provide generalized examples of various components; any or all of these components may be used as appropriate. It can be noted that in some instances, [the components are...]. Figure 7 The illustrated components can be localized into a single physical device and / or distributed among various networked devices, for example, they can be located at different physical locations on vehicles, RSUs or other V2X entities.

[0087] V2X device 700 is shown as including hardware elements that can be electrically coupled (or otherwise communicated) via bus 705. The hardware elements may include processing unit 710, which may include, but is not limited to, one or more general-purpose processors, one or more special-purpose processors (such as DSP chips, graphics accelerator processors, application-specific integrated circuits (ASICs), etc.) and / or other processing structures or components.

[0088] V2X device 700 may also include one or more input devices 770, which may include devices related to user interface (e.g., touch screen, touchpad, microphone, buttons, dial pad, switch, etc.) and / or devices related to navigation, autonomous driving, etc. Similarly, one or more output devices 715 may involve devices that interact with the user (e.g., via display, light-emitting diode (LED), speaker, etc.) and / or devices related to navigation, autonomous driving, etc.

[0089] V2X device 700 may also include wireless communication interface 730, which may include, but is not limited to, modems, network interface cards, infrared communication devices, wireless communication devices and / or chipsets (such as Bluetooth). ® Devices, including IEEE 802.11 devices, IEEE 802.15.4 devices, WiFi devices, WiMax (WiMax Global Access) devices, wide area network (WAN) devices, and / or various cellular devices, etc. The wireless communication interface 730 enables the V2X device 700 to communicate with other V2X devices. This can include various forms of communication as described in the previously described embodiments, including... Figure 4 A, Figure 4 B and Figure 5 The message passing is illustrated in the diagram. Furthermore, it can be used to send direct communication, broadcast wireless signals, receive direct and / or broadcast wireless signals, and so on. Therefore, the wireless communication interface 730 can transmit and / or receive RF signals from various RF channels / bands. Communication using the wireless communication interface 730 can be performed via one or more wireless communication antennas 732 that transmit and / or receive wireless signals 734. According to some embodiments, the wireless communication antennas 732 may include multiple discrete antennas, antenna arrays, or any combination thereof.

[0090] V2X device 700 may further include sensor 740. Sensor 740 may include, but is not limited to, one or more inertial sensors and / or other sensors (e.g., accelerometers, gyroscopes, cameras, magnetometers, altimeters, microphones, proximity sensors, light sensors, barometers, etc.). For example, sensor 740 may be used to determine certain real-time characteristics of the vehicle, such as position, motion state (speed, acceleration), etc. As previously indicated, sensor 740 may be used to help the vehicle determine its position.

[0091] Implementations of the V2X device 700 may also include a Global Navigation Satellite System (GNSS) receiver 780, which is capable of receiving signals 784 from one or more GNSS satellites using an antenna 782 (which may be identical to antenna 732 in some implementations). Positioning based on GNSS signal measurements can be used to determine the current location of the V2X device 700 and can further serve as a basis for determining the location of detected objects. The GNSS receiver 780 can extract the positioning of the V2X device 700 from GNSS satellites of GNSS systems such as the Global Positioning System (GPS) and / or similar satellite systems using conventional techniques.

[0092] V2X device 700 may further include memory 760 and / or be in communication with memory 760. Memory 760 may include, but is not limited to, local and / or network-accessible storage devices, disk drives, drive arrays, optical storage devices, solid-state storage devices such as random access memory (RAM) and / or read-only memory (ROM), which may be programmable, flash-updatable, etc. Such storage devices may be configured to implement any suitable data storage, including but not limited to various file systems, database structures, etc.

[0093] The memory 760 of the V2X device 700 may also include software elements ( Figure 7 (Not shown in the document), these software elements include operating systems, device drivers, executable libraries, and / or other code (such as one or more applications). These software elements may include computer programs provided by various embodiments, and / or may be designed to implement the methods described herein, and / or configure the systems described herein. Software applications stored in memory 760 and executed by processing unit 710 can be used to implement the functionality of a vehicle or RSU as described herein. Furthermore, one or more processes described with respect to the methods discussed herein can be implemented as code and / or instructions in memory 760 executable by V2X device 700 (and / or processing unit 710 or DSP 720 within V2X device 700), including... Figure 6 and Figure 7The functions exemplified in the method. In one respect, such code and / or instructions can be used to configure and / or adapt a general-purpose computer (or other device) to perform one or more operations according to the described method.

[0094] It will be apparent to those skilled in the art that basic variations can be made to suit specific requirements. For example, custom hardware may be used, and / or specific elements may be implemented in hardware, software (including portable software such as applets), or both. Furthermore, connections to other computing devices, such as network input / output devices, may be employed.

[0095] Referring to the accompanying drawings, components that may include memory may include non-transitory machine-readable media. As used herein, the terms "machine-readable media" and "computer-readable media" refer to any storage medium that participates in providing data that enables a machine to operate in a particular manner. In the embodiments provided above, various machine-readable media may be involved in providing instructions / code to a processor and / or other devices for execution. Additionally or alternatively, machine-readable media may be used to store and / or carry such instructions / code. In many specific embodiments, computer-readable media are physical and / or tangible storage media. Such media may take many forms, including but not limited to non-volatile and volatile media. Common forms of computer-readable media include, for example: magnetic and / or optical media, any other physical media with a hole pattern, RAM, programmable ROM (PROM), erasable PROM (EPROM), FLASH-EPROM, any other memory chip or memory cartridge, or any other medium from which a computer can read instructions and / or code.

[0096] The methods, systems, and apparatus discussed herein are examples. Various embodiments may omit, substitute, or add various processes or components as appropriate. For example, features described for some embodiments may be combined in various other embodiments. Different aspects and elements of embodiments may be combined in a similar manner. The various components in the accompanying drawings provided herein may be embodied in hardware and / or software. Furthermore, technology evolves, and therefore many elements are examples that do not limit the scope of this disclosure to those particular examples.

[0097] It has been proven convenient to sometimes refer to such signals as bits, information, values, elements, symbols, characters, variables, items, numbers, numerical symbols, etc., primarily for common use. However, it should be understood that all such terms or similar terms should be associated with appropriate physical quantities and are merely convenient labels. Unless otherwise specifically stated, it is as apparent from the above discussion that throughout this specification, discussions using terms such as “processing,” “calculating,” “determining,” “identifying,” “ascertaining,” “identifying,” “associating,” “measuring,” and “executing” refer to the actions or processes of a specific device such as a dedicated computer or similar dedicated electronic computing device. Therefore, in the context of this specification, a dedicated computer or similar dedicated electronic computing device is capable of manipulating or transforming signals, generally referred to as physical, electronic, electrical, or magnetic quantities in the memory, registers, or other information storage devices, transmitting devices, or display devices of the dedicated computer or similar dedicated electronic computing device.

[0098] As used herein, the terms “and” and “or” may include a variety of meanings, which are also contemplated, at least in part, depending on the context in which such terms are used. Generally, “or,” when used in relation to a list such as A, B, or C, is intended to mean A, B, and C (in the inclusive sense) and A, B, or C (in the exclusive sense). Furthermore, as used herein, the term “one or more” can be used to describe any feature, structure, or characteristic in the singular form, or to describe some combination of features, structures, or characteristics. However, it should be noted that this is merely an illustrative example, and the claimed subject matter is not limited to this example. Additionally, the term “at least one of…” when used in relation to a list such as A, B, or C can be interpreted as meaning any combination of A, B, and / or C, such as A, AB, AA, AAB, AABBCCC, etc.

[0099] Several embodiments have been described, and various modifications, alternative constructions, and equivalents may be used without departing from the scope of this disclosure. For example, the above elements may be components of a larger system, where other rules may take precedence over the application of various embodiments or otherwise modify the application of various embodiments. Furthermore, multiple steps may be performed before, during, or after considering the above elements. Therefore, the above description does not limit the scope of this disclosure.

[0100] Given this description, different implementations may include different combinations of features. Specific implementation examples are described in the following numbered clauses:

[0101] Clause 1. An example method for determining traffic signal and lane association at a traffic intersection for automated vehicle operation, the method being executed by a server and potentially including: obtaining information about the movement of a vehicle using video frames or image frames acquired by the traffic camera that pass through the traffic intersection during a time window; and obtaining road definition information indicating a traffic lane at the traffic intersection, wherein the road definition information corresponds to the movement of the vehicle as indicated in the video frames or image frames acquired by the traffic camera. The method may further include: obtaining traffic light information for a traffic light at the traffic intersection, the traffic light information corresponding to the time window in which the vehicle passes through the traffic intersection, wherein the traffic light information indicates the timing of the traffic light; and determining the traffic signal and lane association by using the information about the vehicle's movement, the road definition information, and the traffic light information to associate the traffic lane with the traffic light. The method may further include: propagating the traffic signal in association with the lane to an automated vehicle (AV) for the AV to navigate through the traffic intersection.

[0102] Clause 2. The method according to Clause 1, wherein the information regarding the movement of the vehicle through the traffic intersection is obtained in response to determining that the vehicle is present at the traffic intersection.

[0103] Clause 3. The method according to any one of Clauses 1 or 2, wherein the determination that the vehicle is present at the traffic intersection is based on: the static location of the vehicle; the proximity of the vehicle to the traffic intersection; the speed of the vehicle; or any combination thereof.

[0104] Clause 4. The method according to any one of Clauses 1 to 3, wherein the propagation of the traffic signal and lane association is performed based on: a priori provisioning method; a location-based knowledge provisioning method; a request-response communication mechanism; or any combination thereof.

[0105] Clause 5. The method according to any one of Clauses 1 to 4, wherein the propagation of the traffic signal and lane association is in response to determining that the AV is located in the traffic lane associated with the traffic signal and lane.

[0106] Clause 6. The method of any one of Clauses 1 to 5, in which the traffic signal is propagated in association with the lane, is performed by a cellular communication interface using a security message.

[0107] Clause 7. The method according to any one of Clauses 1 to 6, wherein the traffic signal and lane association are embedded in an onboard unit (OBU) application layer message or an OBU subscription service.

[0108] Clause 8. The method according to any one of Clauses 1 to 7, wherein the time window includes at least one traffic light cycle.

[0109] Clause 9. An example server for traffic signal and lane association based on a traffic camera at a traffic intersection for automating vehicle operation, the radar unit comprising: a transceiver; a memory; and one or more processors communicatively coupled to the transceiver and the memory. The one or more processors may be configured to: obtain information about the motion of a vehicle using video frames or image frames acquired by the traffic camera that pass through the traffic intersection during a time window; and obtain road definition information indicating traffic lanes at the traffic intersection, wherein the road definition information corresponds to the motion of the vehicle as indicated in the video frames or image frames acquired by the traffic camera. The one or more processors may also be configured to: obtain traffic signal information at the traffic intersection, the traffic signal information corresponding to the time window in which the vehicle passes through the traffic intersection, wherein the traffic signal information indicates the timing of the traffic signal; and determine the traffic signal-lane association by using the information about the movement of the vehicle, the road boundary information, and the traffic signal information to associate the traffic lane with the traffic signal. The one or more processors may also be configured to: propagate the traffic signal-lane association to an automated vehicle (AV) for the AV to navigate through the traffic intersection.

[0110] Clause 10. The server as described in Clause 9, wherein information regarding the movement of the vehicle through the traffic intersection is obtained in response to determining that the vehicle is present at the traffic intersection.

[0111] Clause 11. The server pursuant to any one of Clauses 9 or 10, wherein the determination that the vehicle is present at the traffic intersection is based on: the static location of the vehicle; the proximity of the vehicle to the traffic intersection; the speed of the vehicle; or any combination thereof.

[0112] Clause 12. A server pursuant to any one of Clauses 9 to 11, wherein the propagation of the traffic signal and lane association is performed based on: a priori provisioning methods; location-based knowledge provisioning methods; request-response communication mechanisms; or any combination thereof.

[0113] Clause 13. A server according to any one of Clauses 9 to 12, wherein the propagation of the traffic signal and lane association is in response to determining that the AV is located in the traffic lane associated with the traffic signal and lane.

[0114] Clause 14. The server, pursuant to any one of Clauses 9 to 13, transmits the traffic signal and lane association by means of a security message via a cellular communication interface.

[0115] Clause 15. A server pursuant to any one of Clauses 9 to 14, wherein the traffic signal and lane association are embedded in an Onboard Unit (OBU) application layer message or an OBU subscription service.

[0116] Clause 16. An example apparatus for determining traffic signal-lane association at a traffic intersection for automating vehicle operation, the apparatus comprising: components for obtaining information about the movement of a vehicle using video frames or image frames obtained by the traffic camera during a time window; and components for obtaining road definition information indicating a traffic lane at the traffic intersection, wherein the road definition information corresponds to the movement of the vehicle as indicated in the video frames or image frames obtained by the traffic camera. The apparatus may further comprise: components for obtaining traffic light information of a traffic light at the traffic intersection, the traffic light information corresponding to the time window in which the vehicle passes through the traffic intersection, wherein the traffic light information indicates the timing of the traffic light; and components for determining the traffic signal-lane association by using the information about the vehicle's movement, the road definition information, and the traffic light information to associate the traffic lane with the traffic light. The device may further include components for transmitting the traffic signal in association with the lane to an automated vehicle (AV) for the AV to navigate through the traffic intersection.

[0117] Clause 17. The apparatus according to Clause 16, wherein information regarding the movement of the vehicle through the traffic intersection is obtained in response to determining that the vehicle is present at the traffic intersection.

[0118] Clause 18. The device according to any one of Clauses 16 or 17, wherein the determination that the vehicle is present at the traffic intersection is based on: the static position of the vehicle; the proximity of the vehicle to the traffic intersection; the speed of the vehicle; or any combination thereof.

[0119] Clause 19. The apparatus according to any one of Clauses 16 to 18, wherein the propagation of the traffic signal and lane association is performed based on: a priori provisioning method; a location-based knowledge provisioning method; a request-response communication mechanism; or any combination thereof.

[0120] Clause 20. The apparatus according to any one of Clauses 16 to 19, wherein the transmission of the traffic signal and lane association is in response to determining that the AV is located in the traffic lane associated with the traffic signal and lane.

[0121] Clause 21. In any one of Clauses 16 to 20, the transmission of the traffic signal and lane association is performed by a cellular communication interface using a security message.

[0122] Clause 22. The apparatus according to any one of Clauses 16 to 21, wherein the traffic signal and lane association are embedded in an onboard unit (OBU) application layer message or an OBU subscription service.

[0123] Clause 23. The apparatus according to any one of Clauses 16 to 22, wherein the time window comprises at least one traffic light cycle.

[0124] Clause 24. An example non-transitory computer-readable medium storing traffic camera-based determination instructions for associating traffic signals with lanes at a traffic intersection for automating vehicle operation, the instructions may include code for: obtaining information about the movement of a vehicle using video frames or image frames obtained by the traffic camera during a time window; and obtaining road definition information indicating traffic lanes at the traffic intersection, wherein the road definition information corresponds to the movement of the vehicle as indicated in the video frames or image frames obtained by the traffic camera. The instructions may also include code for: obtaining traffic light information for traffic lights at the traffic intersection, the traffic light information corresponding to the time window in which the vehicle passes through the traffic intersection, wherein the traffic light information indicates the timing of the traffic lights; and determining the traffic signal-lane association by using the information about the vehicle's movement, the road definition information, and the traffic light information to associate the traffic lanes with the traffic lights. The instructions may also include code for propagating the traffic signal in association with the lane to an automated vehicle (AV) for the AV to navigate through the traffic intersection.

[0125] Clause 25. The non-transitory computer-readable medium as described in Clause 24, wherein information regarding the movement of the vehicle through the traffic intersection is obtained in response to determining that the vehicle is present at the traffic intersection.

[0126] Clause 26. A non-transitory computer-readable medium pursuant to any one of Clauses 24 or 25, wherein the determination that the vehicle is present at the traffic intersection is based on: the static location of the vehicle; the proximity of the vehicle to the traffic intersection; the speed of the vehicle; or any combination thereof.

[0127] Clause 27. A non-transitory computer-readable medium pursuant to any one of Clauses 24 to 26, wherein the propagation of the traffic signal and lane association is performed based on: an a priori provisioning method; a location-based knowledge provisioning method; a request-response communication mechanism; or any combination thereof.

[0128] Clause 28. A non-transitory computer-readable medium according to any one of Clauses 24 to 27, wherein the propagation of the traffic signal and lane association is in response to determining that the AV is located in the traffic lane associated with the traffic signal and lane.

[0129] Clause 29. A non-transitory computer-readable medium according to any one of Clauses 24 to 28, wherein the propagation of the traffic signal and lane association is performed by a cellular communication interface using security messages.

[0130] Clause 30. A non-transitory computer-readable medium pursuant to any one of Clauses 24 to 29, wherein the traffic signal and lane association are embedded in an onboard unit (OBU) application layer message or an OBU subscription service.

Claims

1. A traffic camera-based method for determining the association between traffic signals and lanes at a traffic intersection for automated vehicle operation, the method being executed by a server and comprising: Information about the movement of the vehicle is obtained by using video frames or image frames of a vehicle passing through the traffic intersection during a time window, acquired by a traffic camera. Obtain road definition information indicating traffic lanes at the traffic intersection, wherein the road definition information corresponds to the movement of the vehicle as indicated in the video frame or image frame of the vehicle obtained by the traffic camera; Obtain traffic signal information of the traffic lights at the intersection, the traffic signal information corresponding to the time window in which the vehicle passes through the intersection, wherein the traffic signal information indicates the timing of the traffic lights; The traffic lane is associated with the traffic signal by using the information about the movement of the vehicle, the road definition information, and the traffic light information. as well as The traffic signals are associated with lanes and propagated to automated vehicles (AVs) for the AVs to navigate through the traffic intersection.

2. The method of claim 1, wherein the information regarding the movement of the vehicle through the traffic intersection is obtained in response to determining that the vehicle is present at the traffic intersection.

3. The method of claim 1, wherein the determination that the vehicle is present at the traffic intersection is based on: The static positioning of the vehicle; The proximity of the vehicle to the traffic intersection; The speed of the vehicle; or Any combination of them.

4. The method of claim 1, wherein the propagation of the traffic signal and lane association is performed based on the following: Prior supply method; Location-based knowledge provisioning methods; Request-response communication mechanism; or Any combination of them.

5. The method of claim 1, wherein propagating the traffic signal and lane association is in response to determining that the AV is located in the traffic lane associated with the traffic signal and lane.

6. The method of claim 1, wherein the propagation of the traffic signal and lane association is performed by a cellular communication interface using a security message.

7. The method of claim 1, wherein the traffic signal and lane association are embedded in an onboard unit (OBU) application layer message or an OBU subscription service.

8. The method of claim 1, wherein the time window comprises at least one traffic light cycle.

9. A traffic camera-based server for determining traffic signal and lane association at a traffic intersection for use in automated vehicle operation, the server comprising: transceiver; Memory; and One or more processors, said one or more processors being communicatively coupled to said transceiver and said memory, said one or more processors being configured to: Information about the movement of the vehicle is obtained by using video frames or image frames of a vehicle passing through the traffic intersection during a time window, acquired by a traffic camera. Obtain road definition information indicating traffic lanes at the traffic intersection, wherein the road definition information corresponds to the movement of the vehicle as indicated in the video frame or image frame of the vehicle obtained by the traffic camera; Obtain traffic signal information of the traffic lights at the intersection, the traffic signal information corresponding to the time window in which the vehicle passes through the intersection, wherein the traffic signal information indicates the timing of the traffic lights; The traffic lane is associated with the traffic signal by using the information about the movement of the vehicle, the road definition information, and the traffic light information. as well as The traffic signals are associated with lanes and propagated to automated vehicles (AVs) for the AVs to navigate through the traffic intersection.

10. The server of claim 9, wherein the information regarding the movement of the vehicle through the traffic intersection is obtained in response to determining that the vehicle is present at the traffic intersection.

11. The server of claim 9, wherein the determination that the vehicle is present at the traffic intersection is based on: The static positioning of the vehicle; The proximity of the vehicle to the traffic intersection; The speed of the vehicle; or Any combination of them.

12. The server of claim 9, wherein the propagation of the traffic signal and lane association is performed based on the following: Prior supply method; Location-based knowledge provisioning methods; Request-response communication mechanism; or Any combination of them.

13. The server of claim 9, wherein the propagation of the traffic signal and lane association is in response to determining that the AV is located in the traffic lane associated with the traffic signal and lane.

14. The server of claim 9, wherein the propagation of the traffic signal and lane association is performed by a cellular communication interface using security messages.

15. The server of claim 9, wherein the traffic signal and lane association are embedded in an onboard unit (OBU) application layer message or an OBU subscription service.

16. A traffic camera-based device for determining traffic signals and lane association at a traffic intersection for use in automated vehicle operation, the device comprising: A component for obtaining information about the movement of a vehicle by using video frames or image frames of a vehicle passing through the traffic intersection during a time window, obtained by a traffic camera; Components for obtaining road definition information indicating traffic lanes at the traffic intersection, wherein the road definition information corresponds to the movement of the vehicle as indicated in the video frame or image frame of the vehicle obtained by the traffic camera; A component for obtaining traffic signal information at the traffic intersection, the traffic signal information corresponding to the time window in which the vehicle passes through the traffic intersection, wherein the traffic signal information indicates the timing of the traffic signal; A component for determining the association of a traffic signal with a lane by using the information about the movement of the vehicle, the road definition information, and the traffic light information to associate the traffic lane with the traffic light; and Components for transmitting the traffic signals in lane association to automated vehicles (AVs) for the AVs to navigate through the traffic intersection.

17. The apparatus of claim 16, wherein the information regarding the movement of the vehicle through the traffic intersection is obtained in response to determining that the vehicle is present at the traffic intersection.

18. The apparatus of claim 16, wherein the determination that the vehicle is present at the traffic intersection is based on: The static positioning of the vehicle; The proximity of the vehicle to the traffic intersection; The speed of the vehicle; or Any combination of them.

19. The apparatus of claim 16, wherein the propagation of the traffic signal and lane association is performed based on the following: Prior supply method; Location-based knowledge provisioning methods; Request-response communication mechanism; or Any combination of them.

20. The apparatus of claim 16, wherein propagating the traffic signal and lane association is in response to determining that the AV is located in the traffic lane associated with the traffic signal and lane.

21. The apparatus of claim 16, wherein the propagation of the traffic signal and lane association is performed by a cellular communication interface using a security message.

22. The apparatus of claim 16, wherein the traffic signal and lane association are embedded in an onboard unit (OBU) application layer message or an OBU subscription service.

23. The apparatus of claim 16, wherein the time window comprises at least one traffic light cycle.

24. A non-transitory computer-readable medium storing traffic camera-based determination instructions for associating traffic signals with lanes at a traffic intersection for use in the operation of automated vehicles, the instructions comprising codes for: Information about the movement of the vehicle is obtained by using video frames or image frames of a vehicle passing through the traffic intersection during a time window, acquired by a traffic camera. Obtain road definition information indicating traffic lanes at the traffic intersection, wherein the road definition information corresponds to the movement of the vehicle as indicated in the video frame or image frame of the vehicle obtained by the traffic camera; Obtain traffic signal information of the traffic lights at the intersection, the traffic signal information corresponding to the time window in which the vehicle passes through the intersection, wherein the traffic signal information indicates the timing of the traffic lights; The traffic lane is associated with the traffic signal by using the information about the movement of the vehicle, the road definition information, and the traffic light information. as well as The traffic signals are associated with lanes and propagated to automated vehicles (AVs) for the AVs to navigate through the traffic intersection.

25. The computer-readable medium of claim 24, wherein the information regarding the movement of the vehicle through the traffic intersection is obtained in response to determining that the vehicle is present at the traffic intersection.

26. The computer-readable medium of claim 24, wherein the determination that the means of transport is present at the traffic intersection is based on: The static positioning of the vehicle; The proximity of the vehicle to the traffic intersection; The speed of the vehicle; or Any combination of them.

27. The computer-readable medium of claim 24, wherein the propagation of the traffic signal and lane association is performed based on the following: Prior supply method; Location-based knowledge provisioning methods; Request-response communication mechanism; or Any combination of them.

28. The computer-readable medium of claim 24, wherein propagating the traffic signal-lane association is in response to determining that the AV is located in the traffic lane associated with the traffic signal-lane association.

29. The computer-readable medium of claim 24, wherein the propagation of the traffic signal and lane association is performed by a cellular communication interface using security messages.

30. The computer-readable medium of claim 24, wherein the traffic signal and lane association are embedded in an onboard unit (OBU) application layer message or an OBU subscription service.

Images