Traffic light and road association method and system

Abstract

The embodiment of the invention provides a traffic light and road association method and system. The method comprises the following steps: acquiring traffic light data corresponding to traffic lights in a preset range; inputting the traffic light data into a correlation model, and extracting traffic light features, the correlation model being a machine learning model; obtaining road data corresponding to roads in a preset range; inputting road data into the correlation model, and extracting road structure features; and based on the traffic light characteristics and the road structure characteristics, determining an association relationship between the traffic light and the road by using an association model.

Description

Technical Field

[0001] This specification relates to the field of intelligent driving, and particularly to a method and system for associating traffic lights with roads. Background Art

[0002] In the scenario of autonomous driving, a vehicle needs to determine the traffic state of a road so as to determine a moving strategy. Currently, it is a relatively common method to determine the association relationship between traffic lights and roads through a navigation map, and then determine the traffic state of the road. Among them, the navigation map can include a standard map (SD Map), a lane-level map (LD Map), and a high-precision map (HD Map). However, when the real world is inconsistent with the navigation map (for example, the navigation map does not represent the roads in the real world or lacks high-precision traffic light information), this method will not be able to obtain the association relationship between traffic lights and roads, and thus will not be able to determine the moving strategy.

[0003] Therefore, it is necessary to provide a method and system for associating traffic lights with roads, which can obtain the association relationship between traffic lights and roads without relying on a navigation map, so as to determine the moving strategy of a vehicle. Summary of the Invention

[0004] One embodiment of this specification provides a method for associating traffic lights with roads. The method for associating traffic lights with roads includes: obtaining traffic light data corresponding to traffic lights within a preset range; inputting the traffic light data into an association model to extract traffic light features, where the association model is a machine learning model; obtaining road data corresponding to roads within the preset range; inputting the road data into the association model to extract road structure features; and determining the association relationship between traffic lights and roads based on the traffic light features and the road structure features by using the association model.

[0005] In some embodiments, obtaining road data corresponding to roads within a preset range includes: obtaining road information provided in a navigation map; obtaining environmental data within the preset range, where the environmental data includes roads, and the environmental data is collected by a vehicle's sensor at the current time, and the association relationship corresponds to the current time; and determining road data based on the road information and / or the environmental data.

[0006] In some embodiments, obtaining traffic light data corresponding to traffic lights within a preset range includes: obtaining traffic light information provided in a navigation map; obtaining environmental data within the preset range, where the environmental data includes traffic lights, and the environmental data is collected by a vehicle's sensor at the current time, and the association relationship corresponds to the current time; obtaining historical environmental data, where the historical environmental data includes traffic lights, and the historical environmental data is collected by a vehicle's sensor within a first historical time period, and the first historical time period is before the current time; and determining traffic light data based on at least one of the traffic light information, the environmental data, and the historical environmental data.

[0007] In some embodiments, the input of the association model further includes the historical association relationship between the traffic lights and the road corresponding to the second historical time period, and the second historical time period is before the current time.

[0008] In some embodiments, the association relationship includes the control probability of the traffic lights for each preset area of the road.

[0009] In some embodiments, the association relationship further includes the traffic state corresponding to the current time for each preset area.

[0010] In some embodiments, the association relationship further includes the predicted traffic state of each preset area in a future time period, and the future time period is after the current time.

[0011] In some embodiments, the method further includes: obtaining the position of the vehicle at a future time, where the future time is within the future time period; and determining the movement strategy of the vehicle according to the position, the future time, and the association relationship.

[0012] In some embodiments, the association model includes a model based on an attention mechanism.

[0013] One embodiment of this specification provides a traffic light and road association system, which includes: a first acquisition module for acquiring traffic light data corresponding to traffic lights within a preset range; a first extraction module for inputting the traffic light data into an association model to extract traffic light features, where the association model is a machine learning model; a second acquisition module for acquiring road data corresponding to roads within a preset range; a second extraction module for inputting the road data into the association model to extract road structure features; and a determination module for determining the association relationship between the traffic lights and the road based on the traffic light features and the road structure features by using the association model.

[0014] One embodiment of this specification provides a traffic light and road association device, including a processor for executing the traffic light and road association method.

[0015] One embodiment of this specification provides a computer-readable storage medium storing computer instructions, and when a computer reads the computer instructions in the storage medium, the computer executes the traffic light and road association method. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] This specification will be further described by way of exemplary embodiments, which will be described in detail through the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same numbers represent the same structures, where:

[0017] Figure 1It is a schematic diagram of the application scenario of the traffic light and road association system shown in some embodiments of this specification;

[0018] Figure 2 It is a module diagram of the traffic light and road association system shown in some embodiments of this specification;

[0019] Figure 3 It is an exemplary flowchart of the association between the traffic light and the road shown in some embodiments of this specification;

[0020] Figure 4 It is a schematic diagram of the process of determining the association relationship based on the association model shown in some embodiments of this specification;

[0021] Figure 5 It is an exemplary flowchart of determining the movement strategy of the vehicle shown in some embodiments of this specification. Detailed implementation manners

[0022] To more clearly illustrate the technical solutions of the embodiments of this specification, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the figures represent the same structure or operation.

[0023] It should be understood that the "system", "device", "unit" and / or "module" used herein is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the said words can be replaced by other expressions.

[0024] As shown in this specification and the claims, unless the context clearly indicates an exceptional situation, words such as "a", "an", "one" and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0025] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the operations before or after do not necessarily need to be executed precisely in sequence. On the contrary, they can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several operations can be removed from these processes.

[0026] This specification applies to both online and offline scenarios.

[0027] For the online scenario, the traffic light and road association method is executed by the vehicle's processing device. During the vehicle's driving process, traffic light data and road data can be obtained in real time at the vehicle's sensors. The vehicle's processing device can execute the traffic light and road association method on the traffic light data and road data at the current moment, thereby obtaining the association relationship between the traffic light and the road at the current moment. Among them, the traffic light data is determined based on the traffic light information provided by the navigation map, the environmental information obtained by the vehicle's sensors, and / or the historical environmental data obtained by the vehicle's sensors; the road data is determined based on the road information provided by the navigation map and / or the environmental information obtained by the vehicle's sensors. The association relationship may include the control probability of the traffic light for each preset area of the road and the corresponding traffic state of each preset area at the current moment. Further, the processing device can determine the vehicle's movement strategy (e.g., driving or stopping) in real time online based on the corresponding traffic state of each preset area at the current moment.

[0028] For the offline scenario, the traffic light and road association method is executed by the server. The server can obtain traffic light data and road data and execute the traffic light and road association method periodically at a preset time interval (e.g., every hour, every day, every week, etc.), thereby obtaining the association relationship between the traffic light and the road periodically. Among them, the traffic light data is determined based on the traffic light information provided by the navigation map and the environmental information obtained by the sensors of a large number of vehicles; the road data is determined based on the road information provided by the navigation map and the environmental information obtained by the sensors of a large number of vehicles. The preset time interval can be set manually. The association relationship may include the control probability of the traffic light for each preset area of the road. Further, the server can send the above control probability to each vehicle. During the driving process, the vehicle's processing device can determine the corresponding traffic state of each preset area at the current moment online based on the above control probability and the real-time state of the traffic light monitored online, thereby determining the vehicle's movement strategy in real time online. For more content about the movement strategy, see Figure 5 and its related descriptions.

[0029] Figure 1 is a schematic diagram of application scenario 100 of the traffic light and road association system according to some embodiments of this specification. The systems and methods in this application can be applied to vehicle movement control scenarios, such as autonomous driving scenarios (e.g., pure driverless scenarios without a driver, scenarios with an assisted driving module in the vehicle, etc.), robot scenarios, etc.

[0030] The application scenario 100 of the traffic light and road association system is applicable to the above online and offline scenarios. In some embodiments, the application scenario 100 of the traffic light and road association system may include a server 110, a network 120, a road 130, a traffic light 140, and a vehicle 150. In some embodiments, the application scenario 100 of the traffic light and road association system may further include a storage device and / or a user terminal (not shown in the figure).

[0031] In some embodiments, in the offline scenario, the server 110 may be used to process information and / or data related to the traffic light and road association system. In some embodiments, the server 110 may be used to obtain traffic light data and road data, and periodically execute the traffic light and road association method at a preset time interval, so as to periodically obtain the association relationship between the traffic light and the road. Further, the server 110 may send the above control probability to each vehicle 150.

[0032] In some embodiments, the server 110 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction processor (ASIP), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, etc., or any combination of the above.

[0033] In some embodiments, the network 120 may include any suitable wired or wireless network that facilitates information and / or data exchange. For example, the server 110 may obtain environmental data collected by the sensor 150-1 through the network 120. In some embodiments, the server 110, the vehicle 150, and / or the storage device, etc., may transmit information and / or data to one or more other components based on the traffic light and road association system via the network 120.

[0034] The road 130 refers to a linear traffic channel for vehicles, pedestrians, and / or animals, etc. to pass through. In some embodiments, the road 130 may include highways, streets, expressways, country roads, etc. In some embodiments, the road 130 may include straight roads, curved roads, ramps, turning roads, and branch roads, etc.

[0035] The traffic light 140 refers to a traffic signal device installed at intersections or on roads, which is used to indicate and control traffic flow to ensure traffic order and safety. In some embodiments, the traffic light 140 may include a basic traffic light, a countdown traffic light, an arrow light, a pedestrian signal light, a vehicle priority light, an intelligent traffic light, a temporary traffic light, etc. Among them, the basic traffic light uses red, yellow, and green lights to indicate traffic flow. The countdown traffic light adds the function of displaying the remaining time on the basis of the basic traffic light. This traffic light uses numbers or bar displays to show the remaining time of the red or green light, enabling drivers and pedestrians to better master the time for action. The arrow light is used to indicate that vehicles should pass in the direction indicated by the arrow to ensure smooth traffic flow and driving safety. The pedestrian signal light is specifically used to indicate the passage of pedestrians. The vehicle priority light is used to give specific vehicles or buses the opportunity to pass first. The intelligent traffic light is used to automatically adjust the switching of lights and the time interval according to real-time traffic conditions to improve traffic efficiency and reduce congestion. The temporary traffic light is a traffic signal installed at temporary construction sites, events, or in emergency situations to guide traffic flow to ensure the safety of the construction site or event area.

[0036] The vehicle 150 refers to an object that can move autonomously or semi-autonomously. The vehicles in this application may include various vehicles traveling on roads, such as taxis, private cars, carpooling vehicles, shared vehicles, buses, trucks, passenger vehicles, work vehicles (such as sanitation vehicles, etc.), robots (such as food delivery robots, inspection robots, express delivery robots), etc., or any combination thereof.

[0037] In some embodiments, a sensor 150-1 may be provided on the vehicle 150. In some embodiments, a processing device 150-2 may be provided on the vehicle 150.

[0038] The sensor 150-1 refers to a device that can be used to collect environmental data. Among them, the environmental data may include image data reflecting environmental information within a preset range. In some embodiments, the sensor 150-1 may include a structured light depth camera, a time-of-flight depth camera, a binocular stereo camera, etc. In some embodiments, the sensor 150-1 may further include other devices that can collect environmental data. For example, the sensor 150-1 may include a lidar. Among them, the lidar may include a two-dimensional lidar for obtaining two-dimensional point cloud data and a three-dimensional lidar for obtaining three-dimensional point cloud data. In some embodiments, the sensor 150-1 may be provided on the vehicle 150 and move with the movement of the vehicle 150. For more information about environmental data, see Figure 3 and its related descriptions.

[0039] In some embodiments, in an online scenario, the processing device 150-2 can be used to process information and / or data related to the traffic light and road association system. In some embodiments, the processing device 150-2 can be used to obtain traffic light data corresponding to traffic lights within a preset range, input the traffic light data into an association model, and extract traffic light features. Further, the processing device 150-2 can be used to obtain road data corresponding to roads within a preset range, input the road data into the association model, and extract road structure features. Then, the processing device 150-2 can determine the association relationship between the traffic light and the road based on the traffic light features and the road structure features by using the association model. Furthermore, the processing device 150-2 can determine the movement strategy of the vehicle based on the association relationship.

[0040] In some embodiments, in an offline scenario, during the driving process, the processing device 150-2 of the vehicle 150 can, based on the control probability issued by the server 110 and the real-time status of the traffic lights monitored online, determine the traffic status corresponding to the current moment for each preset area online, so as to determine the movement strategy of the vehicle in real time online.

[0041] In some embodiments, the processing device 150-2 can be integrally installed on the vehicle 150. The processing device 150-2 can be centralized or distributed. In some embodiments, the processing device 150-2 can be local or remote. For example, the processing device 150-2 can communicate with other components of the vehicle 150 (such as the sensor 150-1) through the network 120 or directly.

[0042] In some embodiments, the application scenario 100 of the traffic light and road association system may further include some or more other devices, such as a storage device and / or a user terminal.

[0043] The storage device can store data, instructions, and / or any other information. In some embodiments, the storage device can store data and / or instructions related to the traffic light and road association system. For example, the storage device can store the association relationship between the traffic light and the road. Again, for example, the storage device can store environmental data, road data, etc.

[0044] In some embodiments, the storage device can be connected to the network 120 to communicate with one or more other components (such as the server 110, the vehicle 150, etc.) in the application scenario 100 of the traffic light and road association system. One or more components of the application scenario 100 of the traffic light and road association system can access the data or instructions stored in the storage device through the network. In some embodiments, the storage device can be a part of the server 110.

[0045] The user terminal may include a mobile device, a tablet computer, a laptop computer, etc., or any combination thereof. In some embodiments, the user may obtain the movement strategy of the vehicle through the user terminal.

[0046] Figure 2 It is a module diagram of the traffic light and road association system 200 shown in some embodiments of this specification. In some embodiments, the traffic light and road association system 200 may include a first acquisition module 210, a first extraction module 220, a second acquisition module 230, a second extraction model 240, and a determination module 250. The traffic light and road association system 200 may be implemented on the vehicle 150 (e.g., the processing device 150-2) and applied to the online scenario; it may also be implemented on the server 110 and applied to the offline scenario.

[0047] In some embodiments, the first acquisition module 210 may be used to acquire the traffic light data corresponding to the traffic lights within a preset range.

[0048] In some embodiments, the first acquisition module 210 may further be used to acquire the traffic light information provided in the navigation map; acquire the environmental data within a preset range, where the environmental data includes traffic lights, and the environmental data is collected by the vehicle's sensors at the current time, and the association relationship corresponds to the current time; acquire the historical environmental data, where the historical environmental data includes traffic lights, and the historical environmental data is collected by the vehicle's sensors within the first historical time period, and the first historical time period is before the current time; based on at least one of the traffic light information, the environmental data, and the historical environmental data, determine the traffic light data.

[0049] In some embodiments, the first extraction module 220 may be used to input the traffic light data into the association model to extract the traffic light features, and the association model is a machine learning model.

[0050] In some embodiments, the second acquisition module 230 may be used to acquire the road data corresponding to the roads within a preset range.

[0051] In some embodiments, the second acquisition module 230 may further be used to acquire the road information provided in the navigation map; acquire the environmental data within a preset range, where the environmental data includes roads, and the environmental data is collected by the vehicle's sensors at the current time, and the association relationship corresponds to the current time; based on the road information and / or the environmental data, determine the road data.

[0052] In some embodiments, the second extraction model 240 may be used to input the road data into the association model to extract the road structure features.

[0053] In some embodiments, the determination module 250 can be used to determine the association relationship between the traffic light and the road based on the traffic light features and road structure features by using an association model. In some embodiments, the association model includes a model based on the attention mechanism.

[0054] In some embodiments, the input of the association model further includes the historical association relationship between the traffic light and the road corresponding to the second historical time period, and the second historical time period is before the current time. In some embodiments, the association relationship includes the control probability of the traffic light for each preset area of the road. In some embodiments, the association relationship further includes the traffic state corresponding to the current time for each preset area.

[0055] In some embodiments, the association relationship further includes the predicted traffic state of each preset area in a future time period, and the future time period is after the current time. In some embodiments, the determination module 250 can further be used to obtain the position of the vehicle at a future time, where the future time is within the future time period; and determine the movement strategy of the vehicle according to the position, the future time, and the association relationship.

[0056] For more specific content about the first acquisition module 210, the first extraction module 220, the second acquisition module 230, the second extraction model 240, and the determination module 250, see Figures 3 - 5 and its related descriptions.

[0057] It should be understood that Figure 2 the system and its modules shown can be implemented in various ways. It should be noted that the above description of the traffic light and road association system and its modules is only for convenience of description, and does not limit this specification to the scope of the examples given. It can be understood that for those skilled in the art, after understanding the principle of the system, they may, without departing from this principle, make any combination of the various modules, or form a subsystem and connect it with other modules. In some embodiments, Figure 2 the first acquisition module 210, the first extraction module 220, the second acquisition module 230, the second extraction model 240, and the determination module 250 disclosed in

[0058] Figure 3 is an exemplary flowchart of the traffic light and road association according to some embodiments of this specification. As Figure 3 shown, the process 300 includes the following steps. In some embodiments, Figure 3 one or more operations of the process 300 shown can be performed inFigure 1 It is implemented in application scenario 100 of the traffic light and road association system shown. For ease of explanation, process 300 is described by taking an online scenario as an example. Process 300 can be stored in a storage device in the form of instructions and called and / or executed by processing device 150-2 of vehicle 150.

[0059] Step 310: Obtain traffic light data corresponding to traffic lights within a preset range. In some embodiments, step 310 can be executed by first acquisition module 210.

[0060] The preset range refers to the range where the association relationship between traffic lights and roads needs to be determined in advance. For example, the preset range can be a range composed of a circle with the center of a certain intersection as the origin and a preset radius (such as 20m, 30m, 40m, etc.) as the radius. Among them, the preset radius can be set manually. Another example is that in an online scenario, the preset range can be the range composed of the road from the vehicle to the current intersection. It can be understood that the preset range can include the current intersection, traffic lights, and roads. Among them, the current intersection refers to the intersection where traffic lights are set and the vehicle is about to pass through.

[0061] Traffic light data refers to data related to traffic lights that can be recognized by the association model. In some embodiments, the traffic light data can be in vector form.

[0062] In some embodiments, the processing device can obtain traffic light information provided in the navigation map, obtain environmental data within the preset range, and obtain historical environmental data, and then determine the traffic light data based on at least one of the traffic light information, environmental data, and historical environmental data.

[0063] In some embodiments, the navigation map can include a standard map, a lane-level map, a high-precision map, etc.

[0064] Traffic light information refers to information related to traffic lights in the navigation map.

[0065] In some embodiments, the processing device can obtain traffic light information through methods such as data acquisition, data preprocessing, feature extraction, image processing and computer vision technology, semantic analysis and semantic understanding.

[0066] Environmental data refers to data that can reflect the environmental state within a preset range. For example, environmental data may include image data collected by a camera. Among them, the image data may include depth image data (3D image data), RGB (RGB color mode) image data, etc. Another example is that environmental data may include point cloud data collected by a lidar. The point cloud data is a two-dimensional coordinate set or a three-dimensional coordinate set composed of a large number of discrete points, and each point represents a position measured by a laser beam in space. The point cloud data records information such as the position, intensity, and reflectivity of each point and can be used to represent the geometric structure and surface characteristics of the surrounding environment. In some embodiments, the environmental data may include traffic lights.

[0067] In some embodiments, the processing device may collect environmental data at the current time through the sensors of the vehicle. Among them, the sensors may include a camera and / or a lidar. For more information about the sensors, see Figure 1 and its related descriptions.

[0068] The current time refers to the time when the processing device of the vehicle determines the association relationship in real time. It can be understood that the association relationship corresponds to the current time.

[0069] Historical environmental data refers to data that can reflect the historical environmental state within a preset range. In some embodiments, the historical environmental data may include traffic lights. In some embodiments, the processing device may collect environmental data through the sensors of the vehicle within the first historical time period. The types and acquisition methods of the historical environmental data are similar to those of the environmental data and will not be elaborated here.

[0070] As an example, during the driving process of the vehicle, the sensors are online in real time to collect environmental data in the form of data frames. For example, at the current moment, the camera on the vehicle collects a frame of image data, and the lidar on the vehicle rotates once to scan and outputs a frame of point cloud data. The environmental data at the current moment includes a frame (current frame) of image data and point cloud data collected at the current moment. The historical environmental data refers to the environmental data collected by the sensors on the vehicle before the current time. For example, the previous frame or several previous frames of image data and point cloud data of the current frame of image data and point cloud data.

[0071] The first historical time period refers to the time period before the current time. The length of the historical time period can be set artificially.

[0072] In some embodiments, the processing device may determine traffic light data through a traffic light data determination model based on at least one of traffic light information, environmental data, and historical environmental data. Among them, the traffic light data determination model may be a machine learning model. For example, the traffic light data determination model may be a Convolutional Neural Networks (CNN) model, a Deep Neural Networks (DNN) model, etc. Sample traffic light information, sample environmental data, and / or sample historical environmental data can be used as training data to train the traffic light data determination model, so that the traffic light data determination model can output its corresponding traffic light data based on at least one of traffic light information, environmental data, and historical environmental data. The training data may be historical data, and the labels corresponding to the training data can be determined by manual input or historical data.

[0073] In some embodiments, the input of the traffic light data determination model may further include traffic data acquired by roadside devices. The traffic data may include traffic lights. Among them, roadside devices refer to devices installed on or near roads for collecting, monitoring, and processing information related to road traffic. For example, roadside devices may include cameras, lidar, etc. Roadside devices are usually placed at intersections, road edges, traffic lights, bridges, tunnels, etc. to obtain traffic conditions in real time, collect data, control traffic flow, etc.

[0074] When determining traffic light data, traffic light information, environmental data, and historical environmental data are taken into account, so that the determined traffic light data can be more stable and reliable. In addition, since traffic light data can be obtained without relying on a navigation map (that is, the processing device can determine traffic light data through a traffic light data determination model based on at least one of environmental data and historical environmental data), traffic light data can be obtained without relying on a navigation map.

[0075] Step 320: Input the traffic light data into an association model to extract traffic light features. In some embodiments, step 320 may be executed by the first extraction module 220.

[0076] Traffic light features refer to data that can reflect traffic light features. For example, traffic light features may include data reflecting the type of traffic light (e.g., basic traffic light, countdown traffic light, arrow light, etc.), location, shape, status, etc. Among them, for a basic traffic light, the status of the traffic light may include the color of the traffic light (i.e., red, yellow, or green); for a countdown traffic light, the status of the traffic light may include the color of the traffic light and the remaining time; for an arrow light, the status of the traffic light may include the color of the traffic light and the arrow direction. In some embodiments, traffic light features may be in vector form.

[0077] In some embodiments, the processing device may input traffic light data into an association model to extract traffic light features. Among them, the association model may be a machine learning model. For more information about the association model, see Figure 4 its related description.

[0078] Step 330: Obtain road data corresponding to roads within a preset range. In some embodiments, step 330 may be executed by the second acquisition module 230.

[0079] Road data refers to data related to roads that can be recognized by the association model. In some embodiments, the traffic light data may be in vector form.

[0080] In some embodiments, the processing device may obtain road information provided in the navigation map and obtain environmental data within a preset range, and then determine the traffic light data based on the road information and / or the environmental data. Among them, the environmental data may include roads, and the environmental data may be collected by the vehicle's sensors at the current time.

[0081] Road information refers to information related to roads in the navigation map.

[0082] The method for obtaining road information provided in the navigation map is similar to the method for obtaining traffic light information provided in the navigation map, and will not be elaborated here.

[0083] In some embodiments, the processing device may determine road data based on road information and / or environmental data through a road data determination model. Among them, the road data determination model may be a machine learning model. The road data determination model may be a CNN model, a DNN model, etc. At least one of the historical road information and the historical environmental data may be used as training data to train the road data determination model, so that the road data determination model can output its corresponding road data based on the road information and / or the environmental data. The label corresponding to the training data may be determined by manual input or historical data.

[0084] In some embodiments, the input of the traffic light data determination model may further include traffic data obtained by roadside devices. For more information about the navigation map, environmental data, and traffic data, see the above description.

[0085] In some embodiments, when the input of the traffic light data determination model includes road information, environmental data, and / or traffic data, the processing device needs to unify the coordinate systems of the road information, environmental data, and / or traffic data before inputting them into the traffic light data determination model.

[0086] When determining the road data, road information and environmental data are considered, so that the determined road data can be more stable and reliable. In addition, since the road data can be obtained without relying on a navigation map (i.e., the processing device can determine the road data based only on the environmental data through a road data determination model), the road data can be obtained without relying on a navigation map.

[0087] Step 340: Input the road data into the association model to extract road structure features. In some embodiments, step 340 can be executed by the second extraction module 240.

[0088] The lane structure feature refers to the data that can reflect the lane features. For example, the road structure features can include data reflecting the number of lanes, lane width, lane type, lane position, lane shape, lane connection relationship, stop line, crosswalk, flower bed, etc. Among them, the lane type can include straight lanes, left-turn lanes, right-turn lanes, straight-left mixed lanes, straight-right mixed lanes, U-turn lanes, bus-only lanes, non-motor vehicle lanes, etc. In some embodiments, the road structure features can be in vector form. For more information about the lane type of traffic lights, see the above description.

[0089] In some embodiments, the processing device can input the road data into the association model to extract road structure features. Among them, the association model can be a machine learning model. For more information about the association model, see Figure 4 and its related description.

[0090] Step 350: Based on the traffic light features and road structure features, use the association model to determine the association relationship between the traffic light and the road. In some embodiments, step 350 can be executed by the determination module 250.

[0091] The association relationship refers to the mutual connection between the traffic light and the road.

[0092] In some embodiments, the association relationship can include the control probability of the traffic light for each preset area of the road.

[0093] The preset area is an area obtained after dividing a preset range according to preset conditions. Among them, the preset conditions can include dividing the preset range according to lanes. Accordingly, the preset area can include several lanes in the preset range.

[0094] The control probability refers to the control probability of the traffic light for each preset area. The control probability can be represented by a value between 0% and 100%, where 0% represents no control at all and 100% represents full control. In some embodiments, the control probability can be represented by a vector, and each vector element represents the control probability of a traffic light for each preset area. For example, when the preset area includes a left-turn lane, a straight-ahead lane, and a right-turn lane, and includes a left-turn signal, a straight-ahead signal, and a right-turn signal, the control probabilities of the left-turn signal for the left-turn lane, the straight-ahead lane, and the right-turn lane are 100%, 0%, and 0% respectively, the control probabilities of the straight-ahead signal for the left-turn lane, the straight-ahead lane, and the right-turn lane are 0%, 100%, and 0% respectively, and the control probabilities of the right-turn signal for the left-turn lane, the straight-ahead lane, and the right-turn lane are 0%, 0%, and 100% respectively. Then the control probability can be represented by the vector [(100%, 0%, 0%), (0%, 100%, 0%), (0%, 0%, 100%)]. Among them, (100%, 0%, 0%) represents the control probability of the left-turn signal for the left-turn lane, the straight-ahead lane, and the right-turn lane, (0%, 100%, 0%) represents the control probability of the straight-ahead signal for the left-turn lane, the straight-ahead lane, and the right-turn lane, and (0%, 0%, 100%) represents the control probability of the right-turn signal for the left-turn lane, the straight-ahead lane, and the right-turn lane. Another example, when the preset area includes a left-turn lane, a straight-ahead lane, and a right-turn lane, and there is only one circular traffic light, the control probabilities of the circular traffic light for the left-turn lane, the straight-ahead lane, and the right-turn lane are 100%, 100%, and 0% respectively. Then the control probability can be represented by the vector (100%, 100%, 0%). Among them, (100%, 100%, 0%) represents the control probability of the circular traffic light for the left-turn lane, the straight-ahead lane, and the right-turn lane. Another example, when the preset area includes a left-turn lane and a straight / right-turn combined lane, and includes a left-turn signal and a straight-ahead signal, the control probabilities of the left-turn signal for the left-turn lane and the straight / right-turn combined lane are 100% and 0% respectively, and the control probabilities of the straight-ahead signal for the left-turn lane and the straight / right-turn combined lane are 0% and (100%, 0%) respectively. Then the control probability can be represented by the vector {[100%, 0%], [0%, (100%, 0%)]}. Among them, [100%, 0%] represents the control probability of the left-turn signal for the left-turn lane and the straight / right-turn combined lane, 0% in [0%, (100%, 0%)] represents the control probability of the straight-ahead signal for the left-turn lane, and (100%, 0%) represents the control probability of the straight-ahead signal for the straight / right-turn combined lane (that is, the control probability of the straight-ahead signal for the straight part of the straight / right-turn combined lane is 100% and the control probability for the right-turn is 0). Another example, when the preset area includes a left-turn lane and a straight / right-turn combined lane, and includes a circular traffic light, the control probabilities of the circular traffic light for the left-turn lane and the straight / right-turn combined lane are 100% and (100%, 0%) respectively. Then the control probability can be represented by the vector [100%, (100%, 0%)].Among them, 100% represents the control probability of the circular traffic light for the left-turn lane, and (100%, 0%) represents the control probability of the circular traffic light for the straight-through and right-turn mixed lane (that is, the control probability of the circular traffic light for the straight-through in the straight-through and right-turn mixed lane is 100%, and the control probability for the right-turn is 0).

[0095] In both the online scenario and the offline scenario, the association relationship can include the control probability of the traffic light for each preset area of the road.

[0096] It should be noted that the control probability of the traffic light for each preset area of the road output by the association model can conform to a Gaussian distribution. That is, the control probability of the traffic light for each preset area of the road can all be non-zero, and the control probability of the most likely controlled preset area is the highest. For example, if the control probability of a traffic light for three lanes is (90%, 5%, 5%), it can be considered that the traffic light can control the first lane.

[0097] In some embodiments, when applied to the online scenario, the association relationship can further include the traffic state corresponding to the current time for each preset area.

[0098] The traffic state refers to the state that can reflect whether the preset area can be passed at the current time. In some embodiments, the traffic state can include a passable state and a non-passable state. The traffic state can be represented by 0 and 1, where 0 represents the non-passable state and 1 represents the passable state. In some embodiments, the traffic state can be represented by a vector, and each vector element represents the traffic state of a preset area at the current time. For example, when the preset areas include a left-turn lane, a straight-through lane, and a right-turn lane, and at this time the left-turn lane is in a passable state, and the straight-through lane and the right-turn lane are in a non-passable state, then the traffic state can be represented by (1, 0, 0), where 1 represents that the left-turn lane is in a passable state, and the two 0s respectively represent that the straight-through lane and the right-turn lane are in a non-passable state.

[0099] It can be understood that in the online scenario, the vehicle obtains the association relationship in real time, that is, it can obtain the traffic state corresponding to the current time for each preset area. Therefore, the association relationship can include the traffic state corresponding to the current time for each preset area. In the offline scenario, the server obtains the association relationship periodically at a preset time interval, so it cannot obtain the association relationship in real time, that is, it cannot obtain the traffic state corresponding to the current time for each preset area. Therefore, the association relationship does not include the traffic state corresponding to the current time for each preset area.

[0100] For the above reasons, in the online scenario, since the association relationship can include the traffic state corresponding to the current time for each preset area, the processing device can determine the moving strategy of the vehicle in real time online based on the traffic state corresponding to the current time for each preset area.

[0101] In some embodiments, when applied to an online scenario, the association relationship may further include the predicted passing state of each preset area in a future time period. At this time, the processing device may obtain the position of the vehicle at a future time, and then determine the movement strategy of the vehicle according to the above position, the future time, and the association relationship including the predicted passing state of each preset area in the future time period. Wherein, the future time is within the future time period. For more content on determining the movement strategy of the vehicle, see Figure 5 and its related description.

[0102] The future time period refers to the time period after the current time determined based on a second preset rule. Wherein, the second preset rule refers to a preset rule. For example, the preset rule may be that the processing device determines a manually preset time period as the future time period. Another example is that the preset rule may be that the processing device can determine the estimated driving time of the vehicle based on the average speed of the vehicle and the distance of the vehicle from the current intersection, and then determine the estimated driving time as the future time period.

[0103] The predicted passing state refers to the state that can reflect whether a preset area can be passed in a future time period. In some embodiments, the predicted passing states of different time periods in the future time period may be the same. For example, the predicted passing state may be a prohibited passing state or a passable state. In some embodiments, the predicted passing states of different time periods in the future time period may be different. For example, the start time of the future time period is a, the end time is b, which can be expressed as (a, b). There is an intermediate time c between the start time a and the end time b. The passing state from the start time a to the intermediate time c in the future time period may be a prohibited passing state, and the passing state from the intermediate time c to the end time b may be a passable state. At this time, the predicted passing state may be expressed as [(a, c, 0), (c, b, 1)], where (a, c, 0) represents that the time period (a, c) is a prohibited passing state, and (c, b, 1) represents that the time period (c, b) is a passable state.

[0104] In some embodiments, in an online scenario, while the vehicle is passing through a preset range, it will obtain multiple frames of environmental data based on sensors. For each moment corresponding to each frame in the initial target number of frames, the processing device can determine the association relationship between the traffic light and the road based on steps 310 - 350, and predict the association relationship between the traffic light and the road in a future time period. The association model can predict the association relationship between the traffic light and the road in a future time period according to the countdown state of the current traffic light. For example, if the current straight-ahead traffic light is green and the countdown state is 10s remaining, the association model can predict that the straight-ahead lane will be in a passing state in the next 10s. By inputting the historical change pattern of the traffic light into the association model, the association model can further predict the association relationship between the traffic light and the road in a future time period according to the historical change pattern of the traffic light. For example, if the current straight-ahead traffic light is green and the countdown state is 10s remaining, and the historical change pattern of the traffic light is that the green light lasts for 15s and then turns red for 15s, the association model can predict that the straight-ahead lane will be in a passing state in the next 10s and in a non-passing state from 10s to 15s in the future. Among them, the target number can be set manually.

[0105] By determining the association relationship between the traffic light and the road for each moment corresponding to each frame in the initial target number of frames and predicting the association relationship between the traffic light and the road in a future time period, when the vehicle passes through the preset range, there is no need to determine the association relationship every time a frame of environmental data is collected. When the time progresses to the future time period, there is no need to determine the association relationship again, and only the previously determined association relationship needs to be applied. For example, when the time progresses to the future time period, the processing device of the vehicle can execute steps 510 - 520 to obtain the movement strategy of the vehicle. The processing device of the vehicle determines the preset area where the vehicle is located according to the real-time position of the vehicle, determines the passing state of the preset area according to the previously determined association relationship, and determines the movement strategy of the vehicle (such as moving or stopping moving) according to the passing state.

[0106] In some embodiments, the processing device can input the traffic light features and road structure features into the association model to determine the association relationship between the traffic light and the road. For more information about the association relationship model, see Figure 4 and its related descriptions.

[0107] By extracting the traffic light features and road structure features and then determining the association relationship between the traffic light and the road, the movement strategy of the vehicle can be obtained accurately and quickly. In addition, since the traffic light data and road data can be obtained without relying on a navigation map, the association relationship can be obtained without relying on a navigation map, thereby obtaining the movement strategy of the vehicle.

[0108] It should be noted that the above description of the association between the process traffic light and the road is only for illustration and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the association between the process traffic light and the road under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0109] In some embodiments, the processing device can determine whether to execute the method for associating traffic lights with roads based on a first preset rule. Herein, the first preset rule refers to a rule set in advance. For example, when the processing device determines that the distance between the current position of the vehicle and the center of the preset range is a first preset distance, or it is expected that it will take a preset time to reach the center of the preset range from the current position of the vehicle, the processing device can start to execute the method for associating traffic lights with roads; when the processing device determines that the vehicle has left the center of the preset range, the processing device can stop executing the method for associating traffic lights with roads. The first preset distance can be set manually. For another example, when the processing device acquires the signal of the traffic light detected by the sensor, the processing device can start to execute the method for associating traffic lights with roads; when the processing device acquires that the sensor cannot detect the signal of the traffic light (has left the intersection), the processing device can stop executing the method for associating traffic lights with roads.

[0110] In some embodiments, for the offline scenario, the server can implement step 310 - step 350. Since in the offline state, the association relationship does not include the traffic state corresponding to the current time in each preset area, the server cannot determine the moving strategy of the vehicle in real time online. However, the server can send the above control probability to each vehicle. During the driving process, the processing device of the vehicle can, based on the above control probability and the real-time state of the traffic lights monitored online, determine online the traffic state corresponding to the current moment in each preset area, so as to determine the moving strategy of the vehicle in real time online. For more content about the moving strategy, see Figure 5 and its related description.

[0111] Figure 4 is a schematic diagram of the process 400 for determining the association relationship based on the association model according to some embodiments of this specification.

[0112] In some embodiments, the association model 420 can be a deep learning neural network model. Exemplary deep learning neural network models can include a CNN model, a DNN model, a Recurrent Neural Network (RNN) model, a Long Short-Term Memory (LSTM) model, etc. or a combination thereof. In some embodiments, the association model 420 can include a model based on an attention mechanism. Exemplary attention mechanism models can include a Transformer model.

[0113] In some embodiments, the association model 420 may include a traffic light feature extraction layer 420-1, a road structure feature extraction layer 420-2, and an association layer 420-3. The types of machine learning models corresponding to the traffic light feature extraction layer 420-1, the road structure feature extraction layer 420-2, and the association layer 420-3 may be the same or different.

[0114] In some embodiments, the input of the traffic light feature extraction layer 420-1 may include traffic light data 410-1, and the output of the traffic light feature extraction layer 420-1 may include traffic light features 430-1. For more information about traffic light data and traffic light features, see Figure 3 and its related description.

[0115] In some embodiments, the input of the road structure feature extraction layer 420-2 may include road data 410-2, and the output of the road structure feature extraction layer 420-2 may include road structure features 430-2. For more information about road data and road structure features, see Figure 3 and its related description.

[0116] In some embodiments, the input of the association layer 420-3 may include traffic light features 430-1 and road structure features 430-2, and the output of the association layer 420-3 may include an association relationship 440. For more information about the association relationship, see Figure 3 and its related description.

[0117] In some embodiments, the input of the association model 420 further includes a historical association relationship 410-3 between traffic lights and roads corresponding to a second historical time period, that is, the input of the association layer 420-3 further includes a historical association relationship 410-3 between traffic lights and roads corresponding to a second historical time period.

[0118] The historical association relationship 410-3 refers to the mutual connection between traffic lights and roads within the second historical time period.

[0119] The second historical time period refers to the time period before the current time. The length of the historical time period can be set artificially. In some embodiments, the first historical time period and the second historical time period may be the same or different.

[0120] In some embodiments, the processing device may call the historical association relationship 410-3 within a second historical time period stored in the vehicle storage device. As an example, after the vehicle collects the current frame of environmental data at the current time, the processing device on the vehicle will determine the association relationship corresponding to the current time (current frame). The historical association relationship 410-3 may refer to the association relationship determined by the processing device on the vehicle before the current time. For example, the association relationship determined based on the previous frame or several previous frames of environmental data of the current frame of environmental data.

[0121] In some embodiments, the input of the association model 420 further includes the moving state 410-4 of surrounding vehicles at the current time.

[0122] Surrounding vehicles refer to vehicles whose distance from the vehicle is within a second preset distance. Among them, the second preset distance can be set artificially.

[0123] The moving state 410-4 refers to whether the surrounding vehicles are moving and / or the moving speed at the current time, which can reflect the current time.

[0124] In some embodiments, the processing device may extract the moving state 410-4 of the surrounding vehicles of the vehicle at the current time based on the environmental data (current frame of environmental data) and historical environmental data (previous frame or several previous frames of environmental data). For example, for a frame of environmental data, the processing device can obtain the position coordinates of the vehicle, and then based on the point cloud data obtained by the lidar and / or the image data obtained by the camera in the environmental data, determine the distance and azimuth between the surrounding vehicles and the vehicle, so as to estimate the position coordinates of the surrounding vehicles; combining the position coordinates of the surrounding vehicles corresponding to multiple frames of environmental data, the driving trajectory of the surrounding vehicles can be obtained; according to the driving trajectory of the surrounding vehicles, the moving state 410-4 of the surrounding vehicles can be obtained. Further, the processing device may determine the preset area corresponding to the surrounding vehicles based on the coordinate positions of the surrounding vehicles and the navigation map, so as to determine whether the surrounding vehicles in each preset area are in a driving state or a stopped state. Inputting the moving state 410-4 of the surrounding vehicles in each preset area into the association model can be used as auxiliary information for determining the association relationship, making the determined association relationship more accurate and reliable.

[0125] In some embodiments, the traffic light feature extraction layer 420-1, the road structure feature extraction layer 420-2, and the association layer 420-3 can be obtained through joint training. The model parameters of the traffic light feature extraction layer 420-1, the road structure feature extraction layer 420-2, and the association layer 420-3 can be obtained by training an initial traffic light feature extraction layer, an initial road structure extraction layer, and an initial association layer using a plurality of training samples. For example, a plurality of training samples can be obtained. Each training sample may include sample traffic light data and sample road data corresponding to the sample time.

[0126] The training of the initial traffic light feature extraction layer, the initial road structure extraction layer, and the initial association layer may include one or more iterations. By way of example only, in the current iteration, for each training sample, the sample traffic light data corresponding to the sample moment is input into the initial traffic light feature extraction layer to obtain the traffic light features corresponding to the sample moment output by the initial traffic light feature extraction layer; the sample road data corresponding to the sample moment is input into the initial road structure feature extraction layer to obtain the road structure features corresponding to the sample moment output by the initial road structure feature extraction layer; the sample traffic light features corresponding to the sample moment output by the initial traffic light feature extraction layer are input into the initial association layer, and the sample road structure features corresponding to the sample moment output by the initial road structure feature extraction layer are input into the initial association layer to obtain the sample association relationship corresponding to the sample moment output by the initial association layer. The initial traffic light feature extraction layer, the initial road structure extraction layer, and the initial association layer are initial machine learning models in the first iteration and are the machine learning models obtained in the previous iteration in other iterations.

[0127] The label of the training sample may be the association relationship corresponding to the sample moment manually labeled in the historical data, that is, the label of the training sample may include the control probability of the traffic light for each preset area of the road corresponding to the sample moment, the traffic state corresponding to each preset area corresponding to the sample moment, and / or the predicted traffic state of each preset area in the third historical time period. The third historical time period is after the sample moment and is a time period in the historical data.

[0128] During the training process, the processing device may construct a loss function based on the label and the output result of the initial association layer. At the same time, the parameters of the initial traffic light feature extraction layer, the initial road structure extraction layer, and the initial association layer are updated until the preset condition is met and the training is completed. The preset condition may be one or more of the loss function being less than a threshold, converging, or the training cycle reaching a threshold, etc.

[0129] In some embodiments, when the input of the association layer 420-3 includes the historical association relationship 410-3, the training sample further includes the sample association relationship at the fourth historical time point. The sample traffic light features corresponding to the sample moment output by the initial traffic light feature extraction layer, the sample road structure features corresponding to the sample moment output by the initial road structure feature extraction layer, and the sample historical association relationship may be input into the initial association layer together. The fourth historical time period is before the current moment and is a time period in the historical data.

[0130] In some embodiments, when the input of the association layer 420-3 includes the movement state 410-4, the training sample further includes the sample movement states of surrounding vehicles at the current moment. The sample traffic light features corresponding to the sample moment output by the initial traffic light feature extraction layer, the sample road structure features corresponding to the sample moment output by the initial road structure feature extraction layer, and the sample movement state can be input into the initial association layer together.

[0131] By processing traffic light data, road data, historical association relationships, and / or movement states through an association model including a traffic light feature extraction layer, a road structure feature extraction layer, and an association layer, the association relationship can be obtained quickly and accurately. Moreover, joint training is beneficial to solving the problem that it is difficult to obtain labels when training the traffic light feature extraction layer and the road structure feature extraction layer separately. Secondly, jointly training the traffic light feature extraction layer, the road structure feature extraction layer, and the association layer can not only reduce the required number of samples, but also improve the training efficiency.

[0132] It should be noted that the above description of determining the association relationship based on the association model for the process is only for illustration and example, and does not limit the scope of application of this specification. For those skilled in the art, various corrections and changes can be made to the process of determining the association relationship based on the association model under the guidance of this specification. However, these corrections and changes are still within the scope of this specification.

[0133] Figure 5 is an exemplary flowchart of determining the movement strategy of a vehicle shown according to some embodiments of this specification. As Figure 5 shown, process 500 includes the following steps. In some embodiments, Figure 5 one or more operations of the process 500 shown can be implemented in Figure 1 the application scenario 100 of the traffic light and road association system shown. For example, Figure 5 the process 500 shown can be stored in a storage device in the form of instructions and called and / or executed by the processing device 150-2 of the vehicle 150.

[0134] In some embodiments, in an online scenario, when the vehicle passes through a preset range, it will obtain multiple frames of environmental data based on sensors. For each moment corresponding to each frame of the initial target number of frames, the processing device can determine the association relationship between the traffic light and the road based on steps 310 - step 350, and predict the association relationship between the traffic light and the road in the future time period. The target number can be set manually. When the time progresses to the future time period, the processing device of the vehicle can execute steps 510 - step 520 to obtain the movement strategy of the vehicle.

[0135] Step 510, obtain the position information of the vehicle at a future time.

[0136] Future time refers to a time point within a future time period.

[0137] Location information refers to the position coordinates of the vehicle at a future time.

[0138] In some embodiments, the processing device can obtain the position coordinates at a future time through the positioning device of the vehicle itself, and use the position coordinates as the location information of the vehicle at the future time. Among them, the positioning device can include devices such as a Global Positioning System (GPS), in-vehicle sensors, and in-vehicle navigation systems.

[0139] Step 520: Determine the movement strategy of the vehicle according to the location information, future time, and association relationship.

[0140] The movement strategy refers to the movement plan of the vehicle. For example, the movement strategy can include vehicle driving and vehicle stopping.

[0141] In some embodiments, when the vehicle travels to a future time, the processing device can determine the preset area where the vehicle is located according to the location information of the vehicle at that moment. Then, the processing device can determine the predicted traffic state of each preset area in the future time period according to the association relationship, and further determine the traffic state of the preset area at the future time, and then determine the movement strategy of the vehicle according to the above traffic state.

[0142] By determining the predicted traffic state of each preset area in the future time period, and then determining the movement strategy of the vehicle at the future time, it is not necessary to execute steps 310 - 350 at the future time to obtain the movement strategy of the vehicle, thus reducing the operating burden of the processing device. And when the network transmission quality at the future moment is poor or even environmental data cannot be obtained, the movement strategy of the vehicle at the future time can be obtained based on steps 510 - 520, thus ensuring that the obtained movement strategy is not interrupted.

[0143] It should be noted that the above description of determining the movement strategy of the vehicle by the process is only for illustration and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the process of determining the movement strategy of the vehicle under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0144] In some embodiments, the traffic light and road association device includes a processor and a memory; the memory is used to store instructions, and when the instructions are executed by the processor, the device realizes the traffic light and road association method. The traffic light and road association device can be implemented on vehicle 150 (for example, processing device 150 - 2) and applied to an online scenario; it can also be implemented on server 110 and applied to an offline scenario.

[0145] In some embodiments, a computer-readable storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer runs the method for associating a traffic light with a road. The computer instructions of the computer-readable storage medium can be read by a computing device (e.g., processing device 150-2) on vehicle 150 and applied to an online scenario; or can be read by server 110 and applied to an offline scenario.

[0146] The beneficial effects that the embodiments of this specification may bring include but are not limited to: (1) By extracting traffic light features and road structure features and then determining the association relationship between the traffic light and the road, the movement strategy of the vehicle can be accurately and quickly obtained. In addition, since traffic light data and road data can be obtained without relying on a navigation map, the association relationship can be obtained without relying on a navigation map, thereby obtaining the movement strategy of the vehicle. (2) By processing traffic light data, road data, historical association relationships, and / or movement states through an association model including a traffic light feature extraction layer, a road structure feature extraction layer, and an association layer, the association relationship can be obtained quickly and accurately. Moreover, joint training is beneficial to solving the problem of difficult label acquisition when separately training the traffic light feature extraction layer and the road structure feature extraction layer. Secondly, jointly training the traffic light feature extraction layer, the road structure feature extraction layer, and the association layer can not only reduce the required number of samples but also improve the training efficiency. (3) By determining the predicted passing state of each preset area in a future time period and then determining the movement strategy of the vehicle at a future time, the operating burden of the processing device can be reduced. And when the network transmission quality is poor at a future moment or even environmental data cannot be obtained, the movement strategy of the vehicle at a future time can be obtained, thereby ensuring that the obtained movement strategy is not interrupted.

[0147] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are proposed in this specification, so such modifications, improvements, and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0148] At the same time, this specification uses specific terms to describe the embodiments of this specification. Such as "one embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "an embodiment" or "one embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0149] In addition, unless clearly stated in the claims, the order of the processing elements and sequences, the use of numerical and alphabetical characters, or the use of other names described in this specification are not used to limit the order of the processes and methods of this specification. Although various examples are discussed in the above disclosure for some currently useful embodiments of the invention, it should be understood that such details are for illustrative purposes only. The appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that conform to the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only through software solutions, such as installing the described system on existing servers or mobile devices.

[0150] Similarly, it should be noted that, in order to simplify the presentation of the disclosure in this specification and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of this specification, sometimes multiple features are merged into one embodiment, drawing, or description thereof. However, this method of disclosure does not mean that the features required by the subject matter of this specification are more than those mentioned in the claims. In fact, the features of the embodiments are fewer than all the features of the individual embodiments disclosed above.

[0151] In some embodiments, numbers are used to describe the components and the quantity of attributes. It should be understood that such numbers used for the description of the embodiments are modified by the modifiers "about", "approximate" or "substantially" in some examples. Unless otherwise stated, "about", "approximate" or "substantially" indicate that the stated numbers allow a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, and such approximate values may change according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining the general number of digits. Although the numerical ranges and parameters used in some embodiments of this specification to confirm the breadth of their scope are approximate values, in specific embodiments, such numerical settings are as precise as possible within the feasible range.

[0152] For each patent, patent application, patent application publication, and other materials cited in this specification, such as articles, books, specifications, publications, documents, etc., their entire contents are hereby incorporated into this specification by reference. Except for the application history documents that are inconsistent with or conflict with the content of this specification, and except for the documents that limit the broadest scope of the claims of this specification (currently or subsequently appended to this specification). It should be noted that if there are any inconsistencies or conflicts between the descriptions, definitions, and / or the use of terms in the supplementary materials of this specification and the content described in this specification, the descriptions, definitions, and / or the use of terms in this specification shall prevail.

[0153] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be regarded as consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly presented and described in this specification.

Claims

1. A method for associating traffic lights with roads, characterized in that, The method includes: Obtaining traffic light data corresponding to traffic lights within a preset range; Inputting the traffic light data into an association model to extract traffic light features, where the association model is a machine learning model; Obtaining road data corresponding to the road within the preset range; Inputting the road data into the association model to extract road structure features; and Based on the traffic light features and the road structure features, using the association model to determine the association relationship between the traffic light and the road.

2. The method according to claim 1, wherein The obtaining of the road data corresponding to the road within the preset range includes: Obtaining road information provided in a navigation map; Obtaining environmental data within the preset range, where the environmental data includes the road, and the environmental data is collected by a vehicle's sensor at the current time, and the association relationship corresponds to the current time; and Based on the road information and / or the environmental data, determining the road data.

3. The method according to claim 1, characterized in that, The obtaining of the traffic light data corresponding to traffic lights within a preset range includes: Obtaining traffic light information provided in a navigation map; Obtaining environmental data within the preset range, where the environmental data includes the traffic light, and the environmental data is collected by a vehicle's sensor at the current time, and the association relationship corresponds to the current time; Obtaining historical environmental data, where the historical environmental data includes the traffic light, and the historical environmental data is collected by the vehicle's sensor within a first historical time period, and the first historical time period is before the current time; and Based on at least one of the traffic light information, the environmental data, and the historical environmental data, determining the traffic light data.

4. The method according to claim 3, characterized in that, The input of the association model further includes the historical association relationship between the traffic light and the road corresponding to a second historical time period, and the second historical time period is before the current time.

5. The method according to claim 3, wherein The association relationship includes the control probability of the traffic light for each preset area of the road.

6. The method according to claim 5, wherein The association relationship further includes the traffic state corresponding to the current time for each preset area.

7. The method according to claim 6, wherein The association relationship further includes the predicted traffic state for each preset area within a future time period, and the future time period is after the current time.

8. The method according to claim 7, wherein The method further includes: Obtaining the position of the vehicle at a future time, where the future time is within the future time period; and According to the position, the future time, and the association relationship, determining the movement strategy of the vehicle.

9. The method according to claim 1, characterized in that, The association model includes a model based on an attention mechanism.

10. A traffic light and road association system, characterized in that, It includes: A first obtaining module for obtaining traffic light data corresponding to traffic lights within a preset range; A first extraction module for inputting the traffic light data into an association model to extract traffic light features, where the association model is a machine learning model; A second obtaining module for obtaining road data corresponding to the road within the preset range; A second extraction module for inputting the road data into the association model to extract road structure features; And A determination module for based on the traffic light features and the road structure features, using the association model to determine the association relationship between the traffic light and the road.

11. A traffic light and road association device, characterized in that, Comprising a processor, the processor is configured to execute the traffic light and road association method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions, and when a computer reads the computer instructions in the storage medium, the computer executes the traffic light and road association method according to any one of claims 1 to 9.

Images