Traffic light data processing method and device

Abstract

The invention provides a traffic light data processing method and device, and the method comprises the steps: obtaining the traffic light information of each traffic light of a road based on a road image of the road on which a vehicle travels; determining each point cloud coordinate of each traffic light in the point cloud data of the road based on the traffic light information of each traffic light; performing clustering processing on each point cloud coordinate of each traffic light to obtain a center coordinate of each traffic light; and based on the center coordinates of the traffic lights, screening out the traffic lights meeting a preset condition. In the process of determining the traffic light, the traffic light is identified from the road image, and then the final traffic light is determined from the identified traffic lights in combination with the point cloud data of the road, so that a high-precision map does not need to be constructed in the whole process, the capital investment is effectively reduced, and the cost for identifying the traffic light is reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a traffic light data processing method and device. Background Art

[0002] With the continuous research and development and iteration of autonomous driving technology by major automobile companies and Internet companies, the perception technology in the field of autonomous driving has achieved rapid development. Traffic light detection and recognition is a crucial part of the perception technology of autonomous driving vehicles. It is the key to whether autonomous driving vehicles can pass through traffic intersections smoothly, thereby ensuring the safety of the vehicle, pedestrians and other road users.

[0003] When identifying and detecting traffic lights, a combination of vision and high-precision maps is usually used. This solution obtains the traffic light area through a high-precision map and then detects and identifies the area. This solution requires the construction of a high-precision map, which is a very cumbersome and expensive process. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a traffic light data processing method and device. The present invention does not need to build a high-precision map, and can identify traffic lights through collected pictures and point cloud data, effectively reducing the cost of identification.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A traffic light data processing method, comprising:

[0007] Based on the road image of the road on which the vehicle is traveling, acquiring traffic light information of each traffic light on the road;

[0008] Based on the traffic light information of each of the traffic lights, determining the point cloud coordinates of each of the traffic lights in the point cloud data of the road;

[0009] Performing clustering processing on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light;

[0010] Based on the central coordinates of each of the traffic lights, the traffic lights that meet the preset conditions are screened out.

[0011] In the above method, optionally, the acquiring of traffic light information of each traffic light on the road based on the road image of the road on which the vehicle is traveling comprises:

[0012] Inputting the road image into a preset target detection deep learning model to obtain contour information of each traffic light on the road image;

[0013] Based on the contour information of each of the traffic lights, acquiring a detection image of each of the traffic lights in the road image;

[0014] Inputting the detection image of each of the traffic lights into a preset image classification learning model to obtain category information of each of the traffic lights;

[0015] For each of the traffic lights, the outline information and the category information of the traffic light are determined as traffic light information.

[0016] Optionally, the method described above, based on the traffic light information of each traffic light, determining the point cloud coordinates of each traffic light in the point cloud data of the road includes:

[0017] Collecting point cloud data of the road, segmenting the point cloud data, and obtaining point clouds of multiple categories;

[0018] Based on the type information of each point cloud, determining a point cloud that meets a preset category as a target point cloud;

[0019] Based on the traffic light information of each of the traffic lights, determining a matching point cloud of each of the traffic lights in each of the target point clouds;

[0020] For each of the traffic lights, the point cloud coordinates of the matching point cloud of the traffic light are determined as the point cloud coordinates of the traffic light.

[0021] In the above method, optionally, determining the matching point cloud of each traffic light in each target point cloud based on the traffic light information of each traffic light includes:

[0022] Mapping each of the target point clouds onto the road image to obtain a point cloud mapping image;

[0023] For each of the traffic lights, based on the traffic light information of the traffic light, a traffic light outline is determined on the point cloud mapping image, and target point clouds within the traffic light outline are all determined as matching point clouds of the traffic light.

[0024] In the above method, optionally, clustering the point cloud coordinates of each of the traffic lights to obtain the center coordinates of each of the traffic lights includes:

[0025] Using the vehicle as a standard point to construct a grid, the grid comprising a plurality of sub-grids;

[0026] Determine a grid set of each of the traffic lights in each of the sub-grids, wherein the grid set includes a plurality of valid sub-grids;

[0027] For each of the traffic lights, the grid set of the traffic light is used to cluster the point cloud coordinates of the traffic light to obtain the center coordinates of the traffic light.

[0028] In the above method, optionally, determining a grid set of each of the traffic lights in each of the sub-grids includes:

[0029] For each of the point cloud coordinates, determining the subgrid where the point cloud coordinate is located, and determining the subgrid as a valid subgrid of the traffic light to which the point cloud coordinate belongs;

[0030] For each of the traffic lights, each valid sub-grid of the traffic light is combined into a grid set of the traffic light.

[0031] In the above method, optionally, the applying the grid set of the traffic light to cluster the point cloud coordinates of the traffic light to obtain the center coordinates of the traffic light includes:

[0032] Based on the grid set, clustering each point cloud coordinate of the traffic light to obtain each cluster of the traffic light, wherein the cluster contains a plurality of point cloud coordinates;

[0033] The cluster with the largest number of point cloud coordinates is determined as the target cluster;

[0034] The coordinates of each point cloud in the target cluster are averaged to obtain the center coordinates.

[0035] In the above method, optionally, clustering the point cloud coordinates of the traffic light based on the grid set to obtain the clusters of the traffic light includes:

[0036] Selecting target point cloud coordinates from among the point cloud coordinates of the traffic light;

[0037] Determine the adjacent sub-grids of the target point cloud coordinates in each valid sub-grid of the grid set;

[0038] Clustering point cloud coordinates in adjacent subgrids of the target point cloud coordinates;

[0039] Marking each point cloud coordinate belonging to the cluster in the traffic light, and selecting new target point cloud coordinates from the remaining unmarked point cloud coordinates;

[0040] Each of the adjacent sub-grids is deleted from the grid set, and the step of determining the adjacent sub-grid of the target point cloud coordinates in each valid sub-grid of the grid set is returned to be executed until all the point cloud coordinates of the traffic light are marked.

[0041] The above method may optionally further include:

[0042] The traffic light information and three-dimensional information of the traffic lights that meet the preset conditions are fed back to the automatic driving decision system of the vehicle, so that the automatic driving decision system constructs the driving scene of the vehicle and determines the driving strategy of the vehicle based on the traffic light information and the three-dimensional information.

[0043] A traffic light data processing device, comprising:

[0044] an acquisition unit, configured to acquire traffic light information of each traffic light on the road based on a road image of the road on which the vehicle is traveling;

[0045] a determining unit, configured to determine the point cloud coordinates of each of the traffic lights in the point cloud data of the road based on the traffic light information of each of the traffic lights;

[0046] A clustering unit, used for performing clustering processing on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light;

[0047] The screening unit is used to screen out the traffic lights that meet the preset conditions based on the center coordinates of each of the traffic lights.

[0048] In the above device, optionally, the acquisition unit includes:

[0049] A first input subunit, configured to input the road image into a preset target detection deep learning model to obtain contour information of each traffic light on the road image;

[0050] an acquisition subunit, configured to acquire a detection image of each of the traffic lights in the road image based on the contour information of each of the traffic lights;

[0051] A second input subunit, used for inputting the detection image of each of the traffic lights into a preset image classification learning model to obtain category information of each of the traffic lights;

[0052] The first determining subunit is configured to determine, for each of the traffic lights, both the outline information and the category information of the traffic light as traffic light information.

[0053] In the above device, optionally, the determining unit includes:

[0054] A collection subunit, used for collecting point cloud data of the road, segmenting the point cloud data, and obtaining point clouds of multiple categories;

[0055] A second determination subunit is used to determine a point cloud that meets a preset category as a target point cloud based on the type information of each point cloud;

[0056] a third determining subunit, configured to determine a matching point cloud of each of the traffic lights in each of the target point clouds based on the traffic light information of each of the traffic lights;

[0057] The fourth determining subunit is used to determine, for each of the traffic lights, the point cloud coordinates of the matching point cloud of the traffic light as the point cloud coordinates of the traffic light.

[0058] In the above device, optionally, the third determining subunit includes:

[0059] A mapping module, used for mapping each of the target point clouds onto the road image to obtain a point cloud mapping image;

[0060] The first determination module is used to determine, for each of the traffic lights, a traffic light outline on the point cloud mapping image based on the traffic light information of the traffic light, and determine all target point clouds within the traffic light outline as matching point clouds of the traffic light.

[0061] In the above device, optionally, the clustering unit comprises:

[0062] A construction sub-unit, used for constructing a grid using the vehicle as a standard point, wherein the grid includes a plurality of sub-grids;

[0063] A fifth determining subunit, configured to determine a grid set of each of the traffic lights in each of the sub-grids, wherein the grid set includes a plurality of valid sub-grids;

[0064] The clustering subunit is used to cluster the point cloud coordinates of each traffic light using the grid set of the traffic light to obtain the center coordinates of the traffic light.

[0065] In the above device, optionally, the fifth determining subunit includes:

[0066] A second determination module is used to determine, for each of the point cloud coordinates, the subgrid where the point cloud coordinate is located, and determine the subgrid as a valid subgrid of the traffic light to which the point cloud coordinate belongs;

[0067] The composition module is used for composing each valid sub-grid of the traffic light into a grid set of the traffic light for each traffic light.

[0068] In the above device, optionally, the clustering subunit includes:

[0069] A clustering module, used for clustering each point cloud coordinate of the traffic light based on the grid set to obtain each cluster of the traffic light, wherein the cluster contains a plurality of point cloud coordinates;

[0070] A third determination module is used to determine the cluster with the largest number of point cloud coordinates as the target cluster;

[0071] The processing module is used to perform mean processing on the coordinates of each point cloud in the target cluster to obtain the center coordinates.

[0072] In the above device, optionally, the clustering module includes:

[0073] A selection submodule, used for selecting target point cloud coordinates from among the point cloud coordinates of the traffic light;

[0074] A determination submodule, used to determine adjacent subgrids of the target point cloud coordinates in each valid subgrid of the grid set;

[0075] A composition submodule, used for composing point cloud coordinates in adjacent subgrids of the target point cloud coordinates into clusters;

[0076] A marking submodule, used for marking each point cloud coordinate belonging to the cluster in the traffic light, and selecting a new target point cloud coordinate from the remaining unmarked point cloud coordinates;

[0077] The deleting submodule is used to delete each of the adjacent subgrids from the grid set, and return to execute the step of determining the adjacent subgrids of the target point cloud coordinates in each valid subgrid of the grid set until all the point cloud coordinates of the traffic light are marked.

[0078] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned traffic light data processing method.

[0079] An electronic device specifically includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to execute the one or more instructions to execute the above-mentioned traffic light data processing method.

[0080] Compared with the prior art, the present invention has the following advantages:

[0081] The present invention provides a traffic light data processing method and device, including: based on the road image of the road on which the vehicle is traveling, obtaining the traffic light information of each traffic light on the road; based on the traffic light information of each traffic light, determining each point cloud coordinate of each traffic light in the point cloud data of the road; performing clustering processing on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light; based on the center coordinate of each traffic light, screening out the traffic lights that meet the preset conditions. In the process of determining the traffic light, the traffic light is identified from the road image, and then combined with the point cloud data of the road, the final traffic light is determined from the identified traffic lights. The whole process does not require the construction of a high-precision map, effectively reducing the investment of funds and the cost required to identify the traffic light. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0083] Figure 1 A method flow chart of a traffic light data processing method provided by an embodiment of the present invention;

[0084] Figure 2 An example diagram of a road picture collected by a camera of a vehicle provided in an embodiment of the present invention;

[0085] Figure 3 An example diagram of a rendered driving scene provided by an embodiment of the present invention;

[0086] Figure 4 A flow chart of a method for obtaining traffic light information of each traffic light on a road based on a road image of a road on which a vehicle is traveling, provided by an embodiment of the present invention;

[0087] Figure 5 A flow chart of a method for determining the point cloud coordinates of each traffic light in the point cloud data of a road based on the traffic light information of each traffic light provided by an embodiment of the present invention;

[0088] Figure 6 A flow chart of a method for clustering the point cloud coordinates of each traffic light to obtain the center coordinates of each traffic light provided in an embodiment of the present invention;

[0089] Figure 7 A flow chart of traffic light detection provided by an embodiment of the present invention;

[0090] Figure 8A schematic diagram of the structure of a traffic light data processing device provided by an embodiment of the present invention;

[0091] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0093] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0094] As can be seen from the background technology, the traditional process of identifying and detecting traffic lights requires the creation of high-precision maps, which leads to extremely high costs for detection and identification.

[0095] In addition to the above problems, the process of traffic light detection and recognition also has these difficulties: 1. Traffic lights are small targets, which account for a very small proportion of the image and contain limited features, so there are high requirements for the design of target detection neural networks. 2. Traffic lights are static objects, but their categories change over time, which increases the difficulty of detection. 3. Under different light intensities, the colors of red and yellow lights are similar, which can easily lead to model classification errors. 4. The types of traffic lights in different regions vary, which has high requirements for data collection. The traffic light detection network needs to have strong generalization performance. 5. Purely visual traffic light detection and recognition can only obtain the position and category of traffic lights in 2D images, but cannot obtain their distance information.

[0096] At present, there are three solutions for traffic light detection and recognition of autonomous vehicles: pure visual perception, combination of vision and high-precision map, and V2X (Vehicle to X). The pure visual perception solution is low-cost, but it is easily affected by weather and lighting conditions, and it cannot obtain the distance information between the traffic light and the vehicle. The vision and high-precision map combined perception solution obtains the area of ​​interest of the traffic light through the high-precision map, and then detects and identifies the area of ​​interest. This solution gives priority to the traffic lights that the vehicle needs to pay attention to, significantly reducing the false detection and misidentification of irrelevant traffic lights. This solution relies on high-precision maps, but the construction of high-precision maps is cumbersome and expensive. The V2X solution, that is, the autonomous vehicle receives the signal status of the traffic light in real time. The output result of this solution is accurate, but it requires the modification of traffic light equipment and requires a faster network transmission rate. Large-scale implementation requires strong infrastructure and support from government departments, and commercial deployment is difficult.

[0097] In order to solve the above-mentioned problems, the present invention provides a traffic light data processing method and device, which does not require the creation of a high-precision map when identifying and detecting traffic lights, thereby reducing the identification and detection costs of traffic lights.

[0098] The present invention can be used in many general or special computing device environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or devices, etc. Preferably, the execution subject of the present invention can be an intelligent driving system of a vehicle.

[0099] Reference Figure 1 , is a flow chart of a method for processing traffic light data provided by an embodiment of the present invention, and is specifically described as follows:

[0100] S101. Obtain traffic light information of each traffic light on the road based on a road image of the road on which the vehicle is traveling.

[0101] The road image in front of the road where the vehicle is located is collected. Preferably, the road image can be collected using the vehicle's front-view camera.

[0102] The road image is processed using a target detection deep learning model and an image classification learning model, so that traffic light information of each traffic light in the road image can be obtained. The traffic light information includes the category information, location information, etc. of the traffic light.

[0103] S102: Based on the traffic light information of each traffic light, determine the point cloud coordinates of each traffic light in the point cloud data of the road.

[0104] The point cloud data of the road is collected, and then the point cloud data is segmented to obtain multiple categories of point clouds, each of which has point cloud coordinates.

[0105] For each traffic light, based on the position information of the traffic light information of the traffic light, a matching point cloud is determined in each point cloud, and then the point cloud coordinates of the point cloud are determined as the point cloud coordinates of the traffic light.

[0106] Preferably, each traffic light has multiple point cloud coordinates.

[0107] S103 , clustering the point cloud coordinates of each traffic light to obtain the center coordinates of each traffic light.

[0108] When clustering the point cloud coordinates of each traffic light, a breadth-first algorithm, a K-means clustering algorithm, or a Mean shift clustering algorithm may be used for clustering.

[0109] S104: Filter out traffic lights that meet preset conditions based on the center coordinates of each traffic light.

[0110] Furthermore, the center coordinates of the traffic light can be understood as the three-dimensional information of the traffic light, and the three-dimensional information of the traffic light can be obtained by obtaining the center coordinates of the traffic light. The center coordinates include coordinate data of the X axis, the Y axis, and the Z axis.

[0111] Preferably, in the process of screening each traffic light, the traffic lights that do not meet the ODD (Operational Design Domain) conditions can be filtered out first. The ODD conditions specifically include: the traffic lights are not blocked, the traffic lights are not surrounded by dense plants, the height of the traffic lights is above 3.5m, the traffic lights are not tunnel guide lights, and the traffic lights are 10m in front of the vehicle. After filtering out the traffic lights that do not meet the ODD conditions, motion compensation is performed on the remaining traffic lights, and then target tracking processing is performed, and then the traffic lights within 3 meters of the rear axle center of the vehicle are filtered out, and then the remaining traffic lights are processed by NMS (non-maximum suppression). At this point, traffic lights that meet the preset conditions can be obtained.

[0112] The present invention obtains traffic light information of each traffic light on the road based on the road image of the road on which the vehicle is traveling; based on the traffic light information of each traffic light, determines each point cloud coordinate of each traffic light in the point cloud data of the road; performs clustering processing on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light; based on the center coordinate of each traffic light, screens out traffic lights that meet preset conditions. In the process of determining the traffic light, the traffic light is identified from the road image, and then the final traffic light is determined from the identified traffic lights in combination with the point cloud data of the road. The whole process does not require the construction of a high-precision map, effectively reducing the investment of funds and the cost required to identify the traffic light.

[0113] In the method provided by an embodiment of the present invention, after obtaining a traffic light that meets preset conditions, the traffic light information and three-dimensional information of the traffic light can be fed back to the vehicle's automatic driving decision system, so that the automatic driving decision system constructs the vehicle's driving scene based on the traffic light information and the three-dimensional information and determines the vehicle's driving strategy.

[0114] It should be noted that the autonomous driving decision system will render the vehicle's driving scene on the human-computer interaction interface based on traffic light information and three-dimensional information. Figure 2 , which is an example of a road picture collected by a vehicle's camera. Based on the collected road picture, a driving scene is rendered. Figure 3 shown.

[0115] By sending the information of the finally screened traffic lights to the vehicle's autonomous driving decision system, the vehicle's autonomous driving decision system can render a 3D driving scene and provide the driver with intuitive traffic light conditions, making it easier for the driver to understand the road conditions. The autonomous driving decision system can quickly execute the corresponding driving strategy and make the most appropriate driving strategy at the moment to ensure the safety of the vehicle driver, pedestrians and other road users.

[0116] In another example provided by the present invention, the above process of obtaining the traffic light information of the traffic light is described. Figure 4 , is a flow chart of a method for obtaining traffic light information of each traffic light on a road based on a road image of a road on which a vehicle is traveling, provided by an embodiment of the present invention, and is specifically described as follows:

[0117] S201. Input a road image into a preset target detection deep learning model to obtain contour information of each traffic light on the road image.

[0118] The object detection deep learning model is a trained model that is used to identify traffic lights in road images.

[0119] Preferably, the outline information of the traffic light includes the position information, size information, etc. of the traffic light in the road image.

[0120] S202: Based on the contour information of each traffic light, a detection image of each traffic light is obtained in the road image.

[0121] According to the contour information of the traffic light, the traffic light is cropped out from the road image to obtain a detection image of the traffic light.

[0122] S203: Input the detection image of each traffic light into a preset image classification learning model to obtain category information of each traffic light.

[0123] The image classification learning model is a trained model used to identify the category of traffic lights. Preferably, traffic lights are divided into red, yellow, green and black categories, and the category information includes the category information of the lights identified by the image classification learning model.

[0124] S204: For each traffic light, determine both the outline information and the category information of the traffic light as traffic light information.

[0125] By using the target detection deep learning model and the image classification learning model, traffic light information can be extracted from road images. The entire process does not require the construction of a high-precision map. Traffic lights can be detected and their accurate locations can be obtained, effectively reducing costs.

[0126] In another example provided by the present invention, the above process of determining the coordinates of each point cloud of each traffic light is described, referring to Figure 5 , which is a flow chart of a method for determining the point cloud coordinates of each traffic light in the point cloud data of a road based on the traffic light information of each traffic light provided by an embodiment of the present invention, and is specifically described as follows:

[0127] S301, collecting point cloud data of the road, segmenting the point cloud data, and obtaining point clouds of multiple categories.

[0128] LiDAR can be used to collect point cloud data of the road in real time, and the point cloud data can be sent to the laser point cloud segmentation deep learning model to obtain the point clouds output by the laser point cloud segmentation deep learning model.

[0129] It should be noted that the obtained point cloud has corresponding type information, and the type information can characterize the type of the point cloud, such as support pole point cloud, obstacle point cloud, road surface point cloud, sidewalk point cloud, etc.

[0130] S302: Based on the type information of each point cloud, determine a point cloud that meets a preset category as a target point cloud.

[0131] Preferably, the preset category can be a preset point cloud type, and the point cloud with the same preset category is determined as the target point cloud. For example, if the preset category is support rod and high-altitude object, the support rod point cloud and the high-altitude object point cloud are both determined as target point clouds.

[0132] S303: Based on the traffic light information of each traffic light, determine the matching point cloud of each traffic light in each target point cloud.

[0133] The process of determining the matching point cloud for the traffic light is as follows:

[0134] Map each target point cloud onto the road image to obtain a point cloud mapping image;

[0135] For each traffic light, based on the traffic light information of the traffic light, the outline of the traffic light is determined on the point cloud mapping image, and the target point clouds located within the outline of the traffic light are all determined as matching point clouds of the traffic light.

[0136] It should be noted that after mapping each target point cloud onto the road image, for each traffic light, based on the contour information of the traffic light, the target point cloud located in the contour of the traffic light is determined as the matching point cloud of the traffic light, and there are multiple matching point clouds for the traffic light.

[0137] S304: For each traffic light, determine the point cloud coordinates of the matching point cloud of the traffic light as the point cloud coordinates of the traffic light.

[0138] It should be noted that each point cloud has point cloud coordinates. After the matching point cloud of the traffic light is determined, the coordinates of each point cloud of the traffic light can be obtained.

[0139] The present invention collects point cloud data of the road, processes the point cloud data, and obtains the point cloud coordinates of each traffic light. This can be used to perform subsequent screening of traffic lights in combination with actual road conditions, thereby improving the accuracy of screening.

[0140] In another example provided by the present invention, the above process of determining the center coordinates of the traffic light is described, referring to Figure 6 , which is a flow chart of a method for clustering the point cloud coordinates of each traffic light to obtain the center coordinates of each traffic light provided by an embodiment of the present invention, and is specifically described as follows:

[0141] S401, constructing a grid using the vehicle as a standard point, wherein the grid includes multiple sub-grids.

[0142] It should be noted that the network is constructed with the driver's vehicle (i.e., the vehicle coordinate system) as the standard point. The grid contains multiple sub-grids, and the size of the sub-grid can be expressed as (w, h), where w represents the length of the sub-grid and h represents the width of the sub-grid. It should be noted that each sub-grid has corresponding coordinates in the grid, and the coordinates can represent the specific position of the sub-grid in the grid.

[0143] Preferably, the constructed grid is represented by G(x, y), and the sub-grid can be represented by G(i, j), wherein i and j represent the specific coordinates of the sub-grid in the grid G.

[0144] S402: Determine a grid set of each traffic light in each sub-grid, where the grid set includes a plurality of valid sub-grids.

[0145] The process of determining the grid set of traffic lights is as follows:

[0146] For each point cloud coordinate, determine the subgrid where the point cloud coordinate is located, and determine the subgrid as a valid subgrid of the traffic light to which the point cloud coordinate belongs, and further, the valid subgrid covers the point cloud coordinate;

[0147] For each traffic light, each valid sub-grid of the traffic light is grouped into a grid set of the traffic light.

[0148] Preferably, a sub-grid that does not cover the point cloud coordinates is determined as an invalid grid.

[0149] S403: For each traffic light, cluster the point cloud coordinates of the traffic light using the grid set of the traffic light to obtain the center coordinates of the traffic light.

[0150] The process of clustering the point cloud coordinates of each traffic light is the same. Here, the process of clustering the point cloud coordinates of a traffic light is described, and the specific description is shown in S501-S503.

[0151] S501 , clustering each point cloud coordinate of the traffic light based on the grid set to obtain each cluster of the traffic light, wherein the cluster contains multiple point cloud coordinates.

[0152] It should be noted that the specific process of obtaining each cluster of traffic lights is shown in S601 to S606, and the specific description is as follows:

[0153] S601 . Select target point cloud coordinates from among various point cloud coordinates of the traffic light.

[0154] Preferably, when selecting the target point cloud coordinates, the selection may be made randomly, or the point cloud coordinates may be sorted and then selected from front to back; the present invention does not limit the method for selecting the target point cloud coordinates.

[0155] Preferably, when sorting the valid sub-grids, the point cloud coordinates of the traffic lights may be sorted first, and after sorting the point cloud coordinates of the traffic lights, the valid sub-grids in the grid set of the traffic lights are also sorted.

[0156] S602: Determine adjacent sub-grids of the target point cloud coordinates in each valid sub-grid of the grid set of the traffic light.

[0157] It should be noted that the number of adjacent sub-grids of the target point cloud coordinates is at least one.

[0158] Preferably, the process of determining the adjacent sub-grids of the target point cloud coordinates may also be described as follows:

[0159] Step 1: Determine the valid sub-grid to which the target point cloud coordinates belong, and determine the valid sub-grid as the target sub-grid.

[0160] Step 2: Based on the target sub-grid, determine each neighboring sub-grid.

[0161] It should be noted that the neighborhood subgrids are subgrids located around the target subgrid in the grid. For example, if the target subgrid is (4,5), then the neighborhood subgrids are (3,5), (5,5), (4,4) and (4,6). Preferably, the neighborhood subgrids are represented by coordinates.

[0162] Step 3: For each neighborhood sub-grid, determine whether there is a valid sub-grid corresponding to the neighborhood sub-grid in the grid set of the traffic light; when there is a valid sub-grid corresponding to the neighborhood sub-grid in the grid set of the traffic light, execute step 4; when there is no valid sub-grid corresponding to the neighborhood sub-grid in the grid set of the traffic light, execute step 5.

[0163] Step 4: After determining that the neighborhood sub-grid is an adjacent sub-grid of the target point cloud coordinates and determining the neighborhood sub-grid as a new target sub-grid, return to step 2.

[0164] Step 5: Determine whether the neighborhood sub-grid is not an adjacent sub-grid of the target point cloud coordinates.

[0165] Thus, each adjacent sub-grid of the target point cloud coordinates can be obtained.

[0166] S603 , clustering the point cloud coordinates in the adjacent sub-grids of the target point cloud coordinates.

[0167] It should be noted that the cluster contains the point cloud coordinates in each adjacent sub-grid and also the target point cloud coordinates.

[0168] S604: Mark each point cloud coordinate belonging to a cluster in the traffic light, and select a new target point cloud coordinate from the remaining unmarked point cloud coordinates.

[0169] It should be noted that, based on the clustering clusters, each point cloud coordinate of the traffic light is marked, that is, each point cloud coordinate belonging to the clustering cluster is marked, and then a new target point cloud coordinate is selected from the remaining unmarked point cloud coordinates of the traffic light.

[0170] S605 , deleting each adjacent sub-grid from the grid set, and returning to execute step S602 until all point cloud coordinates of the traffic light are marked.

[0171] After selecting new target point cloud coordinates and deleting each adjacent subgrid from the grid set of the traffic light, the grid set of the traffic light is updated, and then the step S602 is returned to be executed until all the point cloud coordinates of the traffic light are marked, and the point cloud coordinates of the traffic light are clustered, so that multiple clusters of the traffic light can be obtained. Preferably, k clusters C are obtained. k .

[0172] S502: Determine the cluster with the largest number of point cloud coordinates as the target cluster.

[0173] Preferably, the target cluster can be C i .

[0174] S503: performing mean processing on the coordinates of each point cloud in the target cluster to obtain the center coordinates.

[0175] It should be noted that the point cloud coordinates can be expressed as (x, y, z); preferably, the number of point cloud coordinates in the target cluster is j, and the center coordinates are expressed as in,

[0176] The present invention provides a method for clustering the point cloud coordinates of a traffic light using a clustering algorithm, creating a grid containing multiple sub-grids based on the position of a vehicle, and then clustering each point cloud coordinate in combination with the sub-grids, thereby obtaining the central coordinates representing the traffic light, and then performing path planning and rendering of a driving scene based on the central coordinates.

[0177] Reference Figure 7 , which is a flow chart of traffic light detection provided by an embodiment of the present invention, is specifically described as follows.

[0178] Step 1: pre-process the road image collected in real time by the vehicle's front-view camera, and send it to the target detection deep learning model for processing to obtain the position of the traffic light in the road image; preferably, the road image is a 2D image, and the target detection deep learning module is specifically such as YOLO X.

[0179] Step 2: Cut out the traffic lights in the road image and send them to the image classification deep learning model for processing to obtain the category of the traffic lights; preferably, the image classification deep learning model can be MobileNet.

[0180] Step 3: Perform visual target tracking on the detected and identified traffic lights. The specific process is to first initialize the detections (detection results) detected in the first frame image as trackers (tracking results). Based on the trackers, the second frame image is predicted according to the Kalman filter algorithm to obtain new trackers. The Hungarian algorithm is used to cascade match the new trackers with the detections of the current frame to obtain a (trackers, detections) matching pair. Finally, the corresponding predicted trackers are updated based on the detections.

[0181] Through steps 1 to 3, the traffic light information of the traffic light in the road image, ie, the position information and the category information, can be obtained.

[0182] Step 4, pre-process the point cloud data of the road collected by the laser radar in real time, and send it to the laser point cloud segmentation deep learning model (such as SalsaNext) for processing to obtain the segmented point cloud. It should be noted that there are ten types of point clouds, such as: 0_road surface, 1_sidewalk, 2_obstacle, 3_high-altitude object, 4_support pole, 5_traffic sign, 6_traffic cone, 7_car or person, 8_wheel block, 9_unknown target; preferably, traffic lights have a high similarity with 3_high-altitude objects and 4_support poles. Therefore, the present invention subsequently processes the point clouds of the two categories of 3_high-altitude objects and 4_support poles. The target point cloud can be screened out from the segmented point cloud. Preferably, the target point cloud is the high-altitude object point cloud and the support rod point cloud, and then motion compensation is performed on the target point cloud according to the motion information of the vehicle. The motion compensation is used to perform motion compensation on the point cloud data frame generated by the lidar according to the measured vehicle motion state parameters to reduce the posture error of the point cloud data caused by motion; preferably, the motion information includes but is not limited to the yaw angle, roll angle, pitch angle and speed in the X, Y and Z directions.

[0183] Step 5: Project the target point cloud obtained in step 4 onto the road image, where the formula for converting BEV (Bird's Eye View) coordinates to pixel coordinates is:

[0184]

[0185]

[0186] It should be noted that in formula (1) and formula (2), (u, v) represents the pixel coordinates. u Represents the target in the image pixel coordinate system u The value of the axis direction, v Represents the target in the image pixel coordinate system v The value of the axis direction; (xc ,y c ,z c ) represents the coordinates of the target point cloud in the camera coordinate system, x c Represents the target point cloud in the camera coordinate system x Axis direction value, y c Represents the value of the target point cloud in the y-axis direction in the camera coordinate system, z c Represents the value of the target point cloud in the z-axis direction in the camera coordinate system; (x, y, z) represents the coordinates of the target point cloud in the BEV coordinate system. x Represents the target point cloud in the BEV coordinate system x The value in the axis direction, y represents the value of the target point cloud in the y-axis direction in the BEV coordinate system, and z represents the value of the target point cloud in the z-axis direction in the BEV coordinate system; I is the camera intrinsic parameter, and E is the camera extrinsic parameter.

[0187] It should be noted that if the pixel coordinates of the target point cloud are within the traffic light detection frame of the road image, it is considered that the target point cloud is successfully matched with the traffic light to which the traffic light detection frame belongs, and the point cloud coordinates of the target point cloud are the point cloud coordinates of the traffic light. It should be noted that there may be multiple traffic light detection frames in the road image. Furthermore, the traffic light detection frame can also be called a traffic light Box.

[0188] Step 6: For each traffic light, use the breadth-first traversal algorithm to cluster the target point cloud successfully matched in step 5, fuse the cluster center points, and calculate the center coordinates of the traffic light in the BEV coordinate system. Preferably, the center coordinates can also be understood as the 3D information of the traffic light.

[0189] The process of clustering the point cloud coordinates of the traffic light is as follows: Step 6-1 to Step 6-6:

[0190] Step 6-1, based on the vehicle coordinate system, a grid G(x,y) is established according to the position of the vehicle. The grid contains multiple subgrids, each subgrid size is (w,h), and the position of the subgrid in the grid is G(i,j). For each point cloud coordinate of the traffic light, determine the subgrid to which each point cloud coordinate belongs, and determine the subgrid to which the point cloud coordinate belongs as a valid subgrid, that is, when a point cloud coordinate exists in a subgrid, the subgrid is a valid subgrid, and when a point cloud coordinate does not exist in a subgrid, the subgrid is an invalid subgrid, and the invalid subgrid can be discarded.

[0191] Step 6-2: Determine the target point cloud coordinates in each point cloud coordinate of the traffic light.

[0192] Preferably, when determining the target point cloud coordinates, the point cloud coordinates may be sorted, and after the point cloud coordinates are sorted, the valid subgrids of the traffic light are also sorted.

[0193] Preferably, the target point cloud coordinates may be selected in order of sorting.

[0194] Step 6-3: Determine the valid sub-grid to which the target point cloud coordinates belong, and determine the valid sub-grid as the target sub-grid.

[0195] Step 6-4: Determine the neighborhood sub-grid G(i',j') of the target sub-grid.

[0196] When determining the neighborhood subgrid G(i',j'), the following formula can be used:

[0197] G(i',j')=G(i+(-1) p ,j+(-1) q ), where p and q are both 0 or 1.

[0198] Thus, multiple neighboring sub-grids of the target sub-grid can be obtained.

[0199] Step 6-5: For each neighborhood sub-grid, determine whether there is a valid sub-grid corresponding to the neighborhood sub-grid in the grid set; if so, execute step 6-6; if not, execute step 6-7.

[0200] Step 6-6: Determine that the neighborhood subgrid is an adjacent subgrid of the target point cloud coordinates, and determine the neighborhood subgrid as a new target subgrid, and then return to execute step 6-4 until there is no adjacent subgrid.

[0201] Step 6-7: Determine whether the neighborhood sub-grid is an adjacent sub-grid of the target point cloud coordinates.

[0202] Step 6-8: Aggregate the point cloud coordinates in all adjacent sub-grids to obtain clusters.

[0203] Preferably, the point cloud coordinates in the cluster include the target point cloud coordinates.

[0204] Step 6-9, mark each point cloud coordinate belonging to the cluster, delete the valid sub-grid to which the marked point cloud coordinate belongs from the grid set, and then select new target point cloud coordinates from the remaining unmarked point cloud coordinates, and then return to step 6-4 until all point cloud coordinates are marked.

[0205] Step 6-10: Get k clusters of traffic lights, and take the cluster with the largest number of point clouds as the target cluster.

[0206] Step 6-11: For each point cloud coordinate in the target cluster, calculate the mean and obtain the center coordinate of the traffic light.

[0207] The center coordinates are expressed as in, is the mean value of the x-axis of each point cloud coordinate in the target cluster. is the mean value of the y-axis of each point cloud coordinate in the target cluster. It is the mean value of the z-axis of each point cloud coordinate in the target cluster.

[0208] At this point, the center coordinates of the traffic light can be obtained.

[0209] Step 7: Filter out traffic lights that do not meet the ODD (Operational Design Domain) conditions based on factors such as height, distance, and angle. The ODD conditions are: the traffic lights are unobstructed, not surrounded by dense plants, are above 3.5 meters in height, are not tunnel guide lights, and are 10 meters in front of the vehicle.

[0210] Step 8: Perform motion compensation on the traffic lights with 3D information according to the vehicle motion information. The compensation method is as follows:

[0211]

[0212] In formula (3), (x t ,y t ,z t ) represents the coordinate of the traffic light at time t in the BEV coordinate system, (x t-Δt ,y t-Δt ,z t-Δt ) is the coordinate of the traffic light in the BEV coordinate system at time t-Δt, R represents the rotation of the vehicle in the Δt time interval, and T represents the translation position change in the Δt time interval. After the compensation is completed, the target tracking processing in the BEV coordinate system is performed (here, the target tracking object in the BEV coordinate system is the center point of the traffic light in the BEV coordinate system, and the target tracking object in step 3 is the 2D Box in the 2D image domain), and the method is the same as step 3.

[0213] Step 9: Perform post-processing such as filtering the traffic lights processed in step 8 within 3 m of the rear axle center of the vehicle, and performing NMS (non-maximum suppression) and the like.

[0214] Step 10: Output the final result to the HMI (human-machine interface) or the autonomous driving decision system. The HMI scene reconstruction effect is shown in Figure 3 Example.

[0215] Based on the image collected by the forward-looking high-definition camera and the point cloud collected by the laser radar, the present invention uses the target detection deep learning model and the image classification deep learning model to realize the detection and recognition of traffic lights in the image domain, and uses the laser point cloud segmentation deep learning model to realize the laser point cloud semantic segmentation, and projects the point cloud onto the 2D image to match it with the traffic light box. The 3D information of the successfully matched traffic light is obtained, and then the traffic light is subjected to target tracking processing under BEV, and finally output to the HMI or the autonomous driving decision system. This technology can be applied to when an autonomous driving car passes in front of a traffic intersection. By real-time detection and identification of the location and category of traffic lights, the autonomous driving decision system can quickly execute the next step of path planning and control to ensure the safety of the car, pedestrians and other road users.

[0216] and Figure 1 Corresponding to the method shown in the figure, the embodiment of the present invention also provides a traffic light data processing device, which can be set in the intelligent driving system of the vehicle, and the device is used to support Figure 1 Specific implementation of the method shown.

[0217] Reference Figure 8 , is a schematic diagram of the structure of a traffic light data processing device provided by an embodiment of the present invention, and is specifically described as follows:

[0218] An acquisition unit 801 is used to acquire traffic light information of each traffic light on the road based on a road image of the road on which the vehicle is traveling;

[0219] A determining unit 802, configured to determine the point cloud coordinates of each of the traffic lights in the point cloud data of the road based on the traffic light information of each of the traffic lights;

[0220] A clustering unit 803 is used to perform clustering processing on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light;

[0221] The screening unit 804 is used to screen out the traffic lights that meet the preset conditions based on the center coordinates of the traffic lights.

[0222] The present invention obtains traffic light information of each traffic light on the road based on the road image of the road on which the vehicle is traveling; based on the traffic light information of each traffic light, determines each point cloud coordinate of each traffic light in the point cloud data of the road; performs clustering processing on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light; based on the center coordinate of each traffic light, screens out traffic lights that meet preset conditions. In the process of determining the traffic light, the traffic light is identified from the road image, and then the final traffic light is determined from the identified traffic lights in combination with the point cloud data of the road. The whole process does not require the construction of a high-precision map, effectively reducing the investment of funds and the cost required to identify the traffic light.

[0223] In another embodiment provided by the present invention, the acquisition unit 801 of the device includes:

[0224] A first input subunit, configured to input the road image into a preset target detection deep learning model to obtain contour information of each traffic light on the road image;

[0225] an acquisition subunit, configured to acquire a detection image of each of the traffic lights in the road image based on the contour information of each of the traffic lights;

[0226] A second input subunit, used for inputting the detection image of each of the traffic lights into a preset image classification learning model to obtain category information of each of the traffic lights;

[0227] The first determining subunit is configured to determine, for each of the traffic lights, both the outline information and the category information of the traffic light as traffic light information.

[0228] In another embodiment provided by the present invention, the determining unit 802 of the device includes:

[0229] A collection subunit, used for collecting point cloud data of the road, segmenting the point cloud data, and obtaining point clouds of multiple categories;

[0230] A second determination subunit is used to determine a point cloud that meets a preset category as a target point cloud based on the type information of each point cloud;

[0231] a third determining subunit, configured to determine a matching point cloud of each of the traffic lights in each of the target point clouds based on the traffic light information of each of the traffic lights;

[0232] The fourth determining subunit is used to determine, for each of the traffic lights, the point cloud coordinates of the matching point cloud of the traffic light as the point cloud coordinates of the traffic light.

[0233] In another embodiment provided by the present invention, the third determining subunit of the device includes:

[0234] A mapping module, used for mapping each of the target point clouds onto the road image to obtain a point cloud mapping image;

[0235] The first determination module is used to determine, for each of the traffic lights, a traffic light outline on the point cloud mapping image based on the traffic light information of the traffic light, and determine all target point clouds within the traffic light outline as matching point clouds of the traffic light.

[0236] In another embodiment provided by the present invention, the clustering unit 803 of the device includes:

[0237] A construction sub-unit, used for constructing a grid using the vehicle as a standard point, wherein the grid includes a plurality of sub-grids;

[0238] A fifth determining subunit, configured to determine a grid set of each of the traffic lights in each of the sub-grids, wherein the grid set includes a plurality of valid sub-grids;

[0239] The clustering subunit is used to cluster the point cloud coordinates of each traffic light using the grid set of the traffic light to obtain the center coordinates of the traffic light.

[0240] In another embodiment provided by the present invention, the fifth determining subunit of the device includes:

[0241] A second determination module is used to determine, for each of the point cloud coordinates, the subgrid where the point cloud coordinate is located, and determine the subgrid as a valid subgrid of the traffic light to which the point cloud coordinate belongs;

[0242] The composition module is used for composing each valid sub-grid of the traffic light into a grid set of the traffic light for each traffic light.

[0243] In another embodiment provided by the present invention, the clustering subunit of the device includes:

[0244] A clustering module, used for clustering each point cloud coordinate of the traffic light based on the grid set to obtain each cluster of the traffic light, wherein the cluster contains a plurality of point cloud coordinates;

[0245] A third determination module is used to determine the cluster with the largest number of point cloud coordinates as the target cluster;

[0246] The processing module is used to perform mean processing on the coordinates of each point cloud in the target cluster to obtain the center coordinates.

[0247] In another embodiment provided by the present invention, the clustering module of the device includes:

[0248] A selection submodule, used for selecting target point cloud coordinates from among the point cloud coordinates of the traffic light;

[0249] A determination submodule, used to determine adjacent subgrids of the target point cloud coordinates in each valid subgrid of the grid set;

[0250] A composition submodule, used for composing point cloud coordinates in adjacent subgrids of the target point cloud coordinates into clusters;

[0251] A marking submodule, used for marking each point cloud coordinate belonging to the cluster in the traffic light, and selecting a new target point cloud coordinate from the remaining unmarked point cloud coordinates;

[0252] The deleting submodule is used to delete each of the adjacent subgrids from the grid set, and return to execute the step of determining the adjacent subgrids of the target point cloud coordinates in each valid subgrid of the grid set until all the point cloud coordinates of the traffic light are marked.

[0253] An embodiment of the present invention further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned traffic light data processing method.

[0254] The embodiment of the present invention further provides an electronic device, the structural diagram of which is shown in FIG. Fig. 9 As shown, it specifically includes a memory 901 and one or more instructions 902, wherein the one or more instructions 902 are stored in the memory 901 and are configured to be executed by one or more processors 903 to perform the above-mentioned traffic light data processing method.

[0255] The specific implementation processes and derivative methods of the above-mentioned embodiments are all within the protection scope of the present invention.

[0256] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0257] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0258] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A traffic light data processing method, characterized in that: include: Based on the road image of the road on which the vehicle is traveling, acquiring traffic light information of each traffic light on the road; Based on the traffic light information of each of the traffic lights, determining the point cloud coordinates of each of the traffic lights in the point cloud data of the road; Performing clustering processing on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light; Based on the central coordinates of each of the traffic lights, the traffic lights that meet the preset conditions are screened out.

2. The method according to claim 1, characterized in that The acquiring, based on the road image of the road on which the vehicle is traveling, traffic light information of each traffic light on the road includes: Inputting the road image into a preset target detection deep learning model to obtain contour information of each traffic light on the road image; Based on the contour information of each of the traffic lights, acquiring a detection image of each of the traffic lights in the road image; Inputting the detection image of each of the traffic lights into a preset image classification learning model to obtain category information of each of the traffic lights; For each of the traffic lights, the outline information and the category information of the traffic light are determined as traffic light information.

3. The method according to claim 1, characterized in that The step of determining the point cloud coordinates of each traffic light in the point cloud data of the road based on the traffic light information of each traffic light comprises: Collecting point cloud data of the road, segmenting the point cloud data, and obtaining point clouds of multiple categories; Based on the type information of each point cloud, determining a point cloud that meets a preset category as a target point cloud; Based on the traffic light information of each of the traffic lights, determining a matching point cloud of each of the traffic lights in each of the target point clouds; For each of the traffic lights, the point cloud coordinates of the matching point cloud of the traffic light are determined as the point cloud coordinates of the traffic light.

4. The method according to claim 3, characterized in that The determining, based on the traffic light information of each of the traffic lights, a matching point cloud of each of the traffic lights in each of the target point clouds comprises: Mapping each of the target point clouds onto the road image to obtain a point cloud mapping image; For each of the traffic lights, based on the traffic light information of the traffic light, a traffic light outline is determined on the point cloud mapping image, and target point clouds within the traffic light outline are all determined as matching point clouds of the traffic light.

5. The method according to claim 1, characterized in that The clustering process is performed on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light, including: Using the vehicle as a standard point to construct a grid, the grid comprising a plurality of sub-grids; Determine a grid set of each of the traffic lights in each of the sub-grids, wherein the grid set includes a plurality of valid sub-grids; For each of the traffic lights, the grid set of the traffic light is used to cluster the point cloud coordinates of the traffic light to obtain the center coordinates of the traffic light.

6. The method according to claim 5, characterized in that The step of determining a grid set of each of the traffic lights in each of the sub-grids includes: For each of the point cloud coordinates, determining the subgrid where the point cloud coordinate is located, and determining the subgrid as a valid subgrid of the traffic light to which the point cloud coordinate belongs; For each of the traffic lights, each valid sub-grid of the traffic light is combined into a grid set of the traffic light.

7. The method according to claim 5, characterized in that The applying the grid set of the traffic light to cluster the point cloud coordinates of the traffic light to obtain the center coordinates of the traffic light includes: Based on the grid set, clustering each point cloud coordinate of the traffic light to obtain each cluster of the traffic light, wherein the cluster contains a plurality of point cloud coordinates; The cluster with the largest number of point cloud coordinates is determined as the target cluster; The coordinates of each point cloud in the target cluster are averaged to obtain the center coordinates.

8. The method according to claim 7, characterized in that The step of clustering the point cloud coordinates of the traffic light based on the grid set to obtain clusters of the traffic light includes: Selecting target point cloud coordinates from among the point cloud coordinates of the traffic light; Determine the adjacent sub-grids of the target point cloud coordinates in each valid sub-grid of the grid set; Clustering point cloud coordinates in adjacent subgrids of the target point cloud coordinates; Marking each point cloud coordinate belonging to the cluster in the traffic light, and selecting new target point cloud coordinates from the remaining unmarked point cloud coordinates; Each of the adjacent sub-grids is deleted from the grid set, and the step of determining the adjacent sub-grid of the target point cloud coordinates in each valid sub-grid of the grid set is returned to be executed until all the point cloud coordinates of the traffic light are marked.

9. The method according to claim 1, characterized in that: Also includes: The traffic light information and three-dimensional information of the traffic lights that meet the preset conditions are fed back to the automatic driving decision system of the vehicle, so that the automatic driving decision system constructs the driving scene of the vehicle and determines the driving strategy of the vehicle based on the traffic light information and the three-dimensional information.

10. A traffic light data processing device, characterized in that: include: an acquisition unit, configured to acquire traffic light information of each traffic light on the road based on a road image of the road on which the vehicle is traveling; a determining unit, configured to determine the point cloud coordinates of each of the traffic lights in the point cloud data of the road based on the traffic light information of each of the traffic lights; A clustering unit, used for performing clustering processing on each point cloud coordinate of each traffic light to obtain the center coordinate of each traffic light; The screening unit is used to screen out the traffic lights that meet the preset conditions based on the center coordinates of each of the traffic lights.

Images