A method and device for generating training samples

Abstract

This specification discloses a method and apparatus for generating training samples. For each traffic light in a constructed high-precision map, the central geographic coordinates of the traffic light are determined based on the traffic light's vector data. A collection range is then determined with the traffic light as the center. Laser points within the collection range within the collected point cloud data are identified as sample data. The traffic light's vector data is used to determine the annotation box for the traffic light in the sample data, which serves as the sample annotation. By utilizing the vector data of the traffic light in the constructed high-precision map to determine the correspondence between the point cloud data and the annotation box, the time-consuming and labor-intensive manual annotation process is avoided, improving annotation efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for generating training samples. Background Art

[0002] With the development of computer technology, the automated production of high-precision maps is also evolving. Traffic lights, as a core element of traffic scenes, are often required to be included in high-precision maps.

[0003] Typically, when building high-precision maps, a pre-trained deep learning model is used to detect or segment point cloud data, identifying the points within the point cloud that belong to traffic lights and, therefore, determining their geographic locations. Training a highly accurate deep learning model requires a large number of training samples.

[0004] In existing technologies, training samples require manual annotation, where the 3D bounding boxes of traffic lights are annotated in point cloud data. However, manual annotation is time-consuming, inefficient, and labor-intensive. Therefore, a method for automatically generating training samples from point cloud data is urgently needed. Summary of the Invention

[0005] The embodiments of this specification provide a method and apparatus for generating training samples, which are used to at least partially solve the problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a method for generating training samples, including:

[0008] Obtaining a constructed high-precision map, parsing the vector data of each traffic light in the high-precision map, and determining the central geographic coordinates of each traffic light;

[0009] For each traffic light, determine a collection range corresponding to the traffic light according to the central geographic coordinates of the traffic light and a preset first distance;

[0010] Determining, from the collected point cloud data, laser points that fall within the collection range as sample data of training samples corresponding to the traffic light;

[0011] Based on the vector data of the traffic light in the high-precision map, the annotation box of the traffic light in the sample data is determined, and a training sample is generated for training the traffic light recognition model.

[0012] Optionally, parsing the vector data of each traffic light in the high-precision map to determine the center geographic coordinates of each traffic light specifically includes:

[0013] Determining data corresponding to a three-dimensional bounding box of each traffic light based on the vector data of each traffic light in the high-precision map;

[0014] For each traffic light, the central geographic coordinates of the traffic light are determined based on the data corresponding to the three-dimensional bounding box of the traffic light.

[0015] Optionally, determining laser points that fall within the acquisition range from the acquired point cloud data as sample data of training samples corresponding to the traffic light specifically includes:

[0016] Determining a location range corresponding to the traffic light according to the central geographic coordinates of the traffic light and a preset second distance;

[0017] Determining point cloud data whose acquisition positions fall within the position range according to the acquisition positions when the acquisition device acquires each frame of point cloud data;

[0018] Determining laser points that fall within the acquisition range from the laser points included in the determined point cloud data;

[0019] Using point cloud data composed of laser points falling within the acquisition range as sample data of a training sample corresponding to the traffic light;

[0020] The second distance is greater than the first distance.

[0021] Optionally, determining a labeling box for the traffic light in the sample data based on the vector data of the traffic light in the high-precision map specifically includes:

[0022] According to the central geographic coordinates of each other traffic light and the collection range corresponding to the traffic light, determining other traffic lights falling within the collection range as traffic lights to be marked;

[0023] Based on the traffic light to be marked in the high-precision map and the vector data of the traffic light, a marking box in the sample data is determined, and the marking box is used to identify the traffic light in the sample data.

[0024] Optionally, determining the annotation box in the sample data according to the vector data of the traffic light to be annotated in the high-precision map specifically includes:

[0025] For each traffic light to be labeled, determining a ratio between a portion of the three-dimensional bounding box of the traffic light to be labeled that falls within the acquisition range and the entire three-dimensional bounding box of the traffic light to be labeled;

[0026] When the ratio is greater than the labeling threshold, a labeling box of the traffic light to be labeled in the training sample is determined according to the vector data of the traffic light to be labeled.

[0027] Obtain several single-frame point cloud data and determine the point cloud data set;

[0028] Segmenting the point cloud data set according to a preset acquisition range to determine sub-data to be identified;

[0029] Input each sub-data to be identified into a pre-trained traffic light recognition model, and determine the three-dimensional bounding box of each traffic light in each sub-data to be identified;

[0030] The traffic light recognition model is trained using training samples generated by any of the above-mentioned methods for generating training samples.

[0031] This specification provides a device for generating training samples, including:

[0032] A parsing module is used to obtain the constructed high-precision map, parse the vector data of each traffic light in the high-precision map, and determine the central geographic coordinates of each traffic light;

[0033] a collection range determination module, configured to determine, for each traffic light, a collection range corresponding to the traffic light based on the central geographic coordinates of the traffic light and a preset first distance;

[0034] A sample data determination module is used to determine the laser points that fall within the acquisition range from the acquired point cloud data as sample data of the training sample corresponding to the traffic light;

[0035] The training sample generation module is used to determine the annotation box of the traffic light in the sample data based on the vector data of the traffic light in the high-precision map, and generate training samples for training the traffic light recognition model.

[0036] The acquisition module is used to obtain a number of single-frame point cloud data and determine the point cloud data set;

[0037] A determination module, configured to segment the point cloud data set according to a preset acquisition range and determine each sub-data to be identified;

[0038] The recognition module is used to input each sub-data to be recognized into a pre-trained traffic light recognition model, and determine the three-dimensional bounding box of each traffic light in each sub-data to be recognized, wherein the traffic light recognition model is trained using the sample data generated by the above-mentioned method for generating training samples.

[0039] This specification provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for generating training samples or the method for identifying traffic lights is implemented.

[0040] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for generating training samples or the method for identifying traffic lights described above is implemented.

[0041] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0042] The method for generating training samples provided in this specification first parses the vector data of each traffic light in the constructed high-precision map, and determines the central geographic coordinates of each traffic light. Then, for each traffic light, the acquisition range corresponding to the traffic light is determined based on the central geographic coordinates of the traffic light and the preset first distance. Then, from the collected point cloud data, the laser points falling within the acquisition range are determined as the sample data of the training samples corresponding to the traffic light. Finally, based on the vector data of the traffic light in the high-precision map, the annotation box of the traffic light in the sample data is determined to generate a training sample. Based on the vector data of the traffic light in the constructed high-precision map, the point cloud data corresponding to the traffic light, that is, the sample data, is determined. And the annotation box of the traffic light in the sample data is determined to generate a training sample, which saves annotation time, improves annotation efficiency, and reduces labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0044] Figure 1 A flowchart of a method for generating training samples provided in this specification;

[0045] Figure 2 This is a schematic diagram of a three-dimensional bounding box of a traffic light in a high-precision map provided in this specification;

[0046] Figure 3This is a schematic diagram of the collection range corresponding to a traffic light provided in this manual;

[0047] Figure 4 This is a schematic diagram of the location range of a collection device provided in this manual;

[0048] Figure 5 A flow chart of a method for identifying traffic lights provided in this manual;

[0049] Figure 6 A schematic diagram of a device for generating training samples provided in this specification;

[0050] Figure 7 A schematic diagram of a device for identifying traffic lights provided in this manual;

[0051] Figure 8 This is a schematic diagram of an electronic device for implementing a method for generating training samples or a method for identifying traffic lights provided in this specification. DETAILED DESCRIPTION

[0052] To make the purpose, technical solutions, and advantages of this specification more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0053] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0054] Figure 1 The following is a flow chart of a method for generating training samples in this specification, which specifically includes the following steps:

[0055] S100: Obtain a constructed high-precision map, parse the vector data of each traffic light in the high-precision map, and determine the central geographic coordinates of each traffic light.

[0056] Currently, many third-party platforms offer various high-precision maps to support autonomous driving. Because these maps contain vector data for traffic lights, the service platform's servers can generate training samples for deep learning models based on these maps.

[0057] Specifically, in one or more embodiments of the present specification, the server of the business platform can obtain a constructed high-precision map, parse the vector data of each traffic light in the high-precision map, and determine the geographic coordinates of each traffic light.

[0058] Among them, for each traffic light in the high-precision map, its vector data may include data corresponding to the three-dimensional bounding box of the traffic light, data corresponding to the direction of the traffic light, and so on. The data corresponding to the three-dimensional bounding box mentioned here can be the geographic coordinates of the eight vertices of the three-dimensional bounding box, or other data that can at least determine the three-dimensional bounding box. For example, it can also be the geographic coordinates of any two diagonal vertices with the farthest distance from each other among the eight vertices of the three-dimensional bounding box, or it can be the geographic coordinates of the center of the three-dimensional bounding box and the length, width, and height of the three-dimensional bounding box. This specification does not limit the specific content of the vector data.

[0059] Therefore, the server of the business platform can parse the vector data of each traffic light to first determine the data corresponding to the three-dimensional bounding box of each traffic light, so as to further determine the central geographic coordinates of each traffic light.

[0060] For example, suppose that in the constructed high-precision map, for each traffic light, the geographic coordinates of the eight vertices of the three-dimensional bounding box of the traffic light are recorded, such as Figure 2 shown.

[0061] Figure 2 This is a schematic diagram of a three-dimensional bounding box of a traffic light in a high-precision map in this specification. For the convenience of explanation, the three circles in the figure represent the red, yellow and green lights of the traffic light respectively. The figure roughly draws a three-dimensional bounding box for the traffic light shown, and marks the 8 vertices of the three-dimensional bounding box, that is, the coordinates of a total of 8 points A to H in the figure. Assume that the high-precision map stores the three-dimensional coordinates of the 8 vertices for the traffic light. Take A (x1, y1, z1) as an example, where (x1, y1) can be the geographical location of the traffic light based on the coordinate system used by the high-precision map (such as the UTM coordinate system), and z1 can be the height of the traffic light based on the ground plane. Then the server can use Figure 2 The midpoints A and H determine the geographical coordinates of the center of the traffic light, that is,

[0062] The server mentioned in this specification can be a server set up on a business platform, or a device such as a desktop computer or a laptop computer that can execute the solution of this specification. For the sake of convenience, the following description will only take the server as the execution subject.

[0063] S102: For each traffic light, determine a collection range corresponding to the traffic light according to the central geographical coordinates of the traffic light and a preset first distance.

[0064] After obtaining the central geographic coordinates of each traffic light in the above manner, the server can further determine the point cloud data collection range corresponding to the traffic light.

[0065] Specifically, in one or more embodiments of the present specification, the server may determine, for each traffic light, a collection range corresponding to the traffic light according to the central geographic coordinates of the traffic light and a preset range.

[0066] The preset first distance can be determined as needed. For example, traffic lights generally appear at intersections, so it can be determined based on the statistical mean of the intersection width (e.g., 10 meters). Of course, this specification does not limit how to determine the first distance.

[0067] Take the intersection as an example to illustrate. For a certain intersection, there may be three traffic lights (i.e., one lane light and two crosswalk lights) at the entrance of each direction of the intersection. Taking the lane light as an example, assuming that the currently selected traffic light is the lane light, the preset first distance is 10 meters. The server can use the central geographic coordinates of the lane light as the center of the collection range, and based on the horizontal and vertical directions of the coordinate system used by the high-precision map, select a cube range with a side length of 10 meters as the collection range corresponding to the lane light. Figure 3 shown.

[0068] Figure 3 This is a schematic diagram of the collection range corresponding to a traffic light in this manual. Figure 3 In the image, the outermost cube represents the acquisition range for the centrally located traffic light. This cube is centered on the central geographic coordinates of the traffic light's 3D bounding box, with a side length of 10 meters, a preset first distance. The dotted line below the traffic light's 3D bounding box within the cube indicates the center of the cube. Since traffic lights are typically at a certain height, the collected point cloud data often includes laser points surrounding the traffic light.

[0069] S104: Determine, from the collected point cloud data, laser points that fall within the collection range as sample data of training samples corresponding to the traffic light.

[0070] After determining the collection range corresponding to the traffic light as described above, the server may determine the sample data of the training sample corresponding to the traffic light based on the collected point cloud data.

[0071] Specifically, in one or more embodiments of this specification, the server can determine, from the collected point cloud data, laser points that fall within the collection range and use them as sample data for the training samples corresponding to the traffic light. The point cloud data can be collected by equipment such as a vehicle-mounted laser radar, and this specification does not limit how the point cloud data is collected.

[0072] For example, continuing with the example of the intersection and lane signal light mentioned in step S102, the server can obtain point cloud data of multiple single frames collected for the intersection, and then determine the laser points in each frame of point cloud data that fall within the collection range corresponding to the lane signal light from the collected point cloud data, and use the point cloud data composed of all laser points within the collection range as the sample data of the training sample corresponding to the traffic light.

[0073] S106: Determine the annotation box of the traffic light in the sample data based on the vector data of the traffic light in the high-precision map, and generate training samples for training the traffic light recognition model.

[0074] After determining the sample data of the training sample corresponding to the traffic light in the above manner, the server may further determine the labeling of the sample data to generate the training sample.

[0075] Specifically, in one or more embodiments of the present specification, the server may determine the labeling box of the traffic light in the sample data based on the vector data of the traffic light in the high-precision map to generate a training sample.

[0076] The annotation box of the traffic light in the sample data may be a three-dimensional bounding box surrounding the traffic light. Therefore, the server may determine the annotation box of the traffic light in the sample data based on the data corresponding to the three-dimensional bounding box of the traffic light in the vector data of the traffic light in the high-precision map. It should be noted that the method for determining the annotation box can be determined as needed, and this specification does not limit the method for determining the annotation box. Correspondingly, the specific annotation data corresponding to the annotation box can be determined based on the method for determining the annotation box.

[0077] For example, Figure 2 For example, the server can Figure 2 The server can use the three-dimensional bounding box of the traffic light in the high-precision map shown in step S100 to determine the annotation box of the traffic light in the sample data. That is, the server can use the central geographic coordinates of the traffic light and the length, width and height of the three-dimensional bounding box determined in step S100 as the annotation data.

[0078] Of course, there is more than one way to determine the annotation box. For example, since the three-dimensional bounding box can be determined based on point A and point H of the three-dimensional bounding box, the server can also use the geographic coordinates of point A and point H as annotation data.

[0079] After determining the labeled box of the traffic light in the sample data, the server can generate a training sample based on the sample data and the labeled data corresponding to the labeled box for training the traffic light recognition model.

[0080] based on Figure 1 The method for generating training samples shown in the figure first parses the vector data of each traffic light in the constructed high-precision map and determines the central geographic coordinates of each traffic light. Then, for each traffic light, the acquisition range corresponding to the traffic light is determined based on the central geographic coordinates of the traffic light and the preset first distance. Then, from the collected point cloud data, the laser points falling within the acquisition range are determined as the sample data of the training samples corresponding to the traffic light. Finally, based on the vector data of the traffic light in the high-precision map, the annotation box of the traffic light in the sample data is determined to generate a training sample. Based on the vector data of the traffic light in the constructed high-precision map, the point cloud data corresponding to the traffic light, that is, the sample data, is determined. And the annotation box of the traffic light in the sample data is determined to generate a training sample, which saves annotation time, improves annotation efficiency, and reduces labor costs.

[0081] In addition, in one or more embodiments of the present specification, in step S104, in order to improve the efficiency of stitching the point cloud, when the server determines the laser points that fall within the collection range from the collected point cloud data as sample data of the training samples corresponding to the traffic light, the server can also filter the collected point cloud data.

[0082] Specifically, the server may first determine the corresponding location range according to the central geographic coordinates of the traffic light and a preset second distance.

[0083] Then, the server determines the point cloud data whose acquisition position falls within the position range according to the acquisition position when the acquisition device acquires each frame of point cloud data.

[0084] Finally, the server determines the laser points that fall within the acquisition range from the laser points contained in the determined point cloud data, and uses the point cloud data composed of the laser points within the acquisition range as the sample data of the training sample corresponding to the traffic light.

[0085] Among them, the preset second distance can be determined as needed, and the second distance is greater than the first distance preset in step S102. For example, assuming that the acquisition range determined in step S102 is a cube with a side length of 10 meters, and the acquisition radius of the acquisition device is 200 meters, it can be seen that when the acquisition position of the acquisition device is greater than 195 meters from the center geographic coordinates of the traffic light, it is difficult for the acquisition device to collect point cloud data within the acquisition range. At this time, the second distance can be preset to 195 meters. This manual does not limit the size of the second distance. The acquisition device mentioned can be an image acquisition vehicle or a drone, etc. that can carry a laser radar to collect point cloud data. This manual does not limit the acquisition device.

[0086] Continuing with the above example, the location range determined by the server can be a square range with a side length of 390 meters and the center being the central geographic coordinates of the traffic light. Then, based on the acquisition position when the acquisition device acquires each frame of point cloud data, the point cloud data whose acquisition position falls within the location range is determined. Finally, from the determined point cloud data, the point cloud data composed of laser points falling within the acquisition range is determined as the sample data of the training sample corresponding to the traffic light. Figure 4 shown.

[0087] Figure 4 This is a schematic diagram of the location range of a collection device in this manual. Figure 4 The central part of the cube can be referred to Figure 3 The corresponding description is that the quadrilateral outside the cube is a square with a side length of twice the preset first distance, that is, 390 meters, with the projection of the central geographic coordinates of the traffic light in the cube on the bottom surface as the center. Figure 4 The positions of three acquisition devices are given as an example. Among them, the acquisition positions of acquisition devices A and acquisition devices C fall within the position range. Therefore, the two acquisition devices can collect point cloud data within the acquisition range. Therefore, the laser points falling within the acquisition range can be determined based on the laser points contained in the point cloud data collected by the two acquisition devices at the corresponding acquisition positions. However, for acquisition device B, since it is not within the position range, it is difficult for acquisition device B to collect point cloud data within the acquisition range. Therefore, the laser points falling within the acquisition range cannot be determined based on the laser points contained in the point cloud data collected by acquisition device B at the corresponding acquisition position.

[0088] It should be noted that in one or more embodiments of the present specification, in step S100, when the server determines the central geographic coordinates of each traffic light based on the data corresponding to the three-dimensional bounding box of each traffic light, the server may also determine the central geographic coordinates of each traffic light based on a plane coordinate system.

[0089] For example, Figure 2 For example, the server can Figure 2 The x1 and y1 of midpoint A and the x4 and y4 of point D determine the central geographic coordinates of the traffic light based on the plane coordinate system, that is,

[0090] Correspondingly, in step S102, when the server determines the collection range corresponding to each traffic light, for each traffic light, the server can use the central geographic coordinates of the traffic light based on the plane coordinate system as the center of the collection range, and select a square range with a side length of 10 meters as the collection range corresponding to the traffic light according to the horizontal and vertical directions of the coordinate system adopted by the high-precision map.

[0091] In addition, in one or more embodiments of the present specification, considering that the sample data determined in step S104 may include multiple traffic lights, in step S106, when the server determines the labeling box of each traffic light in the sample data based on the vector data of the traffic light in the high-precision map, the server can also determine the labeling box of each traffic light contained in the sample data.

[0092] Specifically, the server may first determine other traffic lights that fall within the collection range according to the central geographic coordinates of the other traffic lights and the collection range corresponding to the traffic light, as the traffic lights to be marked.

[0093] Then, the server determines the data corresponding to the three-dimensional bounding box of each traffic light to be marked based on the vector data of each traffic light to be marked in the high-precision map, and determines the data corresponding to the three-dimensional bounding box of the traffic light based on the vector data of the traffic light in the high-precision map.

[0094] Finally, the server determines a labeling box in the sample data based on the data corresponding to the three-dimensional bounding box of each traffic light within the acquisition range. The labeling box is used to identify the traffic light in the sample data.

[0095] The central geographic coordinates of each other traffic light can be obtained in step S100. The server can then determine whether the central geographic coordinates of each other traffic light fall within the corresponding collection range of the traffic light. If so, the traffic light is designated as the traffic light to be annotated. The subsequent determination of the traffic light's three-dimensional bounding box based on the traffic light's vector data in the high-precision map and the further determination of the annotation box in the sample data can be referred to the corresponding description in step S106 and will not be repeated here.

[0096] Of course, the determination of the annotation box corresponding to the traffic light can be fully referred to the corresponding description in step S106. However, due to the limitation of the collection range, there may be incomplete traffic lights for each traffic light to be annotated. Therefore, further, in one or more embodiments of this specification, when the server determines the annotation box in the sample data based on the vector data of the traffic lights to be annotated in the high-precision map, the server can also determine whether each traffic light to be annotated needs to be annotated as needed.

[0097] Specifically, for each traffic light to be labeled, the server may determine a ratio between a portion of the three-dimensional bounding box of the traffic light to be labeled that falls within the acquisition range and the entire three-dimensional bounding box of the traffic light to be labeled.

[0098] When the ratio is greater than the annotation threshold, the server can determine the data corresponding to the three-dimensional bounding box of the traffic light to be labeled based on the vector data of the traffic light to be labeled in the high-precision map, so as to determine the annotation box of the traffic light to be labeled in the training sample.

[0099] The ratio represents the completeness of the traffic light to be annotated within the collection range. Therefore, the annotation threshold can be determined as needed. For example, if the traffic light is more than half complete, it is determined that the traffic light needs to be annotated. Accordingly, the annotation threshold can be 0.5.

[0100] It should be noted that when determining the labeling box for the traffic light to be labeled, the server can refer to the corresponding description in step S106. Of course, there may be incomplete traffic lights that still need to be labeled. In this case, the server can also determine the labeling box for the traffic light to be labeled based on the data corresponding to the three-dimensional bounding box of the traffic light to be labeled and the collection range. In other words, for an incomplete traffic light to be labeled, only the labeling box of the portion that falls within the collection range can be determined.

[0101] Corresponding to Figure 1 The process of generating training samples shown in FIG. 1 is also provided in this specification. The process of a method for identifying traffic lights is also provided in this specification. Figure 5 shown.

[0102] Figure 5 This is a flow chart of a method for identifying traffic lights in this specification, which specifically includes the following steps:

[0103] S200: Acquire a plurality of single-frame point cloud data and determine a point cloud data set.

[0104] Typically, when building a high-precision map, the server of a business platform can detect or segment the point cloud data based on the collected point cloud data using a pre-trained traffic light recognition model, determine the point clouds in the point cloud data that belong to traffic lights, and thus determine the geographical location of the traffic lights.

[0105] Specifically, in one or more embodiments of the present specification, the server of the business platform may first obtain a number of single-frame point cloud data to determine a point cloud data set.

[0106] The point cloud data may be collected by an acquisition device and stored in a storage device, and the service platform server may obtain the point cloud data from the storage device. Of course, the service platform server may also obtain point cloud data from other data platforms. This manual does not restrict how to obtain point cloud data.

[0107] For the acquired single-frame point cloud data, the server of the business platform can use all the laser points contained in all the single-frame point cloud data as the point cloud data set, and can also preset the recognition range, and then determine the laser points that fall within the preset recognition range from the acquired point cloud data of each single frame as the point cloud data set. This manual does not limit how to determine the point cloud data set. The preset recognition range mentioned here can be determined as needed. For example, when identifying the traffic lights at a certain intersection, the preset recognition range can be determined based on the geographical location of the intersection. This manual does not limit the preset recognition range.

[0108] Continuing with the above-mentioned intersection as an example, the server can first obtain point cloud data of multiple single frames collected for the intersection, and then determine the laser points in each frame of point cloud data that fall within the preset recognition range, and obtain point cloud data composed of all laser points within the recognition range as a point cloud dataset.

[0109] The server mentioned in this specification can be a server set up on a business platform, or a device such as a desktop computer or a laptop computer that can execute the solution of this specification. For the sake of convenience, the following description will only take the server as the execution subject.

[0110] S202: Segment the point cloud data set according to a preset acquisition range to determine sub-data to be identified.

[0111] The point cloud data set determined above can be used as the data to be identified. After obtaining the data to be identified, the server can further segment the data to be identified so that the segmented sub-data to be identified can be input into the traffic light recognition model for identification.

[0112] Specifically, in one or more embodiments of the present specification, the server may segment the point cloud data set according to a preset collection range to determine each sub-data to be identified.

[0113] The specific content of the preset collection range may refer to the corresponding description of the collection range in step S102 in the method for generating training samples.

[0114] The determined point cloud data set is used as the data to be identified. In other words, the server can segment the data to be identified to determine each sub-data to be identified.

[0115] S204: Input each sub-data to be identified into a pre-trained traffic light recognition model, and determine a three-dimensional bounding box of each traffic light in each sub-data to be identified.

[0116] After obtaining the sub-data to be identified as described above, the server may identify the traffic lights in the sub-data to be identified based on a pre-trained traffic light recognition model.

[0117] Specifically, in one or more embodiments of the present specification, the server may input each sub-data to be identified into a pre-trained traffic light recognition model to respectively determine a three-dimensional bounding box of each traffic light in each sub-data to be identified.

[0118] The traffic light recognition model is trained using training samples generated by any of the aforementioned methods for generating training samples.

[0119] When determining the three-dimensional bounding box of each traffic light, for each identified traffic light, the server can first determine the boundary value of the traffic light in each dimension based on the geographic coordinates of the laser point contained in the traffic light, and further determine the three-dimensional bounding box of the traffic light.

[0120] For example, continuing with the above-mentioned intersection as an example, the server can input the determined sub-data to be identified into a pre-trained traffic light recognition model to identify the laser points corresponding to each traffic light in the point cloud data corresponding to the intersection, and for each identified traffic light, determine the boundary value of the traffic light in each dimension, and further determine the three-dimensional bounding box of the laser point corresponding to the traffic light.

[0121] The method for generating training samples provided in this specification can be applied to scenarios involving training traffic light recognition models. The service platform server can generate training samples based on a pre-built high-precision map and point cloud data within the corresponding acquisition range for use in training traffic light recognition models. The method provided in this specification generates training samples based on a pre-built high-precision map and point cloud data within the corresponding acquisition range, saving annotation time, improving annotation efficiency, and reducing labor costs.

[0122] The above is a method provided in one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for generating training samples, such as Figure 6 shown.

[0123] Figure 6 A schematic diagram of a device for generating training samples provided in this specification includes:

[0124] The parsing module 400 is used to obtain the constructed high-precision map, parse the vector data of each traffic light in the high-precision map, and determine the central geographic coordinates of each traffic light;

[0125] A collection range determination module 402 is configured to determine, for each traffic light, a collection range corresponding to the traffic light based on the central geographic coordinates of the traffic light and a preset first distance;

[0126] The sample data determination module 404 is used to determine the laser points falling within the acquisition range from the acquired point cloud data as the sample data of the training sample corresponding to the traffic light;

[0127] The training sample generation module 406 is used to determine the labeling box of the traffic light in the sample data based on the vector data of the traffic light in the high-precision map, and generate training samples for training the traffic light recognition model.

[0128] Optionally, the parsing module 400 determines the data corresponding to the three-dimensional bounding box of each traffic light based on the vector data of each traffic light in the high-precision map, and for each traffic light, determines the central geographic coordinates of the traffic light based on the data corresponding to the three-dimensional bounding box of the traffic light.

[0129] Optionally, the sample data determination module 404 determines the position range corresponding to the traffic light based on the central geographic coordinates of the traffic light and a preset second distance, determines the point cloud data whose collection position falls within the position range based on the collection position when the collection device collects each frame of point cloud data, determines the laser points that fall within the collection range from the laser points contained in the determined point cloud data, and uses the point cloud data composed of the laser points that fall within the collection range as the sample data of the training sample corresponding to the traffic light, wherein the second distance is greater than the first distance.

[0130] Optionally, the training sample generation module 406 determines other traffic lights that fall within the collection range based on the central geographic coordinates of each other traffic light and the collection range corresponding to the traffic light, as traffic lights to be labeled, and determines the labeling box in the sample data based on the traffic lights to be labeled and the vector data of the traffic lights in the high-precision map, and the labeling box is used to identify the traffic lights in the sample data.

[0131] Optionally, the training sample generation module 406 determines, for each traffic light to be labeled, a ratio between the portion of the three-dimensional bounding box of the traffic light to be labeled that falls within the acquisition range and the entire three-dimensional bounding box of the traffic light to be labeled; when the ratio is greater than a labeling threshold, the labeling box of the traffic light to be labeled in the training sample is determined based on the vector data of the traffic light to be labeled.

[0132] This specification also provides a corresponding device for identifying traffic lights, such as Figure 7 shown.

[0133] Figure 7 A schematic diagram of a device for identifying traffic lights provided in this manual includes:

[0134] An acquisition module 500 is used to acquire a plurality of single-frame point cloud data and determine a point cloud data set;

[0135] A determination module 502 is configured to segment the point cloud data set according to a preset acquisition range and determine sub-data to be identified;

[0136] The recognition module 504 is used to input each sub-data to be recognized into a pre-trained traffic light recognition model to determine the three-dimensional bounding box of each traffic light in each sub-data to be recognized, wherein the traffic light recognition model is trained using the sample data generated by the above-mentioned method for generating training samples.

[0137] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 The method for generating training samples provided or Figure 5 A method for identifying traffic lights is provided.

[0138] This manual also provides Figure 8 The structural diagram of the electronic device shown in FIG. Figure 8 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The method for generating training samples provided or Figure 5 A method for identifying traffic lights is provided.

[0139] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as the combination of logic device XOR software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic device.

[0140] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0141] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0142] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0143] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0144] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0148] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0149] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0150] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0151] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0152] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0154] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0155] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for generating training samples, characterized in that: include: Obtaining a constructed high-precision map, parsing the vector data of each traffic light in the high-precision map, and determining the central geographic coordinates of each traffic light; For each traffic light, determine a collection range corresponding to the traffic light according to the central geographic coordinates of the traffic light and a preset first distance; Determining, from the collected point cloud data, laser points that fall within the collection range as sample data of training samples corresponding to the traffic light; Determining, based on the vector data of the traffic light in the high-precision map, a labeling box of the traffic light in the sample data, and generating a training sample for training a traffic light recognition model; From the collected point cloud data, laser points falling within the collection range are determined as sample data of training samples corresponding to the traffic light, specifically including: Determining a location range corresponding to the traffic light according to the central geographic coordinates of the traffic light and a preset second distance; Determining point cloud data whose acquisition positions fall within the position range according to the acquisition positions when the acquisition device acquires each frame of point cloud data; Determining laser points that fall within the acquisition range from the laser points included in the determined point cloud data; Using point cloud data composed of laser points falling within the acquisition range as sample data of a training sample corresponding to the traffic light; The second distance is greater than the first distance.

2. The method according to claim 1, wherein Parsing the vector data of each traffic light in the high-precision map to determine the center geographic coordinates of each traffic light, specifically including: Determining data corresponding to a three-dimensional bounding box of each traffic light based on the vector data of each traffic light in the high-precision map; For each traffic light, the central geographic coordinates of the traffic light are determined based on the data corresponding to the three-dimensional bounding box of the traffic light.

3. The method according to claim 1, wherein Determining a labeling box for the traffic light in the sample data based on the vector data of the traffic light in the high-precision map specifically includes: According to the central geographic coordinates of each other traffic light and the collection range corresponding to the traffic light, determining other traffic lights falling within the collection range as traffic lights to be marked; Based on the traffic light to be marked in the high-precision map and the vector data of the traffic light, a marking box in the sample data is determined, and the marking box is used to identify the traffic light in the sample data.

4. The method according to claim 3, wherein Determining a labeling box in the sample data according to the vector data of the traffic light to be labeled in the high-precision map specifically includes: For each traffic light to be labeled, determining a ratio between a portion of the three-dimensional bounding box of the traffic light to be labeled that falls within the acquisition range and the entire three-dimensional bounding box of the traffic light to be labeled; When the ratio is greater than the labeling threshold, a labeling box of the traffic light to be labeled in the training sample is determined according to the vector data of the traffic light to be labeled.

5. A method for identifying traffic lights, characterized in that: include: Obtain several single-frame point cloud data and determine the point cloud data set; Segmenting the point cloud data set according to a preset acquisition range to determine sub-data to be identified; Input each sub-data to be identified into a pre-trained traffic light recognition model, and determine the three-dimensional bounding box of each traffic light in each sub-data to be identified; The traffic light recognition model is obtained by training using training samples generated by the method described in any one of claims 1 to 4 above.

6. A device for generating training samples, characterized in that: include: A parsing module is used to obtain the constructed high-precision map, parse the vector data of each traffic light in the high-precision map, and determine the central geographic coordinates of each traffic light; a collection range determination module, configured to determine, for each traffic light, a collection range corresponding to the traffic light based on the central geographic coordinates of the traffic light and a preset first distance; The sample data determination module is used to determine the laser points that fall within the acquisition range from the collected point cloud data as the sample data of the training samples corresponding to the traffic light. The sample data determination module is used to determine the laser points that fall within the acquisition range from the collected point cloud data as the sample data of the training samples corresponding to the traffic light. Specifically, the module includes: Determining a location range corresponding to the traffic light according to the central geographic coordinates of the traffic light and a preset second distance; Determining point cloud data whose acquisition positions fall within the position range according to the acquisition positions when the acquisition device acquires each frame of point cloud data; Determining laser points that fall within the acquisition range from the laser points included in the determined point cloud data; Using point cloud data composed of laser points falling within the acquisition range as sample data of a training sample corresponding to the traffic light; wherein the second distance is greater than the first distance; The training sample generation module is used to determine the annotation box of the traffic light in the sample data based on the vector data of the traffic light in the high-precision map, and generate training samples for training the traffic light recognition model.

7. A device for identifying traffic lights, characterized in that: include: The acquisition module is used to obtain a number of single-frame point cloud data and determine the point cloud data set; A determination module, configured to segment the point cloud data set according to a preset acquisition range and determine each sub-data to be identified; An identification module is used to input each sub-data to be identified into a pre-trained traffic light recognition model, and respectively determine the three-dimensional bounding box of each traffic light in each sub-data to be identified, wherein the traffic light recognition model is trained using sample data generated by any method described in any one of claims 1 to 4 above.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 or 5 is implemented.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 4 or 5 is implemented.

Images