Methods for generating geometric models of road signs, vehicle control methods, chips, devices, and storage media

Abstract

This application provides a method for generating geometric models of road signs, a vehicle control method, a chip, a device, and a storage medium. The method for generating geometric models of road signs includes: obtaining a second set corresponding to road signs; fitting first geometric models of road signs based on the second set to obtain multiple first geometric models of road signs; clustering the first geometric models of road signs according to their distribution characteristics to obtain multiple third sets; and for each third set, fusing the first geometric models of road signs contained within the third set to obtain the geometric model generation result of the road signs corresponding to the third set. Through the above technical solution, the reconstruction accuracy of road signs can be improved, thereby enhancing the reliability of vehicle lateral control and path planning during autonomous driving.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method for generating geometric models of road signs, a vehicle control method, a chip, a device, and a storage medium. Background Technology

[0002] In the field of autonomous driving, vehicles need to stably identify and track road signs under complex road conditions over long periods. However, current algorithms for generating geometric models of road signs struggle to obtain accurate and stable road sign reconstruction results, impacting the vehicle's lateral control and path planning. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method for generating a geometric model of road signs, a vehicle control method, a chip, a device, and a storage medium to solve the above problems.

[0004] According to a first aspect of the embodiments of this application, a method for generating a geometric model of a road sign is provided, comprising: obtaining a first set, the first set including information of one or more points, the first set being used to represent at least a portion of the road sign at a given time, the information of the points including three-dimensional spatial coordinates representing the position of the points; obtaining one or more second sets based on multiple first sets at different times, the second sets including information of multiple points; determining a first geometric model of the road sign based on the second sets, the first geometric model corresponding to a first spatial domain; the first geometric model having distribution characteristics, the distribution characteristics including spatial domain distribution characteristics; the spatial domain distribution characteristics including orientation information of the first geometric model and distance information between different first geometric models; obtaining one or more third sets based on the distribution characteristics of the first geometric models in multiple different first spatial domains, the third sets including multiple first geometric models, the multiple first geometric models in the third sets being used to generate a second geometric model of the road sign in a second spatial domain, the second spatial domain including multiple first spatial domains.

[0005] According to a second aspect of the embodiments of this application, a vehicle control method is provided, comprising: acquiring a second geometric model of road signs in the current spatial domain of the vehicle; controlling the vehicle to drive based on the second geometric model of the road signs; wherein the second geometric model of the road signs is generated based on a plurality of first geometric models in a third set, the third set being obtained according to the distribution characteristics of the first geometric models in a plurality of different first spatial domains; the first geometric models are determined based on a second set; the second set is generated based on a plurality of first sets at different times; the first set includes information of one or more points, the first set being used to represent at least a portion of the road signs at a certain time, and the information of the points including three-dimensional spatial coordinates for representing the position of the points.

[0006] According to a third aspect of the embodiments of this application, a chip is provided for performing the methods described in the first and second aspects.

[0007] According to a fourth aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first and second aspects.

[0008] According to a fifth aspect of the embodiments of this application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the methods described in the first and second aspects.

[0009] In the above technical solution, firstly, based on the second set, multiple first geometric models of different first spatial domains are obtained. Then, based on the direction information and distance information of each first geometric model, one or more third sets are obtained. Finally, based on the multiple first geometric models in the third set, a second geometric model of road signs in the second spatial domain is generated.

[0010] The above technical solution can generate a geometric model of road signs based on basic data with consistent spatial distribution characteristics, thereby improving the reconstruction accuracy of road signs and thus enhancing the reliability of vehicle lateral control and path planning in the autonomous driving process. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0012] Figure 1 This is a schematic diagram of an intelligent connected system to which the methods provided in the embodiments of this disclosure can be applied; Figure 2 It is based on Figure 1 The illustrated embodiment provides a schematic diagram of a mobile device; Figure 3 This is a schematic diagram illustrating a geometric model generation scenario for road signs provided in an embodiment of this application. Figure 4 A flowchart illustrating a method for generating a geometric model of road signs provided in an embodiment of this application; Figure 5 Another flowchart for the method of generating geometric models of road signs provided in the embodiments of this application; Figure 6 A flowchart of a vehicle control method provided in an embodiment of this application; Figure 7 A schematic diagram of the geometric model generation device for road markings provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0014] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0015] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0016] It should also be noted that the terms "first, second, and third" used in the embodiments of this application are only used to distinguish similar objects and do not represent a specific order of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0017] Furthermore, in the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0018] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and they all fall within the protection scope of the embodiments of this application.

[0019] The method for generating geometric models of road signs provided in this application can be applied to mobile devices such as autonomous vehicles, robots, and drones. Taking an autonomous vehicle as an example, the vehicle is equipped with a camera, a processor, and a memory. The camera is used to acquire image information of the spatial domain in which the vehicle is located, the processor is used to execute the method of this application, and the memory is used to store image sequences and computer instructions. Those skilled in the art should understand that this method is also applicable to other mobile devices.

[0020] Figure 1 An intelligent connected system 100 that can apply the methods provided in the embodiments of this application is illustrated. Figure 1 As shown, the intelligent connected system 100 may include: a mobile device 101, a server 102, and a user terminal 103.

[0021] In some examples, the mobile device 101 can be a vehicle with autonomous driving capabilities, a robot capable of autonomous movement, etc. Autonomous driving, also known as driverless or intelligent driving, refers to a vehicle with autonomous driving capabilities that can perform driving tasks such as environmental perception, decision-making, planning, and control execution. The levels of autonomous driving can refer to the vehicle intelligence classification standards established by the Society of Automotive Engineers (SAE), for example, L0 is manual driving, L1 is driver assistance, L2 is partial autonomous driving, L3 is conditional autonomous driving, L4 is highly automated driving, and L5 is fully automated driving. The above classification of autonomous driving levels is merely an example, and this disclosure does not limit the classification standards and levels of autonomous driving.

[0022] In some examples, server 102 can be a single server or a distributed server cluster consisting of multiple servers, and its deployment can include local servers and / or cloud servers. Server 102 can communicate with mobile device 101 and / or user terminal 103 via a communication network, providing various services to mobile device 101 and / or user terminal 103. For example, the server can receive sensing data sent by mobile device 101, and provide services such as high-precision maps, data analysis, and decision planning to mobile device 101. Alternatively, the server can receive query commands or control commands sent by user terminal 102, providing corresponding services to the user.

[0023] In some examples, the user terminal 103 can be any form of electronic device that provides services to the user, such as a personal computer, laptop, smart tablet, smartphone, smart wearable device, etc. The user can interact with the mobile device or server through the human-computer interaction terminal configured on the mobile device 101, or through the user terminal 103. For example, the user terminal can query the status and / or parameters of the mobile device, or control the mobile device to perform set tasks and / or modify configuration parameters, etc. The user terminal 103 runs an application based on the intelligent network system to achieve interaction with the mobile device or server. This application can be a local application, a web application, or a mini-program, etc., without limitation.

[0024] In some examples, the aforementioned application running on user terminal 103 can provide authentication or authorization services to users. Users who are successfully authenticated and granted the corresponding permissions can query and / or control the mobile device within the scope of the granted permissions.

[0025] The mobile device 101, server 102, and user terminal 103 can communicate via a communication link provided by communication network 104. This communication network 104 can include one or more networks of any type, such as the Internet, Local Area Network (LAN), Wide Area Network (WAN), Virtual Private Network (VPN), Public Switched Telephone Network (PSTN), satellite communication network, Wi-Fi, 2G, 3G, 4G, 5G, 6G, NB-IoT, eMTC, infrared, Bluetooth, NFC, or a combination of these networks. The communication networks between the mobile device 101 and server 102, between the user terminal 103 and server 102, and between the user terminal 103 and mobile device 101 can be the same or different.

[0026] It should be noted that, Figure 1The structure of the intelligent connected system 100 shown is merely illustrative. The intelligent connected system in this embodiment is not limited to the above structure and may include more or fewer devices as needed, and the devices may be combined or separated. For example, the intelligent connected system may not include... Figure 1 The user terminal in the middle.

[0027] The methods provided in this disclosure can be implemented by the mobile device 101, the server 102, or both. Furthermore, those skilled in the art should understand that the methods and optimization methods provided in this disclosure can also be implemented by other devices with data processing capabilities, independent of the intelligent network system; simply importing the scene data collected by the mobile device into that device is sufficient, and no limitation is made herein.

[0028] Figure 2 It is based on Figure 1 The illustrated embodiment provides a schematic diagram of a mobile device 101. As shown... Figure 2 As shown, the mobile device 101 may include a sensing component 1011, a computing platform 1012, an execution component 1013, etc. The sensing component 1011, the computing platform 1012, and the execution component 1013 may be connected via a bus or other means.

[0029] In some examples, the sensing component 1011 can be used to collect information about the mobile device itself or its external environment. The sensing component 1011 may include a visual sensing unit and a motion sensing unit. The visual sensing unit may include one or more cameras. The motion sensing unit may include a wheel speedometer and / or an inertial measurement unit (IMU). In other examples, the sensing component 1011 may also include radar, a positioning and navigation unit, etc., without limitation. The radar may include at least one of lidar, millimeter-wave radar, ultrasonic radar, or other radar types. The wheel speedometer can be of any type, such as a magnetoelectric wheel speedometer, an photoelectric wheel speedometer, a mechanical wheel speedometer, a Hall effect wheel speedometer, or a visual wheel speedometer. The positioning and navigation unit may include at least one of a GPS system, a BeiDou system, or other global positioning systems.

[0030] In some examples, the computing platform 1012 may include a computing-capable device for processing the sensing information collected by the sensing component 1011 to obtain control information, and sending corresponding control commands to the execution component 1013 to cause the execution component 1013 to perform corresponding actions, thereby realizing the control of the mobile device 101. For example, the computing platform 1012 can perform Simultaneous Localization and Mapping (SLAM), path planning, and behavior decision-making on the mobile device, thereby realizing autonomous control of the mobile device. The computing platform 1012 may include at least one processor and at least one memory, and each processor may execute instructions stored in the memory individually or jointly to implement the methods provided in the embodiments of this disclosure. The processor in the embodiments of this disclosure may include at least one of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), a Microcontroller Unit (MCU), or other processors. Memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. In addition to storing instructions, memory can also store data, such as high-definition maps, path information, and data on the location, direction, and speed of mobile devices. The data stored in memory can be accessed and used by the processor.

[0031] In some examples, the computing platform of a mobile device can perform computing tasks independently or communicate with a server to complete computing tasks. For example, the computing platform of a mobile device can cooperate with a server to complete the corresponding computing tasks.

[0032] The computing platform 1012 can be located in the mobile device 101. Some or all of the computing platform 1012 can also be located in the server corresponding to the mobile device. For example, some functions of the computing platform 1012 with high real-time requirements can be located in the mobile device, while other functions with low real-time requirements can be located in the server corresponding to the mobile device.

[0033] In some examples, the execution component 1013 is used to perform corresponding actions based on the control of the computing platform 1012, enabling the mobile device 101 to complete the movement task. The execution component 1013 may include, for example, a power component, a braking component, a transmission component, a steering component, etc.

[0034] It should be noted that, Figure 2 The structure of the mobile device 101 shown is merely illustrative. The mobile device in this embodiment is not limited to the above structure and may include more or fewer components as needed. The device may also be combined or disassembled. For example, the mobile device may not include the aforementioned computing platform. Furthermore, the mobile device may also include communication components, interface components, multimedia components, input components, output components, etc.

[0035] In one exemplary implementation scenario, the mobile device 101 provided in this application embodiment can be a vehicle, and the perception component 1011 of the mobile device can include a camera. During the driving of the mobile device 101, the camera can be used to collect image data of the spatial domain in which it is located, and based on the image data, point cloud data at the corresponding time can be obtained to implement the method provided in this application embodiment. Exemplarily, the camera can send the collected image data to the computing platform 1012, which can be used to obtain the point cloud data at the corresponding time based on the image data, and further execute the method provided in this application embodiment to obtain the geometric model corresponding to the road sign. Further, the computing platform 1012 can send the generated geometric model to the execution component 1013. The geometric model corresponding to the road sign can be used by the execution component 1013 to control the driving of the mobile device 101 in the autonomous driving scenario of the mobile device 101.

[0036] The specific implementation of the geometric model generation method for road signs provided in the embodiments of this application will be described below.

[0037] In autonomous driving scenarios, the accuracy of the geometric model generation results for road markings directly affects the vehicle's lateral control and path planning. Road markings may include, for example, lane lines (such as solid lines, dashed lines, double lines, etc.), directional and directional arrows (such as straight arrows, etc.), curbs or edge lines, speed bumps, etc.

[0038] In some technical solutions, after obtaining the point cloud data of road signs, the point cloud can be clustered based on factors such as point cloud density and the proximity between points in the point cloud. This clusters high-density point cloud regions or points that are geographically close together, resulting in multiple point cloud clusters. Then, road signs can be fitted based on each point cloud cluster to obtain the geometric model of the road signs.

[0039] However, the above-mentioned technical solution, which clusters point clouds as targets, loses the directional information of road signs. This may result in point clouds of road signs with different directional features being clustered into one category. In the subsequent generation of geometric models of road signs, point clouds corresponding to road signs with different directions may be incorrectly used to generate the same road sign, which is inconsistent with the actual situation and results in low accuracy of geometric model generation of road signs.

[0040] To facilitate understanding, the implementation of the above technical solution will be explained below with reference to the relevant accompanying drawings. Figure 3 A schematic diagram of one implementation scenario of the above technical solution is given, such as... Figure 3 As shown, assume there exists such Figure 3 The cloud data shown is shaped like a "Y". Based on the point cloud aggregation strategy described above, the points at the intersections have a higher density and are geographically close. Therefore, these points may be clustered together. Consequently, in the subsequent geometric model generation step for road signs, the points at the intersections will be used to generate the geometric model for the same road sign. Clearly, the generated geometric model of the road sign will not match reality.

[0041] Based on the above problems, embodiments of this application are proposed.

[0042] This application provides a method for generating geometric models of road signs. This method first generates a first geometric model of the road sign based on point cloud data, the first geometric model corresponding to a first spatial domain. Then, one or more third sets can be obtained based on the distribution characteristics of each first geometric model. Furthermore, for each third set, a second geometric model of the road sign in a second spatial domain can be obtained based on each first geometric model of the road sign contained within the third set. The second spatial domain includes multiple first spatial domains.

[0043] The road sign geometric model generation method provided in this application uses the first geometric model of the road signs as the target for clustering. Based on the characteristic that the first geometric model of the road signs contains directional information, it can prevent road sign data with inconsistent directions from being clustered into one class, thereby improving the reconstruction accuracy of subsequent road signs. Still using... Figure 3 For example, based on the method provided in this application embodiment, since the orientations of the first geometric models of road signs corresponding to the point clouds at intersections are inconsistent, the point clouds at intersections will be used to generate geometric models of road signs in different second spatial domains. It can be seen that the method provided in this application embodiment generates geometric models of road signs with high accuracy.

[0044] The implementation process of the geometric model generation method for road signs provided in this application embodiment will be further explained below with reference to the relevant accompanying drawings.

[0045] Figure 4 This is a flowchart illustrating a method for generating a geometric model of road signs according to an embodiment of this application. Figure 4 As shown, the method for generating the geometric model of road signs provided in this application embodiment may include: S201, Get the first set.

[0046] In this embodiment of the application, the first set may include information of one or more points. The first set is used to represent at least a portion of the road signage at a given moment. The information of the points may include three-dimensional spatial coordinates representing the location of the points.

[0047] S202, based on the first set at multiple different times, obtain one or more second sets.

[0048] In this embodiment, the second set may include information about multiple points. The second set can be obtained, for example, by superimposing a first set at different times. The information about the points in the second set may include three-dimensional spatial coordinates representing the location of the points. Furthermore, the information about the points in the second set may also include a collection timestamp, confidence level, etc. The confidence level is related to the scene characteristics and device status at the time of point collection. Scene characteristics may include, for example, the occlusion of road signs, ambient light levels, and weather conditions. Device status may include, for example, the accuracy of the data collected by relevant sensors (such as the imaging effect of the original image used to generate the point).

[0049] S203, based on the second set, determines the first geometric model of the road signage.

[0050] The first geometric model corresponds to a first spatial domain, which may include, for example, local areas of road signs. In other words, the first geometric model of road signs in different local areas can be determined based on a second set. The first geometric model has distribution characteristics, which may include spatial domain distribution characteristics, including directional information of the first geometric model and distance information between different first geometric models.

[0051] Specifically, based on the distances between points in the second set and their relative positions, local point clouds that are close in distance and have the same orientation can be used to generate a first geometric model of road signs in the first spatial domain. The specific algorithm used can be any local fitting algorithm, such as local least squares or sliding window fitting.

[0052] For any generated first geometric model, it can be represented, for example, by a set of sampling points, where a sampling point refers to any point in a second set. For example, road sign first geometric model A: [point 1, point 2, point 3], road sign first geometric model B: [point 3, point 7, point 8, point 9]. Furthermore, each first geometric model can also correspond to descriptive information, which may include a direction vector, confidence level, and collection timestamp. The direction vector of the first geometric model can be determined based on the coordinates of the start and end points. The confidence level can be determined based on the confidence levels of each point included in the first geometric model; for example, it can be determined based on the statistical distribution characteristics (such as the mean) of the confidence levels of each point. The collection timestamp can be determined based on the collection timestamps of each point included in the first geometric model; for example, it can be determined based on the statistical distribution characteristics (such as the mean) of the collection timestamps of each point.

[0053] S204, based on the distribution characteristics of multiple first geometric models in different first spatial domains, obtain one or more third sets, and use multiple first geometric models in the third set to generate second geometric models of road signs in the second spatial domain.

[0054] The third set may include multiple first geometric models, and the multiple first geometric models in the third set can be used to generate a second geometric model of road markings in a second spatial domain, the second spatial domain including multiple first spatial domains.

[0055] In this embodiment, the distribution characteristics of the first geometric model may include spatial domain distribution characteristics and temporal domain distribution characteristics. The spatial domain distribution characteristics may include the orientation information of each first geometric model and the distance information between different first geometric models. The distance information between different first geometric models may be determined, for example, based on the coordinate information of points contained in each first geometric model. The temporal domain distribution characteristics may include the acquisition timestamp corresponding to each first geometric model and the acquisition time difference between different first geometric models.

[0056] In one possible implementation, firstly, the adjacency relationship of each first geometric model can be determined based on the aforementioned spatial domain distribution characteristics and temporal domain distribution characteristics of each first geometric model.

[0057] Adjacency relationships can be used to characterize the proximity of different first geometric models in terms of distance, orientation, and acquisition time. Specifically, firstly, the differences in distance, orientation, and acquisition time between each first geometric model can be calculated separately. Then, the differences in distance, orientation, and acquisition time between any two first geometric models can be normalized to obtain a normalized difference between any two first geometric models. This normalized difference can be used to characterize the comprehensive differences in distance, orientation, and acquisition time between any two first geometric models. Finally, adjacency relationships between each first geometric model can be generated based on the normalized differences between them. The implementation form of adjacency relationships can be, for example, a matrix or a table, etc., and this embodiment of the application is not limited to this.

[0058] Then, based on the adjacency relationships of each first geometric model, the first geometric models can be grouped to obtain multiple third sets. Specifically, based on the adjacency relationships of each first geometric model, first geometric models with normalized differences less than a set threshold can be grouped into the same third set, thus obtaining multiple third sets. In this way, for any third set, the first geometric models contained therein will have proximity in direction, distance, and acquisition time.

[0059] In another possible implementation, a graph-matching-based topological clustering algorithm can be used to group the various first geometric models, resulting in multiple third sets. Here, a graph refers to a data structure composed of nodes and edges. For example, in the scenario of this application embodiment, each node in the graph can be determined based on each first geometric model, and the direction and length of the edges between each node can be determined based on the spatial and temporal distribution characteristics of each first geometric model. Then, based on the topology of the obtained graph, the nodes (i.e., the first geometric models) in the graph can be grouped, and nodes with edge lengths less than a set edge length threshold and edge direction differences less than a set direction difference threshold can be assigned to the same third set, resulting in multiple third sets.

[0060] In this embodiment, when grouping the first geometric models, both the temporal and spatial characteristics of the first geometric models are used as references. This allows first geometric models that simultaneously satisfy temporal proximity, strong directional consistency, and close distance to be grouped into a third set. This improves the generation accuracy of subsequent road markings in the second spatial domain, helping to maintain the smoothness and geometric consistency of the generated geometric models in complex scenarios such as vehicle turning and lane changing, and enhancing the reliability of vehicle lateral control.

[0061] The following describes how to generate a second geometric model for road markings in the second spatial domain based on multiple first geometric models in the third set.

[0062] In this embodiment, for any third set, a second geometric model corresponding to the road sign in the second spatial domain can be generated based on each first geometric model contained in the third set. The second spatial domain includes multiple first spatial domains. In other words, for any third set, the geometric models corresponding to each local region contained in the third set can be merged to obtain a geometric model corresponding to the road sign in a larger region.

[0063] Specifically, for any third set, the weight information of each first geometric model included in the third set can be determined based on the acquisition timestamp and confidence level of each model. For example, the more recent the acquisition timestamp, the greater the corresponding weight; the greater the confidence level, the greater the corresponding weight. Then, based on the weight information, the first geometric models included in the third set can be weighted to obtain the second geometric model of the road sign in the second spatial domain.

[0064] By implementing the above method, the reliance on the recently collected first geometric model and the high-confidence first geometric model can be increased during the generation of the second geometric model, which is beneficial to improving the reconstruction accuracy of the road sign geometric model.

[0065] One possible implementation involves using a sliding window fusion method based on kernel density estimation (KDE) to fuse the various first geometric models within the third set. Specifically, for any window position, a weighted average is performed on the various first geometric models within the window based on their weight information and density distribution, yielding a fusion curve within the window. Then, by moving the window position and repeating the above steps, the complete fusion curves corresponding to each first geometric model within the third set can be obtained. Finally, the complete fusion curves can be parameterized to obtain the second geometric model.

[0066] Furthermore, based on the above implementation, for any window, if the first geometric model is missing at any query position within the window, the KDE sliding window fusion result will not output "empty." Instead, it can generate a corresponding fusion curve based on the density trend of adjacent query positions (i.e., before and after the missing area). In this way, in cases where road sign data is partially missing due to real-world factors such as broken or partially obscured road signs, the above implementation can automatically complete the missing road sign data, thus maintaining the continuity of the generated geometric model results.

[0067] In other implementations, for example, the first geometric models of each road sign within the third set can be fused based on other methods, such as global fitting of basis function (B) splines. This application does not impose any limitations on the embodiments described.

[0068] In the above technical solution, a first geometric model of road signs is first generated in a first spatial domain. Then, the first geometric model is clustered based on distribution characteristics to obtain multiple third sets. Finally, a second geometric model of road signs in a second spatial domain is generated based on the multiple first geometric models in the third sets.

[0069] Based on the above technical solution, a geometric model of road signs can be generated based on basic data with consistent spatial distribution characteristics, thereby improving the reconstruction accuracy of road signs and thus enhancing the reliability of vehicle lateral control and path planning in the autonomous driving process.

[0070] Figure 5 Another flowchart illustrates the method for generating geometric models of road signs provided in this application. (See attached diagram.) Figure 5 As shown, step S202 above may specifically include: S2021, based on the reference coordinate system, performs coordinate transformation on the first set at different times to obtain the first set after coordinate transformation.

[0071] In this embodiment, the first set can be acquired visually. Specifically, cameras can be configured to collect image data, and these cameras may include multiple cameras positioned in the front, rear, left, and right directions of the vehicle body. Furthermore, the collected image data at various times can be input into a target neural network model, which may be, for example, a trained Bird's Eye View Space (BEV) Transformer model. The BEVTransformer model can be used to output the first set corresponding to each time moment based on the image data at each time moment.

[0072] Furthermore, a configurable caching unit is provided for receiving and caching the first set of outputs from the target neural network model.

[0073] S2022, in response to the number of the first set after coordinate transformation exceeding a set threshold, multiple first sets after coordinate transformation are superimposed to obtain a superimposed set.

[0074] In this embodiment, the first set output by the target neural network model can be point cloud data in the vehicle coordinate system. Since the vehicle is in motion during data acquisition, the vehicle coordinate systems corresponding to the first set at different times differ. Therefore, in this embodiment, before superimposing the first sets at different times, the first sets at each time can be transformed to the same reference coordinate system based on the vehicle's pose information at different times. The reference coordinate system can be, for example, the world coordinate system, or it can be the vehicle coordinate system corresponding to the first set at a specific time. Since the coordinates of the same road sign observed at different times remain fixed in the same reference coordinate system, superimposing the first sets at multiple times after coordinate transformation can improve the point cloud density of each road sign segment, preventing sparse point clouds in a single frame due to light, moving object occlusion, noise interference, etc., which would affect the geometric model generation result of the road sign.

[0075] In one possible implementation, the threshold value corresponding to the number of elements in the first set can be a pre-set fixed value. Then, in subsequent implementation steps, a fixed number of coordinate-transformed elements from the first set can be superimposed each time to obtain a superimposed point cloud, which is used to generate the geometric model for subsequent road markings. This implementation method is simple and convenient, and helps improve the real-time performance and efficiency of the computation.

[0076] In another possible implementation, the threshold value corresponding to the number of elements in the first set can be dynamically changed based on changes in the scene. Then, in subsequent implementation steps, a superimposed point cloud can be obtained based on the dynamically changing number of elements in the first set, which is used to generate the geometric model of the road signs. This implementation method improves the scene adaptability of the solution, ensuring that high-precision geometric model generation results for road signs can be output in different scenarios, thus enhancing the reliability of the generated geometric model results.

[0077] In this embodiment, the calculation of the aforementioned set threshold can be performed according to a specific cycle. Specifically, for any execution cycle, the motion parameters of the current vehicle and scene feature parameters can be obtained. The vehicle motion parameters may include, for example, the vehicle's speed, and the scene feature parameters may include, for example, intersection type (such as T-junction, roundabout, etc.), ground marking density, road sign occlusion ratio, etc. Then, the aforementioned set threshold can be determined based on the vehicle's motion parameters and scene feature parameters. For example, the faster the vehicle's speed and the higher the scene complexity, the larger the set threshold value.

[0078] S2023, preprocess the superimposed set to obtain one or more second sets.

[0079] Preprocessing can include filtering, targeting noisy points, points of abrupt changes in direction, and points with low confidence within the overlay set. Specifically, on one hand, isolated points within the overlay set can be filtered based on its density. Isolated points refer to points within a preset range that have no other points. Understandably, isolated points in the overlay set exist only in a small number of points in the first set; therefore, they are typically noise points. On the other hand, points of abrupt changes in direction within the overlay set can be filtered based on its spatial distribution characteristics. Points of abrupt changes in direction may be points of moving objects (such as pedestrians or vehicles) that were not filtered by the algorithm during the first set generation stage. Furthermore, points with confidence levels below a certain threshold within the overlay set can be filtered based on their confidence levels.

[0080] The above preprocessing operations can significantly reduce the interference of noise points on the generated road sign geometric model and improve detection stability.

[0081] In the above technical solution, on the one hand, the first set of road signs is obtained based on vision, which eliminates the need for millimeter-wave radar or lidar, thus reducing hardware costs and system complexity; on the other hand, the first set overlay and filtering mechanism based on multiple time points can reduce the interference of noise points on the road sign results and improve detection stability.

[0082] This application also provides a vehicle control method. Figure 6 This is a flowchart illustrating a vehicle control method provided in an embodiment of this application. Figure 6 As shown, the vehicle control method provided in this application embodiment may include: S301, Obtain the second geometric model of the road signs in the current spatial domain of the vehicle.

[0083] S302, a second geometric model based on road markings, controls vehicle movement.

[0084] In this embodiment of the application, the second geometric model of the road sign can be obtained based on the geometric model generation method of the road sign provided in this embodiment of the application.

[0085] This application also provides a geometric model generation device for road signs, such as... Figure 7 As shown, the device may include a first acquisition module 401, a second acquisition module 402, a first determination module 403, and a second determination module 404.

[0086] The system comprises the following modules: a first acquisition module 401, which acquires a first set including information of one or more points, representing at least a portion of a road sign at a given time, wherein the point information includes three-dimensional spatial coordinates representing the point's location; a second acquisition module 402, which acquires one or more second sets based on the first sets at multiple different times, the second sets including information of multiple points; a first determination module 403, which determines a first geometric model of the road sign based on the second sets, the first geometric model corresponding to a first spatial domain; the first geometric model has distribution characteristics, including spatial domain distribution characteristics; the spatial domain distribution characteristics include orientation information of the first geometric model and distance information between different first geometric models; and a second determination module 404, which obtains one or more third sets based on the distribution characteristics of the first geometric models in multiple different first spatial domains, the third sets including multiple first geometric models, the multiple first geometric models in the third sets being used to generate a second geometric model of the road sign in a second spatial domain, the second spatial domain including multiple first spatial domains.

[0087] In one specific implementation, the point information also includes the acquisition timestamp; the distribution characteristics of the first geometric model also include the temporal distribution characteristics; the temporal distribution characteristics include the acquisition timestamp corresponding to the first geometric model, and the acquisition time difference between different first geometric models; the acquisition timestamp corresponding to the first geometric model is determined based on the acquisition timestamps of the points contained in the first geometric model.

[0088] In one specific implementation, the second determining module 404 is specifically used to determine the adjacency relationship of the first geometric models based on the spatial domain distribution characteristics and temporal domain distribution characteristics of the first geometric models in multiple different first spatial domains. The adjacency relationship is used to characterize the degree of proximity between different first geometric models in terms of distance, direction and acquisition time. Based on the adjacency relationship of the first geometric models, the first geometric models are grouped to obtain one or more third sets.

[0089] In one specific implementation, the point information also includes the collection timestamp and confidence level; the second determining module 404 is further configured to determine the weight information of the multiple first geometric models in the third set based on the collection timestamps and confidence levels of the multiple first geometric models in the third set; based on the weight information, the multiple first geometric models in the third set are weighted to obtain the second geometric model of the road sign in the second spatial domain; wherein, the collection timestamp corresponding to the first geometric model is determined based on the collection timestamp of each point contained in the first geometric model, and the confidence level corresponding to the first geometric model is determined based on the confidence level of each point contained in the first geometric model.

[0090] In one specific implementation, the second acquisition module 402 is specifically used to: perform coordinate transformation on the first set at different times based on the reference coordinate system to obtain the first set after coordinate transformation; in response to the number of the first sets after coordinate transformation exceeding a set threshold, superimpose multiple first sets after coordinate transformation to obtain a superimposed set; preprocess the superimposed set to obtain one or more second sets; the preprocessing includes filtering.

[0091] In one specific implementation, the value of the threshold is related to the vehicle's motion parameters and scene feature parameters.

[0092] The specific implementation process of the geometric model generation method for road signs executed by the above-mentioned device has been described in detail in the foregoing embodiments, and will not be repeated here.

[0093] This application also provides a chip that can be used to execute the road sign geometric model generation method and vehicle control method provided in this application.

[0094] This application also provides an electronic device. Figure 8 A schematic diagram of an electronic device according to an embodiment of this application is shown. This electronic device can be used to execute the road sign geometric model generation method and vehicle control method provided in the embodiments of this application. Specific embodiments of this application do not limit the specific implementation of the electronic device; for example, the electronic device may be a vehicle, a controller within a vehicle, or a cloud server for a vehicle, etc.

[0095] like Figure 8 As shown, the electronic device may include: a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0096] The processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508. Communication interface 504 is used to communicate with other electronic devices or servers. The processor 502 executes program 510, specifically performing the relevant steps in the above-described road sign geometric model generation method and vehicle control method embodiments.

[0097] Specifically, program 510 may include program code that includes computer operation instructions.

[0098] Processor 502 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0099] Memory 506 is used to store program 510. Memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0100] Specifically, program 510 can be used to cause processor 502 to perform the following operations: In an optional implementation, program 510 is further used to cause processor 502 to perform the following operations. The specific implementation of each step in program 510 can be found in the corresponding steps and system descriptions in the above-described road sign geometric model generation method and vehicle control method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0101] This application also provides a computer program product, including computer instructions, which instruct a computing device to perform operations corresponding to any of the road sign geometric model generation methods and vehicle control methods in the above-described plurality of method embodiments. It should be noted that, depending on implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0102] This application also provides a computer-readable storage medium in which the methods described in this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as ASIC or FPGA) for such software processing. It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the geometric model generation method for road signs and the vehicle control method described herein are implemented. Furthermore, when a general-purpose computer accesses code used to implement the geometric model generation method for road signs and the vehicle control method shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the geometric model generation method for road signs and the vehicle control method shown herein.

[0103] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0104] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0105] Furthermore, it should be noted that all actions related to the collection, storage, use, processing, transmission, provision, disclosure, and deletion of data involved in this disclosure are carried out in accordance with the relevant data protection laws and regulations of the country or region where the data is located, and with the full authorization of the relevant data owner.

[0106] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0107] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0108] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0109] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0110] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A method for generating a geometric model of road signs, characterized in that, include: Obtain a first set, the first set including information of one or more points, the first set being used to represent at least a portion of the road signage at a given time, the information of the points including three-dimensional spatial coordinates representing the location of the points; Based on the first set at multiple different times, obtain one or more second sets, the second set including information of multiple points; Based on the second set, a first geometric model of the road sign is determined, and the first geometric model corresponds to a first spatial domain; The first geometric model has distribution characteristics, which include spatial domain distribution characteristics; the spatial domain distribution characteristics include orientation information of the first geometric model and distance information between different first geometric models; Based on the distribution characteristics of the first geometric models in multiple different first spatial domains, one or more third sets are obtained, the third sets including multiple first geometric models, the multiple first geometric models in the third sets being used to generate second geometric models of road markings in a second spatial domain, the second spatial domain including multiple first spatial domains.

2. The method according to claim 1, characterized in that, The information at the point also includes the collection timestamp; The distribution characteristics of the first geometric model also include temporal distribution characteristics; the temporal distribution characteristics include the acquisition timestamp corresponding to the first geometric model, and the acquisition time difference between different first geometric models; The acquisition timestamp corresponding to the first geometric model is determined based on the acquisition timestamps of the points contained in the first geometric model.

3. The method according to claim 2, characterized in that, Based on the distribution characteristics of the first geometric models in multiple different first spatial domains, one or more third sets are obtained, including: Based on the spatial domain distribution characteristics and temporal domain distribution characteristics of the first geometric models in multiple different first spatial domains, the adjacency relationship of the first geometric models is determined. The adjacency relationship is used to characterize the degree of proximity between different first geometric models in terms of distance, direction and acquisition time. Based on the adjacency relationship of the first geometric model, the first geometric model is grouped to obtain one or more third sets.

4. The method according to claim 1, characterized in that, The information at the point also includes the collection timestamp and confidence level; Based on the plurality of first geometric models in the third set, a second geometric model for road markings in the second spatial domain is generated, including: Based on the acquisition timestamps and confidence levels of the plurality of first geometric models in the third set, the weight information of the plurality of first geometric models in the third set is determined; Based on the weight information, the multiple first geometric models in the third set are weighted to obtain the second geometric model of road markings in the second spatial domain; Wherein, the acquisition timestamp corresponding to the first geometric model is determined based on the acquisition timestamps of each point contained in the first geometric model, and the confidence level corresponding to the first geometric model is determined based on the confidence level of each point contained in the first geometric model.

5. The method according to claim 1, characterized in that, Based on the first set at multiple different times, obtain one or more second sets, including: Based on the reference coordinate system, coordinate transformation is performed on the first set at different times to obtain the first set after coordinate transformation; If the number of first sets after coordinate transformation exceeds a set threshold, multiple first sets after coordinate transformation are superimposed to obtain a superimposed set; The superimposed set is preprocessed to obtain one or more second sets; the preprocessing includes filtering.

6. The method according to claim 5, characterized in that, The value of the set threshold is related to the vehicle's motion parameters and scene feature parameters.

7. A vehicle control method, characterized in that, include: Obtain the second geometric model of road signs in the current spatial domain of the vehicle; The vehicle's movement is controlled based on a second geometric model of the road markings. The second geometric model of the road sign is generated based on multiple first geometric models in a third set, which is obtained according to the distribution characteristics of the first geometric models in multiple different first spatial domains. The first geometric model is determined based on a second set; the second set is generated based on a first set at multiple different times; the first set includes information on one or more points, which is used to represent at least a portion of the road signage at a given time, and the information on the points includes three-dimensional spatial coordinates representing the location of the points.

8. A chip, characterized in that, include: A processor for performing the method as described in any one of claims 1-7.

9. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the method as described in any one of claims 1 to 7.

Images