A road modeling method, apparatus, and device

Abstract

The application discloses a road modeling method, device and equipment, and the road modeling method cyclically performs the following steps: acquiring a road environment image obtained by a self-vehicle; selecting a plurality of first target vehicles around the self-vehicle in a current image frame as candidate vehicle queues in a candidate region according to a first road curvature, a current speed of the self-vehicle and structure information of the road; determining a second road curvature of the road in the candidate region according to driving information of all the first target vehicles in the candidate vehicle queues; constructing a virtual driving area of the current self-vehicle in the current image frame according to the second road curvature and the current speed of the self-vehicle; and updating the first road curvature by using the second road curvature. The application determines the virtual driving area of the self-vehicle by using a pure visual post-processing technology, so that road modeling failure caused by missing of lane lines due to limitations of a self-vehicle perception system is avoided.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and more particularly to a road modeling method, apparatus, and device. Background Technology

[0002] Autonomous driving systems have developed rapidly in recent years. The following methods are currently used for road modeling of autonomous driving.

[0003] 1. Lane lines in images are identified using traditional machine vision or deep learning techniques, then post-processed to the feature level, fitted, and the viewpoint is transformed to the vehicle coordinate system for use by autonomous driving systems.

[0004] This solution has high requirements for the clarity of lane lines and road markings. In low light or extreme weather conditions, or in scenarios where the ground markings are unclear, lane lines are prone to misidentification. In addition, in congested scenarios, lane lines are often obscured by other vehicles around the vehicle, resulting in missing or invisible lane lines.

[0005] 2. Track the trajectory information of objects based on radar or vision, and model the road using the trajectory information of vehicles ahead.

[0006] In this scheme, under congested scenarios, due to the characteristics of radar sensors, such as near-range reflection, multipath problems, and blind spots, near-range radar data exhibits significant interference, resulting in numerous virtual scenes and ultimately leading to poor trajectory tracking performance and low accuracy of the road model used for trajectory prediction.

[0007] In summary, in congested scenarios, the installation angles of sensors and cameras, as well as their inherent principles, limit perception, often resulting in abnormal lane line detection results or even the complete absence of lane lines in the sensor's field of view. Therefore, traditional road modeling techniques are severely limited in congested scenarios. Summary of the Invention

[0008] In view of the above, the present invention aims to provide a road modeling method, apparatus, and device, and accordingly proposes a computer-readable storage medium. Through pure vision post-processing technology, the target vehicle is determined in the vehicle's driving environment based on the vehicle's current speed and prior road curvature. Based on the target vehicle's driving information, a suitable road curvature is determined for the vehicle, thereby determining the vehicle's virtual driving area and avoiding road modeling failure caused by lane line loss due to limitations of the vehicle's perception system.

[0009] The technical solution adopted in this invention is as follows:

[0010] In a first aspect, the present invention provides a road modeling method, which iteratively performs the following steps:

[0011] Acquire road environment images obtained by the vehicle;

[0012] Based on the curvature of the first road, the current speed of the vehicle, and the structural information of the road, multiple first target vehicles around the vehicle in the current image frame are selected as candidate vehicle queues within the candidate region.

[0013] The second road curvature of the road within the candidate area is determined based on the driving information of all first target vehicles in the candidate vehicle queue;

[0014] Based on the curvature of the second road and the vehicle's current speed, a virtual driving area for the current vehicle is constructed in the current image frame;

[0015] Update the curvature of the first path using the curvature of the second path.

[0016] In one possible implementation, multiple first target vehicles around the vehicle in the current image frame are selected as a candidate vehicle queue within the candidate region based on the first road curvature information, the vehicle's current speed, and the road structure information. Specifically, this includes:

[0017] Candidate regions are determined in the current image frame based on the curvature of the first road, the current speed of the vehicle, the preset time, and the structural information of the road. The candidate regions are arc-shaped regions.

[0018] Vehicles that fall into the candidate region in the current image frame are designated as first target vehicles, and all first target vehicles form a candidate vehicle queue.

[0019] In one possible implementation, candidate regions are determined in the current image frame based on the first road curvature, the vehicle's current speed, a preset time, and road structure information, specifically including:

[0020] The angle of the arc-shaped area is determined based on the vehicle's current speed, preset time, and the curvature of the first road.

[0021] The reciprocal of the curvature of the first road is used as the radius of curvature of the virtual trajectory of the vehicle in the arc region;

[0022] Using the virtual trajectory of the vehicle as the centerline of the first lane where the vehicle is located, the first candidate area of ​​the first lane is determined based on the angle and radius of curvature.

[0023] Based on the first candidate region, determine at least one second candidate region around the first lane and combine the first candidate region and all second candidate regions into a candidate region.

[0024] In one possible implementation, the second road curvature within the candidate area is determined based on the driving information of all first target vehicles in the candidate vehicle queue, specifically including:

[0025] For each first target vehicle, the first heading angular velocity of the first target vehicle in the current image frame is obtained by smoothing the actual heading angular velocity and time forgetting factor of the first target vehicle in the current image frame using the current image frame and the first preset number of previous image frames.

[0026] The second road curvature is determined using the first heading angular velocity of all first target vehicles.

[0027] In one possible implementation, the second road curvature is determined using the first heading angular velocity of all first target vehicles, specifically including:

[0028] Perform a normality test using the first heading angular velocities of all first target vehicles in the current image frame;

[0029] If the normality test passes, the average of the first heading angular velocities of all first target vehicles in the current image frame and the previous first preset number of image frames is taken as the second heading angular velocity, and the second road curvature is calculated based on the second heading angular velocity and the current vehicle speed.

[0030] In one possible implementation, if the normality test fails, the correlation between the longitudinal distance between all first target vehicles and the vehicle in the current image frame and the first heading angular velocity of the first target vehicle is calculated. First target vehicles with a correlation greater than a first threshold are selected as second target vehicles. The proportion of the number of second target vehicles in the candidate vehicle queue is counted, and then it is determined whether the proportion is greater than the second threshold.

[0031] If the ratio is greater than the second threshold, the average of the first heading angular velocities of all first target vehicles in the current image frame and the previous first preset number of image frames is taken as the second heading angular velocity, and the second road curvature is calculated based on the second heading angular velocity and the current vehicle speed.

[0032] In one possible implementation, if the proportion is less than or equal to the second threshold, the current image frame is discarded, and the average of the first heading angular velocities of all first target vehicles in the consecutive second preset number of image frames before the current image frame is taken as the third heading angular velocity, and the second road curvature is calculated based on the third heading angular velocity and the current vehicle speed.

[0033] In a second aspect, the present invention provides a road modeling device, including an image acquisition module, a queue selection module, a curvature calculation module, a virtual driving area construction module, and an update module;

[0034] The image acquisition module is used to acquire road environment images obtained by the vehicle.

[0035] The queue selection module is used to select multiple first target vehicles around the vehicle in the current image frame as candidate vehicle queues within the candidate area based on the first road curvature, the vehicle's current speed, and the road structure information.

[0036] The curvature calculation module is used to determine the second road curvature of the road within the candidate area based on the driving information of all the first target vehicles in the candidate vehicle queue;

[0037] The virtual driving area construction module is used to construct the virtual driving area of ​​the current vehicle in the current image frame based on the curvature of the second road and the current speed of the vehicle.

[0038] The update module is used to update the curvature of the first road using the curvature of the second road.

[0039] In one possible implementation, the queue selection module includes a candidate region determination module and a first target vehicle determination module;

[0040] The candidate region determination module is used to determine candidate regions in the current image frame based on the first road curvature, the vehicle's current speed, the preset time, and the road structure information. The candidate regions are arc-shaped regions.

[0041] The first target vehicle determination module is used to identify vehicles that fall into the candidate region in the current image frame as first target vehicles, and all first target vehicles form a candidate vehicle queue.

[0042] In one possible implementation, the curvature calculation module includes a heading angular velocity processing module and a curvature determination module;

[0043] The heading angular velocity processing module is used to smooth the actual heading angular velocity and time forgetting factor of the first target vehicle in the current image frame and the previous first preset number of image frames for each first target vehicle, so as to obtain the first heading angular velocity of the first target vehicle in the current image frame.

[0044] The curvature determination module is used to determine the second road curvature using the first heading angular velocity of all first target vehicles.

[0045] Thirdly, the present invention provides a road modeling device, comprising:

[0046] One or more processors, memory, and one or more computer programs, wherein the one or more computer programs are stored in memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the road modeling method described above.

[0047] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method as described in the first aspect or any possible implementation thereof.

[0048] The concept of this invention is to determine the target vehicle in the driving environment of the vehicle by using a purely visual post-processing technique based on the current speed of the vehicle and the prior road curvature, and to determine the road curvature suitable for the vehicle based on the driving information of the target vehicle, thereby determining the virtual driving area of ​​the vehicle and avoiding road modeling failure caused by lane line loss due to the limitations of the vehicle perception system.

[0049] Furthermore, the road modeling of this invention adopts a circular arc model, which takes into account both curves and straight sections (a special case of curves) simultaneously, resulting in a stronger ability to detect curves in advance and improving the accuracy of road modeling.

[0050] In addition, this invention uses the angular velocity information of the vehicle ahead to predict the road curvature, thereby modeling the road. It eliminates the dependence on lane lines and the influence of congested environment on sensing devices, thus achieving better dynamic performance and providing more accurate road information in congested scenarios. Attached Figure Description

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings, wherein:

[0052] Figure 1 A flowchart of the road modeling method provided by the present invention;

[0053] Figure 2 An example of a road environment image obtained by a vehicle;

[0054] Figure 3 This is an example of a candidate region in the first loop provided by the present invention;

[0055] Figure 4 This is an example of a candidate region in a non-first-cycle provided by the present invention;

[0056] Figure 5 This is an example of the virtual driving area of ​​the current vehicle provided by the present invention;

[0057] Figure 6 A structural diagram of the road modeling device provided by the present invention;

[0058] Figure 7 A schematic diagram of the road modeling device provided by the present invention. Detailed Implementation

[0059] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0060] This invention utilizes a purely visual post-processing technique to identify target vehicles within the vehicle's driving environment based on the vehicle's current speed and prior road curvature. It then determines a suitable road curvature for the vehicle based on the target vehicle's driving information, thereby defining the vehicle's virtual driving area. This avoids road modeling failures caused by lane line omissions due to limitations in the vehicle's perception system. Furthermore, the road modeling employed in this invention uses a circular arc model, simultaneously considering curves and straight sections (a special case of curves), resulting in stronger early curve perception and improved road modeling accuracy. Additionally, this invention uses the yaw rate information of vehicles ahead to predict road curvature, thus modeling the road and eliminating reliance on lane lines and the influence of congested environments on sensing devices. Therefore, it offers better dynamic performance and provides more accurate road information in congested scenarios.

[0061] It should be noted that the road modeling method, apparatus and equipment of this application are applicable not only to road congestion scenarios, but also to non-congestion scenarios.

[0062] In view of the aforementioned core concept, the present invention provides an embodiment of at least one road modeling method, such as... Figure 1 As shown, the method iteratively performs the following steps:

[0063] S110: Acquire road environment images obtained by the vehicle.

[0064] Specifically, the road environment image displays the situation within the field of view (FOV) of the vehicle's camera, including the road and obstacles. Obstacles are obtained using the vehicle's camera and perception capabilities. It should be noted that the road environment image is implemented using existing technology, and this invention does not limit its scope.

[0065] Understandably, road environment images show the road environment in front of the vehicle. Figure 2 An example of a road environment image is given, such as Figure 2 As shown, the road environment image includes information about the lane in the direction of travel and information about the opposite lane, but in actual use, the area of ​​interest is often the information about the lane in the direction of travel.

[0066] S120: Based on the curvature of the first road (kappa1) and the vehicle's current speed (V) ego In addition, the road structure information is used to select multiple first target vehicles around the vehicle in the current image frame as a candidate vehicle queue within the candidate area.

[0067] It should be noted that the first road curvature is the second road curvature obtained in the previous cycle (i.e., the result obtained in step S130 in this process), which is the prior road curvature.

[0068] In one possible implementation, in the first loop after the vehicle is powered on, the first road curvature is the road curvature of the straight section (0). In another possible implementation, in the first loop after the vehicle is powered on, the first road curvature can be the road curvature when the vehicle was last powered off.

[0069] The road's structural information includes the width d of the lanes and the number of lanes in this direction.

[0070] In one possible implementation, based on the first road curvature kappa1 and the vehicle's current speed V... ego In addition to selecting multiple primary target vehicles around the vehicle in the current image frame based on road structure information, this is used as a candidate vehicle queue within the candidate region, specifically including:

[0071] S1201: Based on the first road curvature kappa1 and the vehicle's current speed V ego Based on a preset time t and road structure information, candidate regions are determined in the current image frame. These candidate regions are arc-shaped. The angle between the radius of a point (x, y) within the candidate region and the starting line is...

[0072] In one possible implementation, determining the candidate region includes the following steps:

[0073] P1: Based on the vehicle's current speed V ego The angle of the arc-shaped region is determined by the preset time t and the curvature of the first road kappa1.

[0074]

[0075] In one possible implementation, t = 5s. Understandably, t can be any other preset time.

[0076] P2: The reciprocal of the curvature of the first road, kappa1, is used as the radius of curvature R1 of the vehicle's virtual trajectory in the arc region.

[0077] R1=1 / kappa1 (2)

[0078] Wherein, the radius of curvature R1 is the starting point of the vehicle, and the virtual trajectory of the vehicle is an arc-shaped trajectory extending along the direction of the vehicle's motion, with the line connecting the center of the circle and the vehicle as the starting point (hereinafter referred to as the starting line), as shown below. Figure 4 As shown.

[0079] P3: Using the virtual trajectory of the vehicle as the centerline of the first lane where the vehicle is located, based on the angle... The first candidate region A1 of the first lane is determined by the radius of curvature R1.

[0080] The point (x1, y1) in the first candidate region A1 is represented as follows:

[0081]

[0082] in, This represents the angle between the radius of point (x1, y1) and the starting line.

[0083] P4: Based on the first candidate region A1, determine the second candidate region A2 of at least one second lane around the first lane, and combine the first candidate region A1 and all the second candidate regions A2 into the entire candidate region.

[0084] The point (x2, y2) on the second candidate region A2 of the second lane adjacent to the inside of the first lane is represented as follows:

[0085]

[0086] in, This represents the angle between the radius of the point (x2, y2) and the starting line.

[0087] The point (x3, y3) on the second candidate region A3 of the second lane adjacent to the outer side of the first lane is represented as follows:

[0088]

[0089] in, This represents the angle between the radius of the point (x3, y3) and the starting line.

[0090] The second candidate areas for other lane lines were determined in a similar manner.

[0091] In one possible implementation, the candidate region is a three-lane arc-shaped area centered on the first lane, such as... Figure 4 As shown.

[0092] It should be noted that if the curvature of the first road in the first loop is 0 (the curvature of the straight road), the candidate region is a rectangle. Figure 3 An example is given where the width of the candidate region is 3d.

[0093] S1202: The vehicles that fall into the candidate region in the current image frame are taken as the first target vehicles, and all the first target vehicles form a candidate vehicle queue.

[0094] S130: Determine the second road curvature, denoted as kappa2, based on the driving information of all first target vehicles in the candidate vehicle queue.

[0095] In one possible implementation, the second road curvature of the road within the candidate area can be determined by comprehensively analyzing the speed vectors of all the first target vehicles, i.e., the road curvature of the road on which the vehicle is about to travel.

[0096] In another possible implementation, the second road curvature within the candidate region is determined by the heading angular velocity of all first target vehicles, specifically including:

[0097] S1301: For each first target vehicle, the actual heading angular velocity and time forgetting factor of the first target vehicle in the current image frame and the previous first preset number of image frames are used for smoothing to obtain the first heading angular velocity of the first target vehicle in the current image frame, so as to obtain a more accurate first heading angular velocity.

[0098] Wherein, the first heading angular velocity of the i-th first target vehicle in the fi-th frame is expressed as:

[0099]

[0100] in, Let be the actual heading angular velocity of the i-th first target vehicle in the fi-th frame. Let be the actual heading angular velocity of the i-th first target vehicle in the (fi-1)-th frame. Let be the actual heading angular velocity of the i-th first target vehicle in the (fi-2)-th frame.

[0101] S1302: Determine the second road curvature using the first heading angular velocity of all first target vehicles.

[0102] In one possible implementation, the average of the first heading angular velocities of all first target vehicles in the current graphics frame is used as the second heading angular velocity, and the second road curvature is calculated based on the second heading angular velocity and the current vehicle speed.

[0103] Specifically, the angular velocity w of a vehicle's heading is related to the road curvature kappa of the road it is on as follows:

[0104] w = v * kappa (7)

[0105] Where v represents the vehicle's speed.

[0106] Therefore, after determining the second heading angular velocity, the second road curvature can be determined by the above formula (7).

[0107] In a preferred embodiment, determining the second road curvature using the first heading angular velocity of all first target vehicles specifically includes:

[0108] Q1: Perform a normality test using the first heading angular velocities of all first target vehicles in the current image frame. If the normality test passes, proceed to Q2; otherwise, proceed to Q3.

[0109] Q2: If the normality test passes, it means that all the first target vehicles are in the curve or on the straight road. Then, the average of the first heading angular velocities of all the first target vehicles in the current image frame and the previous first preset number of image frames is taken as the second heading angular velocity. The second road curvature is calculated based on the second heading angular velocity and the current speed of the vehicle. Please refer to formula (7).

[0110] Q3: Calculate the correlation between the longitudinal distance between all first target vehicles and the vehicle in the current image frame and the first heading angular velocity of the first target vehicles. First target vehicles with a correlation greater than a first threshold are designated as second target vehicles. The proportion of second target vehicles in the candidate vehicle queue is then calculated, and it is determined whether this proportion is greater than a second threshold. If yes, it indicates that some first target vehicles are in the transition zone from a straight road to a curve or from a curve to a straight road, so proceed to Q2; otherwise, proceed to Q4.

[0111] Q4: If the ratio is less than or equal to the second threshold, it indicates that some first target vehicles in the current image frame have irregular driving conditions. In this case, the current image frame is discarded, and the average of the first heading angular velocities of all first target vehicles in the second preset number of consecutive image frames before the current image frame is taken as the third heading angular velocity. The second road curvature is calculated based on the third heading angular velocity and the current vehicle speed. Please refer to formula (7).

[0112] In one possible implementation, the second preset quantity is the first preset quantity plus one.

[0113] S140: Construct the virtual driving area of ​​the current vehicle in the current image frame based on the curvature of the second road and the current speed of the vehicle.

[0114] The virtual driving area is the area of ​​the single lane where the vehicle is located.

[0115] If kappa2 ≠ 0, please refer to steps P1-P3 to determine the virtual driving area, that is:

[0116]

[0117] R² = 1 / kappa² (9)

[0118] Figure 5 An example of a virtual driving area is shown.

[0119] If kappa2 = 0, then the virtual driving area is a rectangular area with a length of V. ego *T1, width is lane width d.

[0120] In one possible implementation, T1 = 4s. Understandably, T1 can also be other values.

[0121] The virtual driving area can be transmitted to the vehicle's decision unit, which then controls the vehicle's controller to drive the vehicle within that virtual driving area.

[0122] S150: Update the curvature of the first road using the curvature of the second road.

[0123] Corresponding to the above embodiments and preferred solutions, the present invention also provides an embodiment of a road modeling device, such as... Figure 6 As shown, it may specifically include an image acquisition module 610, a queue selection module 620, a curvature calculation module 630, a virtual driving area construction module 640, and an update module 650.

[0124] The image acquisition module 610 is used to acquire road environment images obtained by the vehicle.

[0125] The queue selection module 620 is used to select multiple first target vehicles around the vehicle in the current image frame as candidate vehicle queues within the candidate area based on the first road curvature, the vehicle's current speed, and the road structure information.

[0126] The curvature calculation module 630 is used to determine the second road curvature of the road within the candidate area based on the driving information of all first target vehicles in the candidate vehicle queue.

[0127] The virtual driving area construction module 640 is used to construct the virtual driving area of ​​the current vehicle in the current image frame based on the curvature of the second road and the current speed of the vehicle.

[0128] The update module 650 is used to update the curvature of the first road using the curvature of the second road.

[0129] In one possible implementation, the queue selection module 620 includes a candidate region determination module 6201 and a first target vehicle determination module 6202.

[0130] The candidate region determination module 6201 is used to determine a candidate region in the current image frame based on the first road curvature, the current speed of the vehicle, a preset time, and the structural information of the road. The candidate region is an arc-shaped region.

[0131] The first target vehicle determination module 6202 is used to identify vehicles falling into the candidate region in the current image frame as the first target vehicles, and all the first target vehicles form the candidate vehicle queue.

[0132] In one possible implementation, the curvature calculation module 630 includes a heading angular velocity processing module 6301 and a curvature determination module 6302.

[0133] The heading angular velocity processing module 6301 is used to smooth the actual heading angular velocity and time forgetting factor of the first target vehicle in the current image frame and the previous first preset number of image frames for each first target vehicle, so as to obtain the first heading angular velocity of the first target vehicle in the current image frame.

[0134] The curvature determination module 6302 is used to determine the second road curvature using the first heading angular velocity of all the first target vehicles.

[0135] The above should be understood Figure 6 The division of components in the road modeling device shown is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or physically separated. These components can be implemented entirely in software via processing element calls; entirely in hardware; or partially in software via processing element calls and partially in hardware. For example, a particular module can be a separate processing element or integrated into a chip in an electronic device. The implementation of other components is similar. Furthermore, these components can be fully or partially integrated together or implemented independently. During implementation, each step of the above method or each of the above components can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0136] For example, these components can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, these components can be integrated together to form a System-On-a-Chip (SOC).

[0137] Based on the above embodiments and preferred solutions, those skilled in the art will understand that, in practice, the present invention is applicable to various implementation methods. The present invention is illustrated by the following carrier:

[0138] (1) A road modeling device, which may include:

[0139] One or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the steps / functions of the foregoing embodiments or equivalent embodiments.

[0140] Figure 7 This is a schematic diagram illustrating the structure of an embodiment of the road modeling device of the present invention. The device can be an electronic device or a circuit device built into such an electronic device. The electronic device can be a PC, a smart terminal (mobile phone, tablet, watch, etc.), a smart TV, a smart screen, a robot, a drone, an ICV, a smart (car) vehicle, or in-vehicle equipment, etc. This embodiment does not limit the specific form of the road modeling device.

[0141] Specifically, such as Figure 7 As shown, the road modeling device 900 includes a processor 910 and a memory 930. The processor 910 and the memory 930 can communicate with each other via an internal connection to transmit control and / or data signals. The memory 930 stores computer programs, and the processor 910 retrieves and runs these programs from the memory 930. The processor 910 and memory 930 can be combined into a single processing unit, but more commonly they are independent components. The processor 910 executes the program code stored in the memory 930 to achieve the aforementioned functions. In specific implementations, the memory 930 can be integrated into the processor 910, or it can be independent of the processor 910.

[0142] In addition, to further enhance the functionality of the road modeling device 900, the device 900 may also include one or more of the following: an input unit 960, a display unit 970, an audio circuit 980, a camera 990, and a sensor 901. The audio circuit may also include a speaker 982, a microphone 984, etc. The display unit 970 may include a display screen.

[0143] Furthermore, the aforementioned road modeling device 900 may also include a power supply 950 for providing electrical power to various devices or circuits in the device 900.

[0144] It should be understood that Figure 7The road modeling device 900 shown is capable of implementing the various processes of the method provided in the foregoing embodiments. The operation and / or function of each component in the device 900 can respectively implement the corresponding processes in the above method embodiments. For details, please refer to the foregoing descriptions of the embodiments of methods, devices, etc.; to avoid repetition, detailed descriptions are appropriately omitted here.

[0145] It should be understood that Figure 7 The processor 910 in the road modeling device 900 shown can be a system-on-a-chip (SoC). The processor 910 may include a central processing unit (CPU) and may further include other types of processors, such as a graphics processing unit (GPU), which will be described in detail below.

[0146] In summary, the various processors or processing units inside the processor 910 can work together to implement the previous method flow, and the corresponding software programs of each processor or processing unit can be stored in the memory 930.

[0147] (2) A readable storage medium storing a computer program or the above-described device, which, when executed, causes a computer to perform the steps / functions of the foregoing embodiments or equivalent embodiments.

[0148] In several embodiments provided by this invention, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, certain technical solutions of this invention, or the parts that contribute to the prior art, or parts of such technical solutions, can be embodied in the form of software products as described below.

[0149] (3) A computer program product (which may include the above-described apparatus) that, when run on a terminal device, causes the terminal device to execute the road modeling method of the foregoing embodiments or equivalent embodiments.

[0150] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the above implementation methods can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the above-mentioned computer program products may include, but are not limited to, APP; continuing from the foregoing, the above-mentioned device / terminal may be a computer device (e.g., mobile phone, PC terminal, cloud platform, server, server cluster, or network communication device such as media gateway, etc.). Furthermore, the hardware structure of the computer device may specifically include: at least one processor, at least one communication interface, at least one memory, and at least one communication bus; the processor, communication interface, and memory can all communicate with each other through the communication bus. The processor may be a central processing unit (CPU), DSP, microcontroller, or digital signal processor, and may also include a GPU, an embedded neural network processing unit (NPU), and an image signal processor (ISP). The processor may also include a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. Furthermore, the processor may have the function of operating one or more software programs, which may be stored in a storage medium such as a memory. The aforementioned memory / storage medium may include: non-volatile memory, such as a non-removable disk, USB flash drive, portable hard drive, optical disc, etc., as well as read-only memory (ROM), random access memory (RAM), etc.

[0151] In this embodiment of the invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0152] Those skilled in the art will recognize that the modules, units, and method steps described in the embodiments disclosed in this specification can be implemented using electronic hardware, computer software, and a combination of electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0153] Furthermore, the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. In particular, for embodiments such as apparatus and devices, since they are basically similar to the method embodiments, the relevant parts can be referred to the description of the method embodiments. The apparatus, devices, and other embodiments described above are merely illustrative, and the modules, units, etc., described as separate components may or may not be physically separate, that is, they may be located in one place or distributed in multiple places, such as nodes in a system network. Specifically, some or all of the modules and units can be selected according to actual needs to achieve the purpose of the above-described embodiment solutions. Those skilled in the art can understand and implement this without creative effort.

[0154] The above description of the structure, features, and effects of the present invention is based on the embodiments shown in the figures. However, the above are only preferred embodiments of the present invention. It should be noted that the technical features involved in the above embodiments and their preferred methods can be reasonably combined and matched by those skilled in the art to form a variety of equivalent solutions without departing from or changing the design concept and technical effects of the present invention. Therefore, the present invention is not limited to the scope of implementation shown in the figures. Any changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments, that do not exceed the spirit covered by the specification and figures, should be within the protection scope of the present invention.

Claims

1. A road modeling method, characterized in that, Repeat the following steps: Acquire road environment images obtained by the vehicle; Based on the curvature of the first road, the current speed of the vehicle, and the structural information of the road, multiple first target vehicles around the vehicle in the current image frame are selected as candidate vehicle queues within the candidate region. The second road curvature of the road in the candidate area is determined based on the driving information of all first target vehicles in the candidate vehicle queue. Specifically, the second road curvature is determined by using the first heading angular velocity of all first target vehicles, including performing a normality test on the first heading angular velocity of all first target vehicles in the current image frame and filtering out the first heading angular velocity used to determine the second road curvature. Based on the curvature of the second road and the vehicle's current speed, a virtual driving area for the current vehicle is constructed in the current image frame; The curvature of the first road is updated using the curvature of the second road.

2. The road modeling method according to claim 1, characterized in that, Based on the curvature information of the first road, the current speed of the vehicle, and the structural information of the road, multiple first target vehicles around the vehicle in the current image frame are selected as candidate vehicles within the candidate region, specifically including: Candidate regions are determined in the current image frame based on the curvature of the first road, the current speed of the vehicle, the preset time, and the structural information of the road. The candidate regions are arc-shaped regions. Vehicles falling into the candidate region in the current image frame are designated as the first target vehicles, and all first target vehicles form the candidate vehicle queue.

3. The road modeling method according to claim 2, characterized in that, Based on the curvature of the first road, the current speed of the vehicle, a preset time, and the structural information of the road, candidate regions are determined in the current image frame, specifically including: The angle of the arc-shaped region is determined based on the vehicle's current speed, the preset time, and the curvature of the first road. The reciprocal of the curvature of the first road is used as the radius of curvature of the virtual trajectory of the vehicle in the arc region; Using the virtual trajectory of the vehicle as the centerline of the first lane where the vehicle is located, the first candidate area of ​​the first lane is determined based on the angle and the radius of curvature. Based on the first candidate region, determine a second candidate region for at least one second lane surrounding the first lane, and combine the first candidate region and all second candidate regions into the candidate region.

4. The road modeling method according to claim 1, characterized in that, The methods for obtaining the first heading angular velocity include: For each first target vehicle, the first heading angular velocity of the first target vehicle in the current image frame is obtained by smoothing the actual heading angular velocity and time forgetting factor of the first target vehicle in the current image frame using the current image frame and the first preset number of previous image frames.

5. The road modeling method according to claim 1, characterized in that, The second road curvature is determined using the first heading angular velocity of all first target vehicles, specifically including: If the normality test passes, the average of the first heading angular velocities of all first target vehicles in the current image frame and the previous first preset number of image frames is taken as the second heading angular velocity, and the second road curvature is calculated based on the second heading angular velocity and the current vehicle speed.

6. The road modeling method according to claim 1, characterized in that, If the normality test fails, the correlation between the longitudinal distance between all first target vehicles and the vehicle in the current image frame and the first heading angular velocity of the first target vehicle is calculated. First target vehicles with a correlation greater than the first threshold are selected as second target vehicles. The proportion of the number of second target vehicles in the candidate vehicle queue is counted, and then it is determined whether the proportion is greater than the second threshold. If the ratio is greater than the second threshold, the average of the first heading angular velocities of all first target vehicles in the current image frame and the previous first preset number of image frames is taken as the second heading angular velocity, and the second road curvature is calculated based on the second heading angular velocity and the current vehicle speed.

7. The road modeling method according to claim 6, characterized in that, If the ratio is less than or equal to the second threshold, the current image frame is discarded, and the average of the first heading angular velocities of all first target vehicles in the second consecutive preset number of image frames before the current image frame is taken as the third heading angular velocity. The second road curvature is then calculated based on the third heading angular velocity and the current vehicle speed.

8. A road modeling device, characterized in that, It includes an image acquisition module, a queue selection module, a curvature calculation module, a virtual driving area construction module, and an update module; The image acquisition module is used to acquire road environment images obtained by the vehicle. The queue selection module is used to select multiple first target vehicles around the vehicle in the current image frame as candidate vehicle queues within the candidate area based on the first road curvature, the vehicle's current speed, and the road structure information. The curvature calculation module is used to determine the second road curvature of the road in the candidate area based on the driving information of all first target vehicles in the candidate vehicle queue. Specifically, it uses the first heading angular velocity of all first target vehicles to determine the second road curvature, including performing a normality test on the first heading angular velocity of all first target vehicles in the current image frame and filtering out the first heading angular velocity used to determine the second road curvature. The virtual driving area construction module is used to construct the virtual driving area of ​​the current vehicle in the current image frame based on the curvature of the second road and the current speed of the vehicle. The update module is used to update the curvature of the first road using the curvature of the second road.

9. The road modeling device according to claim 8, characterized in that, The queue selection module includes a candidate region determination module and a first target vehicle determination module; The candidate region determination module is used to determine a candidate region in the current image frame based on the first road curvature, the current speed of the vehicle, a preset time, and the structural information of the road. The candidate region is an arc-shaped region. The first target vehicle determination module is used to identify vehicles that fall into the candidate region in the current image frame as the first target vehicles, and all the first target vehicles form the candidate vehicle queue.

10. The road modeling device according to claim 8, characterized in that, The curvature calculation module includes a heading angular velocity processing module and a curvature determination module; The heading angular velocity processing module is used to smooth the actual heading angular velocity and time forgetting factor of the first target vehicle in the current image frame and the previous first preset number of image frames for each first target vehicle, so as to obtain the first heading angular velocity of the first target vehicle in the current image frame. The curvature determination module is used to determine the second road curvature using the first heading angular velocity of all first target vehicles.

11. A road modeling device, characterized in that, include: One or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the road modeling method as described in any one of claims 1 to 7.

Images