Virtual track identification method and device, equipment, medium and computer program product
By using a deep reinforcement learning neural network model to identify virtual tracks under unsupervised conditions, this method solves the problems of difficult dataset preparation and poor adaptability in traditional methods, and achieves low-cost and highly adaptable virtual track identification.
Patent Information
- Application Number
- CN202411123919.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2026-03-03
AI Technical Summary
Existing virtual track recognition methods require a large amount of manual preparation of image datasets, and the coupling of control and perception results in poor adaptability, making them unable to adapt to different vehicle control characteristics.
Using a deep reinforcement learning neural network model, images of the road surface in front of the vehicle are captured in simulated or real road scenarios. The geometric curve equation parameters of the virtual track are output through unsupervised learning and decoupled from the vehicle control, allowing the system to autonomously learn and adapt to different vehicle characteristics.
It eliminates the need for large image datasets, reduces manpower and material costs, adapts to different vehicle control characteristics, and improves the applicability and efficiency of virtual track recognition.
Smart Images
Figure CN121600482A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving technology, and in particular to a virtual track recognition method, device, equipment, medium, and computer program product. Background Technology
[0002] Virtual lane recognition is a crucial task in autonomous driving, helping vehicles accurately locate and navigate on roads, avoiding lane departures and collisions. Virtual lane recognition primarily falls into two categories: traditional machine vision methods and deep learning methods. Traditional machine vision methods are complex to design and suffer from poor detection accuracy and adaptability. Deep learning methods utilize deep neural networks to learn and infer from large amounts of image data, autonomously learning lane image features. However, deep learning is supervised learning, requiring manual preparation of large lane image datasets and significant manpower and resources for cleaning, filtering, and standardizing these datasets. Furthermore, traditional deep learning methods employ an end-to-end structure, inputting images and outputting directional control data, thus coupling perception and control together. This makes them highly correlated with vehicle control characteristics, requiring relearning when those characteristics change. Therefore, addressing these technical shortcomings has become a pressing issue for those skilled in the art. Summary of the Invention
[0003] The purpose of this application is to provide a virtual track recognition method that does not require a large image dataset, can greatly reduce manpower and material costs, and has strong adaptability.
[0004] To address the aforementioned technical problems, this application provides a virtual track recognition method, comprising:
[0005] Capture images of the road surface in front of the vehicle in simulated or real road scenarios;
[0006] The image of the road in front of the vehicle is input into a deep reinforcement learning neural network model, and the deep reinforcement learning neural network model outputs the equation parameters of the geometric curve corresponding to the virtual track.
[0007] The equation parameters are output to the vehicle lateral control system so that the vehicle lateral control system performs vehicle lateral control based on the equation parameters.
[0008] The reward is determined based on the vehicle lateral control result of the vehicle lateral control system, and the deep reinforcement learning neural network model is updated based on the reward.
[0009] Once the reward meets the preset conditions, the image of the road in front of the vehicle is input into the deep reinforcement learning neural network model, and the virtual track is identified through the deep reinforcement learning neural network model.
[0010] In some embodiments, determining the reward based on the vehicle lateral control result of the vehicle lateral control system includes:
[0011] The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track, as well as the number of times the wheels cross the lane lines.
[0012] In some embodiments, the reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track, as well as the number of times the wheels cross the lane lines.
[0013] Calculate the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives;
[0014] Count the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment;
[0015] The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives, and the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment.
[0016] In some embodiments, the reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment is reached, and the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment.
[0017] The negative reward at the next shooting moment is calculated based on offset_value = [lane_offset × (-cross_lane_value)] / (0.5 ×lane_width);
[0018] The reward is calculated based on Reward = offset_value + n * (-cross_lane_value);
[0019] Where offset_value represents the negative reward, lane_offset represents the deviation between the target reference point on the vehicle and the virtual track when the next shooting time is reached, -cross_lane_value represents the reward when any wheel crosses the lane line, cross_lane_value is a positive value, lane_width represents the lane width, n represents the number of times the wheel crosses the lane line between the next shooting time and the previous shooting time, and Reward represents the reward.
[0020] In some embodiments, the step of waiting until the reward meets a preset condition includes:
[0021] The reward will continue to exceed the preset threshold for a predetermined number of consecutive preset number of times.
[0022] In some embodiments, inputting the image of the road surface in front of the vehicle into a deep reinforcement learning neural network model includes:
[0023] Multiple adjacent frames of road images in front of the vehicle are input into the deep reinforcement learning neural network model.
[0024] In some embodiments, multiple adjacent images of the road surface in front of the vehicle are captured by the same camera; or, multiple adjacent images of the road surface in front of the vehicle are captured by multiple cameras.
[0025] In some embodiments, before inputting the image of the road surface in front of the vehicle into the deep reinforcement learning neural network model, the method further includes:
[0026] The image of the road surface in front of the vehicle is preprocessed to remove invalid image areas and retain the road surface area.
[0027] In some embodiments, before inputting the image of the road surface in front of the vehicle into the deep reinforcement learning neural network model, the method further includes:
[0028] Convert the RGB format image of the road in front of the vehicle into YUV format.
[0029] In some embodiments, the deep reinforcement learning neural network model outputs four floating-point numbers, which are the coefficients of the cubic, quadratic, and linear terms of the geometric curve corresponding to the virtual track, as well as the constant term value.
[0030] To address the aforementioned technical problems, this application also provides a virtual track recognition device, comprising:
[0031] The camera module is used to capture images of the road surface in front of the vehicle in simulated or real road scenarios.
[0032] The input module is used to input the image of the ground road in front of the vehicle into the deep reinforcement learning neural network model, and the deep reinforcement learning neural network model outputs the equation parameters of the geometric curve corresponding to the virtual track.
[0033] An output module is used to output the equation parameters to the vehicle lateral control system, so that the vehicle lateral control system can perform vehicle lateral control according to the equation parameters.
[0034] An update module is used to determine a reward based on the vehicle lateral control result of the vehicle lateral control system, and update the deep reinforcement learning neural network model based on the reward.
[0035] The recognition module is used to input the image of the ground road in front of the vehicle into the deep reinforcement learning neural network model until the reward meets the preset conditions, and then recognize the virtual track through the deep reinforcement learning neural network model.
[0036] To address the aforementioned technical problems, this application also provides a virtual track recognition device, comprising:
[0037] Memory, used to store computer programs;
[0038] A processor is configured to implement the virtual track recognition method described above when executing the computer program.
[0039] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the virtual track recognition method described above.
[0040] To address the aforementioned technical problems, this application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the virtual track recognition method described above.
[0041] The virtual track recognition method provided in this application includes: capturing an image of the ground road in front of a vehicle in a simulated or real road scene; inputting the image of the ground road in front of the vehicle into a deep reinforcement learning neural network model, and having the deep reinforcement learning neural network model output equation parameters of the geometric curve corresponding to the virtual track; outputting the equation parameters to a vehicle lateral control system, so that the vehicle lateral control system performs vehicle lateral control according to the equation parameters; determining a reward based on the vehicle lateral control result of the vehicle lateral control system, and updating the deep reinforcement learning neural network model according to the reward; until the reward meets a preset condition, inputting the image of the ground road in front of the vehicle into the deep reinforcement learning neural network model, and recognizing the virtual track through the deep reinforcement learning neural network model.
[0042] As can be seen, the virtual track recognition method provided in this application adopts an unsupervised autonomous learning approach. This process requires no manual intervention or preparation of large sample datasets, significantly reducing human and material costs. Furthermore, this virtual track recognition method decouples control from perception; virtual track recognition is independent of vehicle control. The vehicle's lateral control system controls the vehicle based on the perception results, i.e., the virtual track recognition results. After autonomous learning, the model can adapt to vehicles with different control characteristics without needing to relearn on vehicles with different control features, thus enhancing its applicability.
[0043] The virtual track recognition device, equipment, computer-readable storage medium, and computer program product provided in this application all have the aforementioned technical effects. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 A flowchart illustrating a virtual track recognition method provided in an embodiment of this application;
[0046] Figure 2 This is a schematic diagram of the structure of a deep reinforcement learning neural network model provided in an embodiment of this application;
[0047] Figure 3 This is a schematic diagram illustrating how a linear proportional relationship is used to determine negative rewards, as provided in an embodiment of this application.
[0048] Figure 4 A schematic diagram illustrating a method for determining negative rewards using a nonlinear high-order curve relationship, provided as an embodiment of this application;
[0049] Figure 5 This is a schematic diagram of a virtual track recognition device provided in an embodiment of this application;
[0050] Figure 6 This is a schematic diagram of a virtual track recognition device provided in an embodiment of this application. Detailed Implementation
[0051] The core of this application is to provide a virtual track recognition method that does not require a large image dataset, can greatly reduce manpower and material costs, and has strong adaptability.
[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0053] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a virtual track recognition method provided in an embodiment of this application. (Refer to...) Figure 1 As shown, the method includes:
[0054] S101: Capture images of the road surface in front of the vehicle in simulated or real road scenarios.
[0055] Setting up a reinforcement learning environment involves creating simulated or real-world road scenarios. This environment detects the vehicle's position within the lane and whether it has crossed the lane markings. In either the simulated or real-world scenario, the vehicle is placed within the lane. This lane is empty of other vehicles, and all four wheels of the vehicle are not crossing the lane markings on either side. A forward-facing camera captures an image of the road ahead of the vehicle.
[0056] S102: Input the image of the road surface in front of the vehicle into the deep reinforcement learning neural network model, and output the equation parameters of the geometric curve corresponding to the virtual track from the deep reinforcement learning neural network model.
[0057] refer to Figure 2 As shown, a deep reinforcement learning neural network model can include a CNN convolutional network, a backbone network, and an output. The CNN convolutional network can be a network structure based on various attention mechanisms according to the transformer principle. The backbone network can adopt any network structure; for example, it can be a variant of various typical classification networks such as ResNet and MobileNet. The deep reinforcement learning neural network first uses the CNN convolutional network to extract features from the input image, and then the backbone network performs subsequent processing. The output of the deep reinforcement learning neural network model is a post-connected MLP network structure, followed by a network output structure, which is a one-layer fully connected network structure. This fully connected network structure uses several nodes; for example, it uses four nodes, meaning the deep reinforcement learning neural network model outputs four floating-point numbers. When an image of the road in front of the vehicle is input into the deep reinforcement learning neural network model, the model outputs the geometric curve parameters corresponding to the virtual track, i.e., the lane centerline.
[0058] In some embodiments, the deep reinforcement learning neural network model outputs four floating-point numbers, which are the coefficients of the cubic, quadratic, and linear terms of the geometric curve corresponding to the virtual track, as well as the constant term value.
[0059] In this embodiment, a 3rd-order curve is used to fit the virtual orbit. Therefore, the deep reinforcement learning neural network model outputs four floating-point numbers, which represent the coefficients of the 3rd, 2nd, 1st, and constant terms of the 3rd-order curve equation, respectively.
[0060] You can input a single frame of image into the deep reinforcement learning neural network each time, or you can input multiple frames of images into the deep reinforcement learning neural network each time (including inputting two frames of images).
[0061] In some embodiments, inputting the image of the road surface in front of the vehicle into a deep reinforcement learning neural network model includes:
[0062] Multiple adjacent frames of road images in front of the vehicle are input into the deep reinforcement learning neural network model.
[0063] In this embodiment, each input of multiple adjacent frames of road images in front of the vehicle to the deep reinforcement learning neural network model allows the deep reinforcement learning neural network model to obtain subtle differences between these adjacent frames, providing the deep reinforcement learning neural network model with richer image detail information, which is beneficial for the autonomous learning of the deep reinforcement learning neural network model.
[0064] In some embodiments, multiple adjacent images of the road surface in front of the vehicle are captured by the same camera; or, multiple adjacent images of the road surface in front of the vehicle are captured by multiple cameras.
[0065] In some embodiments, before inputting the image of the road surface in front of the vehicle into the deep reinforcement learning neural network model, the method further includes:
[0066] The image of the road surface in front of the vehicle is preprocessed to remove invalid image areas and retain the road surface area.
[0067] This embodiment breaks down the original image of the road in front of the vehicle into parts, retaining the road surface area and removing other invalid image areas and information. This reduces the size of the image fed into the deep reinforcement learning neural network model, which helps to reduce the amount of image data processing and improve the autonomous learning efficiency of the deep reinforcement learning neural network model.
[0068] In some embodiments, before inputting the image of the road surface in front of the vehicle into the deep reinforcement learning neural network model, the method further includes:
[0069] Convert the RGB format image of the road in front of the vehicle into YUV format.
[0070] Converting the original image of the road in front of the vehicle from RGB format to YUV format helps reduce the amount of image data processing and improves the autonomous learning efficiency of deep reinforcement learning neural network models.
[0071] S103: Output the equation parameters to the vehicle lateral control system so that the vehicle lateral control system performs vehicle lateral control according to the equation parameters;
[0072] S104: Determine the reward based on the vehicle lateral control of the vehicle lateral control system, and update the deep reinforcement learning neural network model based on the reward result;
[0073] The equation parameters output by the deep reinforcement learning neural network model are fed to the vehicle's lateral control system. The lateral control system then uses these parameters to perform lateral control on the vehicle, ensuring it follows the curve corresponding to the equation parameters output by the deep reinforcement learning neural network model. A reward is determined based on the lateral control results from the lateral control system, and the deep reinforcement learning neural network model is updated accordingly.
[0074] In some embodiments, determining the reward based on the vehicle lateral control result of the vehicle lateral control system includes:
[0075] The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track, as well as the number of times the wheels cross the lane lines.
[0076] The target reference point can be the vehicle control center. The vehicle control center is used as the origin of the vehicle coordinate system, with the front of the vehicle in the positive X-axis direction, the left side in the positive Y-axis direction, and the vertical direction in the positive Z-axis direction. A learning reward and penalty mechanism is established: when the output of the deep reinforcement learning neural network model analyzes the image and meets the requirements, a positive reward is given; otherwise, a penalty (negative reward) is given. Based on the reward, backpropagation is performed on the deep reinforcement learning neural network model to update the weight coefficients of each node.
[0077] In some embodiments, the reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track, as well as the number of times the wheels cross the lane lines.
[0078] Calculate the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives;
[0079] Count the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment;
[0080] The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives, and the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment.
[0081] In some embodiments, the reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives, and the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment.
[0082] The negative reward at the next shooting moment is calculated based on offset_value = [lane_offset × (-cross_lane_value)] / (0.5 ×lane_width);
[0083] The reward is calculated based on Reward = offset_value + n * (-cross_lane_value);
[0084] Where offset_value represents the negative reward, lane_offset represents the deviation between the target reference point on the vehicle and the virtual track when the next shooting time is reached, -cross_lane_value represents the reward when any wheel crosses the lane line, cross_lane_value is a positive value, lane_width represents the lane width, n represents the number of times the wheel crosses the lane line between the next shooting time and the previous shooting time, and Reward represents the reward.
[0085] Combination Figure 3 As shown, the positive reward can be zero. When any of the four wheels crosses the lane line, the reward is negative, specifically represented by the score -cross_lane_value (cross_lane_value is a positive number), and the vehicle returns to the starting center position of the lane. No reward is calculated during this period.
[0086] The reward is determined based on whether the vehicle's center position is in the center of its lane. Let the lane width be *lane_width*, and the lateral distance of the vehicle's center position from the virtual track be *lane_offset*. The negative reward score is *offset_value*, calculated using formula (1):
[0087] offset_value = [lane_offset × (-cross_lane_value)] / (0.5 × lane_width) < 0 (1)
[0088] One or more frames of images are fed into a deep reinforcement learning neural network model to calculate the equation parameters of the geometric curve corresponding to the virtual track. The vehicle maintains a constant speed and follows this geometric curve until the next time the camera captures an image. First, the deviation between the vehicle control center and the virtual track, lane_offset, is calculated at the next camera capture time, and the offset_value is calculated. Then, the number of times the wheels cross the line during this time period is counted, assuming it is n times (n>=0), and the negative reward for the wheels crossing the line is calculated. The formula for calculating the total reward during this period is:
[0089] Reward = offset_value + n * (-cross_lane_value) < 0 (2)
[0090] The time interval between each two image captures is defined as one round. The weight coefficients of each node in the deep reinforcement learning neural network model are updated through backpropagation based on the reward score obtained in each round.
[0091] It is clear that when calculating the negative reward for a vehicle's control center deviating from the lane center, in addition to using a linear proportional relationship, a non-linear, higher-order curve relationship can also be used. For example, refer to... Figure 4 As shown, a nonlinear quadratic curve relationship can be used for calculation. In this nonlinear high-order curve, when the vehicle control center is located at the center of the lane, the slope of the high-order curve is 0. When the vehicle control center is close to the lane center, the change in vehicle position corresponds to a relatively small change in the score, and the high-order curve is gentle with a small slope in this local range. When the vehicle is close to the lane line at the edge of the lane, the change in vehicle position corresponds to a relatively large change in the score, and the high-order curve is steep with a large slope in this local range. Using a nonlinear high-order curve to calculate the offset_value is beneficial for improving the efficiency of autonomous learning and reducing the back-and-forth oscillation of the vehicle control center near the lane center during driving.
[0092] S105: Until the reward meets the preset conditions, input the image of the ground road in front of the vehicle into the deep reinforcement learning neural network model, and identify the virtual track through the deep reinforcement learning neural network model.
[0093] Once the reward meets the preset conditions, it indicates that the deep reinforcement learning neural network model has completed autonomous learning. After that, the image of the ground road in front of the vehicle is input into the deep reinforcement learning neural network model. The equation parameters of the geometric curve output by the deep reinforcement learning neural network model correspond to the virtual track, thereby realizing the recognition of the virtual track.
[0094] In some embodiments, the step of waiting until the reward meets a preset condition includes:
[0095] The reward will continue to exceed the preset threshold for a predetermined number of consecutive preset number of times.
[0096] If the reward can be continuously greater than the preset threshold, the deep reinforcement learning network model is considered to have completed autonomous learning. After that, the captured image of the road in front of the vehicle is input into the deep reinforcement learning neural network model. The equation parameters of the geometric curve output by the deep reinforcement learning neural network model correspond to the virtual track, thereby realizing the recognition of the virtual track.
[0097] Furthermore, the deep reinforcement learning neural network model output network structure can also output the three-dimensional coordinates of k points located on the virtual track. For example, 3×k floating-point numbers represent the three-dimensional coordinates (x, y, z) of k points in the vehicle coordinate system. The k points can be distributed at equal intervals along the X-axis on the center line, or they can be distributed at equal time intervals T along the X-axis. In this case, the formula for calculating the distribution spacing of the k points is formula (3):
[0098] Dist = Vt × T (3)
[0099] In formula (3), Vt is the vehicle's current instantaneous speed.
[0100] The vehicle lateral control system performs lateral control of the vehicle based on the output of a deep reinforcement learning neural network model.
[0101] In summary, the virtual track recognition method provided in this application adopts an unsupervised autonomous learning approach. This process requires no manual intervention or preparation of large sample datasets, significantly reducing human and material costs. Furthermore, this virtual track recognition method decouples control from perception; virtual track recognition is independent of vehicle control. The vehicle's lateral control system controls the vehicle based on the perception results. After autonomous learning, it can adapt to vehicles with different control characteristics without needing to relearn on vehicles with different control features, thus enhancing its applicability.
[0102] This application also provides a virtual track recognition device, which is described below and can be referred to in conjunction with the method described above. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of an apparatus provided in an embodiment of this application, in conjunction with... Figure 5 As shown, the device includes:
[0103] The shooting module 10 is used to capture images of the ground road in front of the vehicle in simulated or real road scenarios;
[0104] Input module 20 is used to input the ground road image in front of the vehicle into the deep reinforcement learning neural network model, and the deep reinforcement learning neural network model outputs the equation parameters of the geometric curve corresponding to the virtual track.
[0105] Output module 30 is used to output the equation parameters to the vehicle lateral control system so that the vehicle lateral control system can perform vehicle lateral control according to the equation parameters;
[0106] The update module 40 is used to determine a reward based on the vehicle lateral control result of the vehicle lateral control system, and update the deep reinforcement learning neural network model based on the reward.
[0107] The recognition module 50 is used to input the image of the ground road in front of the vehicle into the deep reinforcement learning neural network model until the reward meets the preset conditions, and to recognize the virtual track through the deep reinforcement learning neural network model.
[0108] Based on the above embodiments, as a specific implementation method, the updating module 40 is specifically used for:
[0109] The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track, as well as the number of times the wheels cross the lane lines.
[0110] Based on the above embodiments, as a specific implementation method, the updating module 40 includes:
[0111] The first calculation unit is used to calculate the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives;
[0112] The statistics unit is used to count the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment;
[0113] The second calculation unit is used to calculate the reward based on the deviation between the target reference point on the vehicle and the virtual track when the next shooting time is reached, and the number of times the wheels cross the lane line between the next shooting time and the previous shooting time.
[0114] Based on the above embodiments, as a specific implementation method, the second computing unit is specifically used for:
[0115] The negative reward at the next shooting moment is calculated based on offset_value = [lane_offset × (-cross_lane_value)] / (0.5 ×lane_width);
[0116] The reward is calculated based on Reward = offset_value + n * (-cross_lane_value);
[0117] Where offset_value represents the negative reward, lane_offset represents the deviation between the target reference point on the vehicle and the virtual track when the next shooting time is reached, -cross_lane_value represents the reward when any wheel crosses the lane line, cross_lane_value is a positive value, lane_width represents the lane width, n represents the number of times the wheel crosses the lane line between the next shooting time and the previous shooting time, and Reward represents the reward.
[0118] Based on the above embodiments, as a specific implementation method, the step of waiting until the reward meets the preset conditions includes:
[0119] The reward will continue to exceed the preset threshold for a predetermined number of consecutive preset number of times.
[0120] Based on the above embodiments, as a specific implementation method, the input module 20 is specifically used for:
[0121] Multiple adjacent frames of road images in front of the vehicle are input into the deep reinforcement learning neural network model.
[0122] Based on the above embodiments, as a specific implementation method, multiple adjacent frames of road images in front of the vehicle are captured by the same shooting device; or, multiple adjacent frames of road images in front of the vehicle are captured by multiple shooting devices.
[0123] Based on the above embodiments, as a specific implementation method, it further includes:
[0124] The preprocessing module is used to preprocess the ground road image in front of the vehicle before inputting it into the deep reinforcement learning neural network model, removing invalid image areas and retaining the road surface area.
[0125] Based on the above embodiments, as a specific implementation method, it further includes:
[0126] The conversion module is used to convert the RGB format image of the road in front of the vehicle into YUV format before inputting it into the deep reinforcement learning neural network model.
[0127] In some embodiments, the deep reinforcement learning neural network model outputs four floating-point numbers, which are the coefficients of the cubic, quadratic, and linear terms of the geometric curve corresponding to the virtual track, as well as the constant term value.
[0128] This application also provides a virtual track recognition device, for reference... Figure 6 As shown, the device includes a memory 1 and a processor 2.
[0129] Memory 1 is used to store computer programs;
[0130] Processor 2 is used to execute computer programs to perform the following steps:
[0131] In a simulated or real road scenario, an image of the road surface in front of the vehicle is captured. This image is then input into a deep reinforcement learning neural network model, which outputs the equation parameters of the geometric curve corresponding to the virtual track. These equation parameters are then output to the vehicle's lateral control system, enabling the system to perform lateral control based on them. A reward is determined based on the lateral control result, and the deep reinforcement learning neural network model is updated accordingly. Once the reward meets a preset condition, the image of the road surface in front of the vehicle is input into the deep reinforcement learning neural network model, which then identifies the virtual track.
[0132] For a description of the equipment provided in this application, please refer to the above method embodiments; further details will not be provided here.
[0133] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the following steps:
[0134] In a simulated or real road scenario, an image of the road surface in front of the vehicle is captured. This image is then input into a deep reinforcement learning neural network model, which outputs the equation parameters of the geometric curve corresponding to the virtual track. These equation parameters are then output to the vehicle's lateral control system, enabling the system to perform lateral control based on them. A reward is determined based on the lateral control result, and the deep reinforcement learning neural network model is updated accordingly. Once the reward meets a preset condition, the image of the road surface in front of the vehicle is input into the deep reinforcement learning neural network model, which then identifies the virtual track.
[0135] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0136] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.
[0137] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, performs the following steps:
[0138] In a simulated or real road scenario, an image of the road surface in front of the vehicle is captured. This image is then input into a deep reinforcement learning neural network model, which outputs the equation parameters of the geometric curve corresponding to the virtual track. These equation parameters are then output to the vehicle's lateral control system, enabling the system to perform lateral control based on them. A reward is determined based on the lateral control result, and the deep reinforcement learning neural network model is updated accordingly. Once the reward meets a preset condition, the image of the road surface in front of the vehicle is input into the deep reinforcement learning neural network model, which then identifies the virtual track.
[0139] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses, devices, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant details can be found in the method section.
[0140] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0141] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0142] The virtual track recognition method, apparatus, device, medium, and computer program product provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.
Claims
1. A virtual track recognition method, characterized in that, include: Capture images of the road surface in front of the vehicle in simulated or real road scenarios; The image of the road in front of the vehicle is input into a deep reinforcement learning neural network model, and the deep reinforcement learning neural network model outputs the equation parameters of the geometric curve corresponding to the virtual track. The equation parameters are output to the vehicle lateral control system so that the vehicle lateral control system performs vehicle lateral control based on the equation parameters. The reward is determined based on the vehicle lateral control result of the vehicle lateral control system, and the deep reinforcement learning neural network model is updated based on the reward. Once the reward meets the preset conditions, the image of the road in front of the vehicle is input into the deep reinforcement learning neural network model, and the virtual track is identified through the deep reinforcement learning neural network model.
2. The virtual track recognition method according to claim 1, characterized in that, The determination of rewards based on the vehicle lateral control results of the vehicle lateral control system includes: The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track, as well as the number of times the wheels cross the lane lines.
3. The virtual track recognition method according to claim 2, characterized in that, The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track, as well as the number of times the wheels cross the lane lines. Calculate the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives; Count the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment; The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives, and the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment.
4. The virtual track recognition method according to claim 3, characterized in that, The reward is calculated based on the deviation between the target reference point on the vehicle and the virtual track when the next shooting moment arrives, and the number of times the wheels cross the lane lines between the next shooting moment and the previous shooting moment. The negative reward at the next shooting moment is calculated using offset_value = [lane_offset × (-cross_lane_value)] / (0.5 × lane_width); The reward is calculated based on Reward = offset_value + n * (-cross_lane_value); Where offset_value represents the negative reward, lane_offset represents the deviation between the target reference point on the vehicle and the virtual track when the next shooting time is reached, -cross_lane_value represents the reward when any wheel crosses the lane line, cross_lane_value is a positive value, lane_width represents the lane width, n represents the number of times the wheel crosses the lane line between the next shooting time and the previous shooting time, and Reward represents the reward.
5. The virtual track recognition method according to claim 1, characterized in that, The condition until the reward meets the preset conditions includes: The reward will continue to exceed the preset threshold for a predetermined number of consecutive preset number of times.
6. The virtual track recognition method according to claim 1, characterized in that, Inputting the image of the road surface in front of the vehicle into a deep reinforcement learning neural network model includes: Multiple adjacent frames of road images in front of the vehicle are input into the deep reinforcement learning neural network model.
7. The virtual track recognition method according to claim 6, characterized in that, Multiple adjacent images of the road surface in front of the vehicle are captured by the same camera; or, multiple adjacent images of the road surface in front of the vehicle are captured by multiple cameras.
8. The virtual track recognition method according to claim 1, characterized in that, Before inputting the image of the road surface in front of the vehicle into the deep reinforcement learning neural network model, the following steps are also included: The image of the road surface in front of the vehicle is preprocessed to remove invalid image areas and retain the road surface area.
9. The virtual track recognition method according to claim 1, characterized in that, Before inputting the image of the road surface in front of the vehicle into the deep reinforcement learning neural network model, the following steps are also included: Convert the RGB format image of the road in front of the vehicle into YUV format.
10. The virtual track recognition method according to claim 1, characterized in that, The deep reinforcement learning neural network model outputs four floating-point numbers, which are the coefficients of the cubic, quadratic, and linear terms of the geometric curve corresponding to the virtual track, as well as the constant term.
11. A virtual track recognition device, characterized in that, include: The camera module is used to capture images of the road surface in front of the vehicle in simulated or real road scenarios. The input module is used to input the image of the ground road in front of the vehicle into the deep reinforcement learning neural network model, and the deep reinforcement learning neural network model outputs the equation parameters of the geometric curve corresponding to the virtual track. An output module is used to output the equation parameters to the vehicle lateral control system, so that the vehicle lateral control system can perform vehicle lateral control according to the equation parameters. An update module is used to determine a reward based on the vehicle lateral control result of the vehicle lateral control system, and update the deep reinforcement learning neural network model based on the reward. The recognition module is used to input the image of the ground road in front of the vehicle into the deep reinforcement learning neural network model until the reward meets the preset conditions, and then recognize the virtual track through the deep reinforcement learning neural network model.
12. A virtual track recognition device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the virtual track recognition method as described in any one of claims 1 to 10 when executing the computer program.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the virtual track identification method as described in any one of claims 1 to 10.
14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the virtual track recognition method as described in any one of claims 1 to 10.