Path planning method, vehicle driving control method, related device and vehicle

Abstract

The invention relates to a path planning method, a vehicle driving control method, a related device and a vehicle, and the method comprises the steps: recognizing a guide line in a driving region based on a ground image of the driving region; generating key points based on the guide line; selecting a target point from the key points; and determining path planning information according to the target point. According to the method, the guide line is arranged on the ground, the ground image is collected in real time to recognize the guide line, then the vehicle is guided to run along the preset guide line based on the key point of the guide line, and finally accurate control over running of the vehicle along the guide line is achieved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a path planning method, a vehicle driving control method, related devices, and a vehicle. Background Technology

[0002] In recent years, the automotive industry has developed rapidly, placing higher demands on the level of intelligence in driving. Related technologies mainly employ proportional-integral or lookup table methods to keep the vehicle on a preset line (e.g., lane center line), but this method relies heavily on experience and still suffers from insufficient control precision. Summary of the Invention

[0003] This application provides a vehicle driving control method, related devices, and a vehicle to solve the above-mentioned problems.

[0004] To achieve the above objectives, according to a first aspect of this application, a path planning method is provided, the method comprising:

[0005] Based on ground images of the driving area, identify guide lines within the driving area;

[0006] Key points are generated based on the guide lines;

[0007] Select the target point from the key points;

[0008] Path planning information is determined based on the target point.

[0009] Optionally, generating key points based on the guide lines includes:

[0010] The guide line is discretized to obtain multiple key points on the guide line.

[0011] Optionally, selecting a target point from the key points and determining path planning information based on the target point includes:

[0012] Select the key point closest to the current location from among the multiple key points as the target point;

[0013] Based on the target point and the current location, the path planning information is determined.

[0014] Optionally, identifying guide lines within the driving area based on ground images of the driving area includes:

[0015] Based on the ground image, the detection box of the guide line is determined using a target detection model.

[0016] Optionally, determining the detection box of the guide line based on the ground image using a target detection model includes:

[0017] Based on the target detection model, multiple preset anchor boxes are generated for the ground image;

[0018] Regression is performed on multiple preset anchor frames to obtain the detection frame of the guide line.

[0019] Optionally, generating multiple preset anchor boxes for the ground image based on the target detection model includes:

[0020] Based on the preset parameters of the target detection model, multiple preset anchor frames are generated for the ground image. The preset parameters include at least one of the initial position, anchor frame angle, and anchor frame length.

[0021] Optionally, determining the detection box of the guide line based on the ground image using a target detection model includes:

[0022] The ground image is downsampled to obtain a feature image of the ground image;

[0023] Based on the feature image, the detection box of the guide line is determined using the target detection model.

[0024] Optionally, determining the detection box of the guide line based on the feature image and the target detection model includes:

[0025] Based on the feature image, the detection box of the guide line in the feature image is obtained through the target detection model;

[0026] The detection box in the feature image is projected onto the ground image to obtain the detection box of the guide line in the ground image.

[0027] Optionally, projecting the detection box in the feature image onto the ground image to obtain the detection box of the guide line in the ground image includes:

[0028] The detection bounding box in the ground image is obtained based on the size information of the detection bounding box in the feature image and the ground image.

[0029] Optionally, the target detection model is trained through the following steps:

[0030] The sample ground image is input into the initial model to obtain multiple anchor frame regions of the sample ground image, wherein the anchor frame regions are candidate regions that may contain guide lines;

[0031] Based on the multiple anchor frame regions, an attention matrix is ​​obtained, which is used to describe the association strength between different anchor frame regions;

[0032] Based on the attention matrix, the feature vector corresponding to each anchor box region, and the labeled data of the sample ground image, the loss value of the initial model is determined;

[0033] The initial model is trained based on the loss value to obtain the target detection model.

[0034] According to a second aspect of this application, embodiments of this application also provide a vehicle driving control method, the method comprising:

[0035] The path planning information is determined according to the aforementioned method;

[0036] Based on the route planning information and vehicle status parameters, determine the driving parameters;

[0037] The vehicle is controlled to drive according to the driving parameters.

[0038] Optionally, the driving parameters include steering angle and / or driving speed.

[0039] According to a third aspect of this application, embodiments of this application also provide a vehicle driving control system, the system including a vision module and a control module, wherein,

[0040] The vision module is used to acquire ground images of the driving area;

[0041] The control module is used to control vehicle movement according to any of the vehicle movement control methods described above.

[0042] According to a fourth aspect of this application, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the path planning methods provided in embodiments of this application.

[0043] According to a fifth aspect of this application, embodiments of this application also provide an electronic device, comprising:

[0044] A memory on which computer programs are stored;

[0045] A processor is configured to execute the computer program in the memory to implement any of the path planning methods provided in the embodiments of this application.

[0046] According to a sixth aspect of this application, embodiments of this application also provide a vehicle, including the aforementioned electronic device, or the aforementioned vehicle driving control system and chassis, wherein the chassis is adapted to adjust the vehicle driving state according to driving parameters.

[0047] Some embodiments in this specification include at least the following beneficial effects: By setting guide lines on the ground and acquiring ground images in real time to identify the guide lines, and then guiding the vehicle along the preset guide lines based on key points of the guide lines, it is beneficial to accurately control the vehicle's movement along the guide lines. Compared with traditional magnetic nail or magnetic strip guidance systems, since the guide lines are sprayed or pasted, there is no need to pre-bury or lay them on the ground, avoiding irreversible damage to the ground. Moreover, the construction and maintenance of the guide lines are more convenient, requiring no professional equipment or personnel, thus reducing the long-term operating costs of the system. At the same time, by acquiring pixel data of the ground guide lines in real time through a camera, the high sampling frequency can provide high-precision positioning information in real time, ensuring that the vehicle can accurately track even in complex road conditions.

[0048] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0051] Figure 1 These are application scenario diagrams of a vehicle driving control system based on some embodiments of this specification;

[0052] Figure 2 This is an exemplary flowchart of a path planning method according to some embodiments of this specification;

[0053] Figure 3 This is an exemplary flowchart of a detection frame for determining a guide line, as shown in some embodiments of this specification;

[0054] Figure 4 These are exemplary schematic diagrams of guide lines shown according to some embodiments of this specification;

[0055] Figure 5 This is an exemplary flowchart of a vehicle driving control method according to some embodiments of this specification;

[0056] Figure 6 This is a schematic diagram of the structure of a vehicle driving control system according to some embodiments of this specification;

[0057] Figure 7This is a schematic diagram of the structure of an electronic device according to some embodiments of this specification;

[0058] Figure 8 This is an exemplary schematic diagram of a vehicle according to some embodiments of this specification. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0060] To facilitate understanding of the implementation schemes provided in this application, the relevant application background of the path planning method and vehicle driving control method provided in this application will be explained first.

[0061] Currently, existing vehicle guidance technologies largely rely on complex sensor systems or high-precision map data, which are costly and poorly adaptable to different environments. For example, some methods for guiding autonomous vehicles to bus stops require high-precision positioning systems and complex algorithms to plan trajectories, but these technologies face problems such as high costs and susceptibility to environmental interference in practical applications. Meanwhile, traditional bus stops use magnetic nails or strips as guidance paths, requiring pre-embedding or paving of the ground, causing irreversible damage.

[0062] Therefore, some embodiments of this specification provide a path planning method and a vehicle driving control method. By acquiring ground images in real time to identify guide lines, and then guiding the vehicle along the preset guide lines based on key points of the guide lines, the method ultimately achieves precise control of the vehicle's movement along the guide lines. Furthermore, the deep learning-based guide line perception algorithm can adapt to different lighting conditions and complex road conditions, improving the robustness and reliability of the system.

[0063] Figure 1 These are application scenario diagrams of a vehicle driving control system based on some embodiments of this specification.

[0064] like Figure 1 As shown, embodiments of this application provide a vehicle driving control method, system, electronic device, vehicle, and storage medium. The vehicle driving control system can be integrated into an electronic device, which can be a server, a terminal, or other similar device.

[0065] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN) acceleration services, and big data and artificial intelligence platforms. The terminal can include, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0066] The electronic device may be an in-vehicle terminal integrated into the vehicle, or a device that interacts with the vehicle for data exchange. The vehicle may be a gasoline-powered vehicle, a plug-in hybrid electric vehicle, or a new energy vehicle, etc. This application does not make any specific limitations on this.

[0067] Please see Figure 1 Taking the integration of vehicle driving control systems into electronic devices as an example, Figure 1 This is a schematic diagram of an implementation scenario of the vehicle driving control system provided in this application embodiment. The electronic device can be a server or a terminal. The electronic device can identify guide lines in the driving area based on the ground image of the vehicle in the driving area; generate a planned path using key points on the guide lines as target points; and control the vehicle to drive according to the guide lines based on the planned path.

[0068] It should be noted that, Figure 1 The schematic diagram illustrating the implementation environment of the vehicle driving control system is merely an example. The implementation environment of the vehicle driving control system described in this application is intended to more clearly illustrate the technical solutions of the embodiments of this application and does not constitute a limitation on the technical solutions provided in the embodiments of this application. Those skilled in the art will understand that, with the evolution of vehicle driving control and the emergence of new business scenarios, the technical solutions provided in this application are equally applicable to similar technical problems.

[0069] The solutions provided in this application are specifically illustrated through the following embodiments. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0070] Figure 2 This is an exemplary flowchart of a path planning method according to some embodiments of this specification. In some embodiments, process 200 may be executed based on an electronic device. Figure 2 As shown, process 200 includes the following steps.

[0071] Step 210: Identify guide lines within the driving area based on the ground image of the driving area.

[0072] The operating area refers to the specific area in a public transportation system where passengers can get on and off the bus.

[0073] In some embodiments, electronic devices may locate and determine the travel area (e.g., the location and extent of a station) based on map data in a geographic information system.

[0074] Ground images refer to photographs or video frames of the ground surface in the driving area.

[0075] In some embodiments, the electronic device can communicate with an image acquisition device installed on the vehicle. After confirming that the vehicle has entered or is about to enter the driving area, the electronic device can obtain ground images of the driving area in real time and periodically based on the image acquisition device.

[0076] An image acquisition device is a device capable of acquiring image information. Exemplary image acquisition devices may include portable digital cameras, webcams, etc. In some embodiments, the image acquisition device can perform image acquisition operations in response to an acquisition command sent by an electronic device. For example, when the image acquisition device receives an acquisition command sent by an electronic device, it acquires a ground image.

[0077] In some embodiments, the image acquisition device can be installed inside or outside the vehicle's windshield. A lateral position along the center symmetry line of the windshield allows for better scanning of the guide line, measurement of the lateral distance between the vehicle's centerline and the guide line, and transmission of signals to electronic devices to control the electro-hydraulic steering system, ensuring the vehicle's longitudinal symmetry plane coincides with the guide line. The image acquisition device can also be installed at the top of the windshield for a better field of view and to perform a large-scale scan of the guide line in advance; the installation height can also be selected based on specific circumstances.

[0078] In some embodiments, the electronic device can continuously receive the vehicle's current location information based on the positioning device installed on the vehicle, and compare the current location information with the geographical coordinates of the platform to determine whether the vehicle has entered or is about to enter the driving area.

[0079] Guide lines are ground markings that are drawn or set in advance within a driving area to guide vehicles to travel along a predetermined route.

[0080] In some embodiments, the guide line can be a straight line, a curve, or a line with special markings. The guide line can be located between the wheels, on the left or right side of the vehicle's direction of travel.

[0081] In some embodiments, the guide lines may have specific colors (such as white, yellow) and shapes (such as straight lines, curves, dashed lines) so that the image acquisition devices on the vehicle can easily identify them.

[0082] In some embodiments, guide lines within a driving area can be identified based on ground images in various ways. For example, guide lines can be extracted from the ground image based on edge detection algorithms (such as the Canny algorithm). Another example is that a threshold can be set according to the color range of the guide lines, and pixels conforming to that color range can be extracted as guide lines. Yet another example is that guide lines can also be identified using other image processing methods and deep learning methods.

[0083] Step 220: Generate key points based on guide lines.

[0084] Key points are specific locations on a guideline, such as the start point, end point, and turning points. Key points are helpful for path planning and vehicle control.

[0085] In some embodiments, sampling can be performed based on guide lines (e.g., uniform sampling) or by using feature point extraction methods (e.g., Harris corner detection) to obtain multiple key points.

[0086] Step 230: Select the target point from the key points.

[0087] The target point is a key point used to control the movement of the vehicle.

[0088] In some embodiments, a target point can be obtained from multiple key points based on a random selection method or a preset selection rule (e.g., selection based on a preset point interval).

[0089] Step 240: Determine the path planning information based on the target point.

[0090] Path planning information refers to information related to the vehicle's driving path generated based on the target point.

[0091] In some embodiments, the optimal path from the first target point to the last target point can be calculated based on a path planning algorithm (such as Dijkstra's algorithm, RRT (Rapidly-exploring Random Tree) algorithm, etc.). Alternatively, the connection between multiple key points can be used as the path planning information.

[0092] Controlling vehicle movement refers to adjusting the vehicle's direction, speed, and position according to the planned path information through the vehicle control system (such as the steering system, braking system, and power system) so that the vehicle can travel along the guide line.

[0093] It should be noted that guide lines are actual markings drawn on the ground to provide a reference trajectory for the vehicle, helping it determine its direction and position. The planned path information is a driving path dynamically generated by the vehicle control system based on key points on the guide lines or other navigation information (such as the vehicle's position, speed, and environmental information (such as obstacles)).

[0094] In some embodiments of this specification, by identifying key points on the guide line and generating planned path information, vehicles can achieve high-precision parking; in dynamic environments, such as intersections or driving areas, the planned path information can be adjusted according to real-time environmental information to ensure safe passage of vehicles.

[0095] In some embodiments, generating key points based on guide lines includes:

[0096] Discretize the guide line to obtain multiple key points on the guide line.

[0097] In some embodiments, the identified continuous guide lines can be segmented into multiple discrete key points using uniform sampling, curvature-based sampling, or other sampling methods.

[0098] In some embodiments, planned path information can be generated sequentially using each key point as a target point, so as to control the vehicle to travel along the guide line based on multiple planned path information. For example, when the vehicle reaches the current target point, the next discretized key point is selected as the new target point, and the above process is repeated until the vehicle has passed all the key points.

[0099] In some embodiments of this specification, by discretizing the guide lines and generating multiple key points, the vehicle can perform precise path planning and tracking at each key point, ensuring that the vehicle always travels along the predetermined path; this helps to dynamically adjust the planned path information. For example, when the vehicle discovers a deviation or encounters an obstacle during driving, the planned path information can be recalculated based on real-time data, ensuring the safety and accuracy of the vehicle.

[0100] In some embodiments, a target point is selected from the key points; path planning information is determined based on the target point, including:

[0101] Select the key point closest to the current location from multiple key points as the target point;

[0102] Based on the target point and the current location, determine the path planning information.

[0103] The current location point refers to the precise spatial position of the vehicle at the current moment. In some embodiments, the current location point can be represented by coordinates in a coordinate system (such as latitude and longitude in a geodetic coordinate system, or coordinates in a Cartesian coordinate system).

[0104] The current time can be any time, depending on when the electronic device processes the relevant instructions.

[0105] In some embodiments, the electronic device may communicate with a positioning device on the vehicle (e.g., a global positioning system, an inertial measurement unit, etc.) to obtain the vehicle's current location in real time and periodically.

[0106] In some embodiments, the distance between the vehicle's current location and all key points on the guide line can be calculated, and the key point with the smallest distance can be selected as the target point.

[0107] In some embodiments of this specification, by selecting the key point closest to the vehicle's current location as the target point, the vehicle can travel along the guide line more in real time and accurately, thereby improving the real-time performance and accuracy of path planning.

[0108] In some embodiments, the current position of the vehicle can be selected as the starting point, and the nearest key point to the current position can be selected as the target point at the current moment. The optimal path from the current position to the target point can be generated using a path planning algorithm (such as A*, Dijkstra, RRT, etc.). Based on the generated planned path information, the vehicle can be controlled by the vehicle control system (such as the chassis) to achieve path tracking (such as steering, acceleration, deceleration, etc.).

[0109] Figure 4 This is an exemplary schematic diagram of a guide line according to some embodiments of this specification.

[0110] As shown in the figure, electronic devices can control vehicles to automatically enter and stop at stations based on identified guide lines.

[0111] In some embodiments, when controlling the vehicle to automatically enter a parking area, the electronic equipment simultaneously monitors the surrounding environment for obstacles. For example, the electronic equipment can use radar devices installed on the vehicle, such as ultrasonic radar or infrared radar. The ultrasonic radar around the vehicle continuously monitors the environment; when an obstacle is detected, it immediately measures and calculates the distance between the obstacle and the vehicle, and sends this distance to the electronic equipment. If the electronic equipment determines that the distance is less than a preset safety threshold (which can be flexibly set according to actual conditions), it will trigger a safety alert mechanism, such as a buzzer sounding or a warning light flashing, prompting the driver to intervene quickly and prevent accidents.

[0112] In some embodiments, the method further includes:

[0113] If the vehicle's current location is not a platform parking point, continue with the step of identifying guide lines within the driving area based on the ground image of the vehicle in the driving area.

[0114] A platform parking spot refers to the specific location where a vehicle needs to stop within the operating area.

[0115] In some embodiments, the operation of inputting ground images into a target detection model can be performed based on at least one iteration. The at least one iteration includes: processing the ground images based on the target detection model, identifying the guide line of the current iteration, selecting the nearest key point on the guide line as the target point to generate planned path information, and controlling the vehicle to drive according to the guide line based on the planned path information.

[0116] In some embodiments, the input to the object detection model is related to the iteration round. During the first iteration, the input to the object detection model includes a first ground image captured upon receiving the entry command; during subsequent iterations, the input to the object detection model includes a second ground image. The second ground image refers to a ground image captured in real-time after the first ground image.

[0117] In some embodiments, in at least one iteration (excluding the last iteration), the output of the object detection model is the detection box of the guide line for the current iteration. In the last iteration, the output of the object detection model is a notification that no guide line was detected.

[0118] In some embodiments, in the first iteration, the first ground image can be processed using a target detection model to generate the detection box of the guide line for the first iteration, so as to obtain the iteration result of the first iteration (including the detection box of the guide line for the first iteration and the corresponding vehicle driving state). Based on the corresponding vehicle driving state of the first iteration, it can be determined whether the preset conditions are met. In response to the vehicle driving state corresponding to the current round being that it has not reached the platform parking point, the next iteration is continued based on the second ground image.

[0119] In some embodiments, in at least one iteration (excluding the last iteration), the target detection model processes the second ground image to generate a detection box for the guide line of the current iteration, selects the nearest key point on the guide line as the target point to generate planned path information, and controls the vehicle to drive according to the guide line based on the planned path information to obtain the iteration result of the current iteration (including the detection box of the guide line of the current iteration and the corresponding vehicle driving state); it can be determined whether a preset condition is met based on the vehicle driving state corresponding to the current iteration; in response to the vehicle driving state corresponding to the current iteration not reaching the platform stopping point, the second ground image is obtained, and the next iteration continues. In the last iteration, it can be determined whether a preset condition is met based on the vehicle driving state corresponding to the last iteration; in response to the vehicle driving state corresponding to the last iteration reaching the platform stopping point, a termination prompt signal is issued, and the iteration is terminated.

[0120] Preset conditions are the criteria used to determine whether an evaluation iteration should terminate. For example, preset conditions may include whether the vehicle has arrived at the platform parking point.

[0121] In some embodiments of this specification, by traversing each key point, the vehicle can accurately follow the guide line, ensuring that the vehicle stops precisely at the preset position on the platform, which helps passengers get on and off the vehicle and improves the passenger experience.

[0122] In some embodiments, the method further includes:

[0123] Upon detecting an entry command, the ground image acquired by the image acquisition device is identified to obtain the guide line within the driving area.

[0124] Inbound instructions are commands used in automated driving or intelligent transportation systems to assist vehicles in entering and exiting platforms. These instructions can be generated by the vehicle control system (e.g., the vehicle control unit (VCU)) or sent to the vehicle by an external system (e.g., a platform intelligent control system).

[0125] In some embodiments, the vehicle control system automatically generates a stop instruction based on a preset driving route and the vehicle's current position. For example, while the bus is in motion, an image acquisition device continuously monitors the ground image in the direction of the bus's movement. When a starting sign (e.g., an icon with preset lettering) is detected, the system automatically controls the vehicle, or, based on user input, controls the vehicle to enter an automatic parking mode.

[0126] In some embodiments, during the automatic entry of the vehicle into the station according to the station guide line, if driver operation is detected, such as the driver touching the steering wheel or the vehicle speed falling below a preset speed range, the vehicle is controlled to exit the automatic parking mode. The vehicle will then proceed according to the driver's operation (i.e., decelerate to the platform parking point). It should be noted that, in the above embodiments, when the vehicle enters the automatic parking mode, the vehicle speed can also be controlled by the driver by controlling the accelerator and brake pedals.

[0127] In some embodiments of this specification, precise entry instructions and automatic control reduce human error and improve the safety of vehicle entry; moreover, automated entry processes can reduce parking time, improve operational efficiency, and automated entry and passenger boarding / alighting can provide a more convenient passenger experience.

[0128] Figure 3 This is an exemplary flowchart of a detection block for determining a guide line according to some embodiments of this specification. In some embodiments, process 300 may be performed based on an electronic device. Figure 3 As shown, process 300 includes the following steps.

[0129] In some embodiments, identifying guide lines within the driving area based on ground images of the vehicle in the driving area includes:

[0130] Step 310: Based on the ground image, determine the detection box of the guide line using the target detection model.

[0131] A detection box is used to identify the position and size of the detected guide lines in an image. In some embodiments, the detection box may be a line-shaped detection box.

[0132] In some embodiments, the object detection model can output information related to the detection boxes of guide lines. For example, the location of the detection box in the ground image, which can be represented by a set of coordinates, such as the coordinates of the top-left and bottom-right corners of the detection box, or by using the center point coordinates plus width and height. Another example is the size of the detection box, i.e., its width and height, reflecting the actual size of the detected object in the ground image. Yet another example is the category label associated with each detection box, indicating the category to which the object contained within the detection box belongs (e.g., "guide line," "pedestrian," "car," etc.). Yet another example is a confidence score, a value between 0 and 1, representing the model's confidence in the detection result. A high score indicates a high probability that the object detection model's detection result is correct, while a low score indicates a low probability that the object detection model's detection result is correct.

[0133] In some embodiments, when identifying guide lines in a ground image, the ground image needs to be preprocessed. This preprocessing includes grayscale conversion, median filtering, and contour finding of the ground image before identifying guide lines in the ground image or image frame stream.

[0134] An object detection model is a mathematical or computational model used to identify guide lines from ground images.

[0135] In some embodiments, the object detection model is a machine learning model. For example, the object detection model may include any one or a combination of Convolutional Neural Networks (CNN) models, Neural Networks (NN) models, or other custom model structures.

[0136] In some embodiments, the input to the target detection model may include a ground image, and the output may include a detection bounding box with guide lines.

[0137] In some embodiments, the object detection model can be trained using a large number of labeled training samples through various feasible methods. For example, parameters can be updated using gradient descent. An exemplary training process includes: inputting multiple training samples labeled with the object detection model into an initial object detection model; constructing a loss function using the labels and the results of the initial object detection model; and iteratively updating the parameters of the initial object detection model based on the loss function using gradient descent or other methods. The model training is complete when preset conditions are met, resulting in a trained object detection model. These preset conditions may include loss function convergence, the number of iterations reaching a threshold, etc.

[0138] In some embodiments, the training samples include at least sample ground images. The training samples may be obtained based on historical data.

[0139] In some embodiments, the label may include a detection bounding box corresponding to a guide line in the sample ground image. The label can be obtained through a processor or manual annotation.

[0140] In some embodiments of this specification, the target detection model can more accurately identify and locate guide lines, maintaining high recognition accuracy even in complex or changing environments. For autonomous vehicles or automatic navigation systems, accurate detection of guide lines can adjust the driving path in a timely manner, avoid deviation from the predetermined route, and help to accurately reach the preset position of the platform.

[0141] In some embodiments, the detection bounding box of the guide line is determined based on a ground image using a target detection model, including:

[0142] Based on the target detection model, multiple preset anchor boxes are generated for the ground image;

[0143] Regression is performed on multiple preset anchor frames to obtain the detection frame of the guide line.

[0144] A preset anchor box is a predefined bounding box used to represent the position and size of guide lines that may exist in a ground image.

[0145] Preset anchor frames are used to generate candidate regions to estimate the probability of the presence and location of the guide lines.

[0146] In some embodiments, a series of convolutions can be performed on the ground image to output one or more feature images. Each point on the feature image represents a region on the input ground image, and its receptive field corresponds to a location in the original ground image. For each point on the feature image, the width and height of a preset anchor box are determined according to a set size and aspect ratio, generating a set of preset anchor boxes. Each preset anchor box corresponds to a specific center point (usually the center of a region in the original ground image corresponding to that pixel), and each preset anchor box is generated with a different size and aspect ratio.

[0147] In some embodiments, the preset anchor box can be represented by four values, including: center point coordinates (cx, cy) and width and height (w, h). For each preset anchor box and its corresponding real detection box, the regression is performed on the offset of the center point coordinates of the preset anchor box to the real detection box and the scaling ratio of the width and height. The position and size of the preset anchor box can be adjusted by the offset and the scaling ratio of the width and height to more closely approximate the real detection box.

[0148] In some embodiments of this specification, a preset anchor box is generated by a target detection model, and a regression operation is performed on the preset anchor box to obtain an accurate detection box for the guide line, which improves the accuracy of guide line recognition and helps with vehicle path planning and control.

[0149] In some embodiments, based on an object detection model, multiple preset anchor boxes are generated from the ground image, including:

[0150] Based on the preset parameters of the target detection model, multiple preset anchor boxes are generated for the ground image. The preset parameters include at least one of the initial position, anchor box angle, and anchor box length.

[0151] Preset parameters are a series of parameters used to define the position, size, and shape of a preset anchor frame.

[0152] The initial position refers to the representation of the predefined anchor box's location in the image. For example, the initial position can include the coordinates of the center point of the predefined anchor box relative to a grid cell in the feature image, mapped back to the original ground image. The anchor box angle describes the orientation of the predefined anchor box. The anchor box angle is typically an angle value representing the rotation angle of the predefined anchor box relative to a specified direction (e.g., the horizontal direction).

[0153] In some embodiments, the preset parameters can be set based on experience or obtained by clustering training samples to cover as many real guide line shapes and positions as possible.

[0154] In some embodiments of this specification, by determining preset parameters, the target detection model can more accurately locate and identify guide lines.

[0155] In some embodiments, the detection bounding box of the guide line is determined based on a ground image using a target detection model, including:

[0156] Downsampling is performed on the ground image to obtain its feature image;

[0157] Based on the feature image, the detection box of the guide line is determined by the target detection model.

[0158] A feature image is an image that reflects the characteristics of a ground image. For example, a feature image can include information such as texture and color in a land surface image. Feature images retain the main features of the ground image while requiring lower resolution and less computation.

[0159] Downsampling refers to the operation of reducing the resolution of an image.

[0160] In some embodiments, ground images can be processed based on mean downsampling and bilinear interpolation downsampling to obtain feature images.

[0161] In some embodiments of this specification, downsampling processing can efficiently and accurately determine the detection frame of the guide line.

[0162] In some embodiments, the detection bounding box of the guide line is determined based on the feature image using a target detection model, including:

[0163] Based on the feature image, the detection box of the guide line in the feature image is obtained through the target detection model;

[0164] The detection boxes in the feature image are projected onto the ground image to obtain the detection boxes of the guide lines in the ground image.

[0165] In some embodiments, the downsampled feature image is input into the target detection model to generate multiple preset anchor boxes, and classification and regression operations are performed on the multiple preset anchor boxes. The classification operation is used to determine whether there is a guide line in the preset anchor box, and the regression operation is used to adjust the position and size of the preset anchor boxes to obtain the detection box of the guide line in the feature image.

[0166] In some embodiments, non-maximum suppression can be applied to the regressed preset anchor boxes to remove redundant anchor boxes, and the retained anchor boxes can be used as detection boxes for guide lines in the feature image.

[0167] In some embodiments, the projection relationship between the feature image and the ground image can be determined, and the detection boxes in the feature image can be projected onto the ground image.

[0168] In some embodiments, projecting the detection box in the feature image onto the ground image to obtain the detection box of the guide line in the ground image includes:

[0169] The detection boxes in the ground image are obtained based on the size information of the detection boxes in the feature image and the ground image.

[0170] For example, if the detection bounding box of the detected guide line in the feature image consists of 72 pixels, then the output of the target detection model contains the coordinate information (x, y) of the 72 pixels. The coordinate information of each pixel consists of a number from 0 to 1. By multiplying the coordinate information (x, y) by the size of the original ground image, the detection bounding box of the guide line in the ground image is obtained. For example, if the width of the original ground image is 1920, then the x value in the feature image is 0, indicating that the horizontal coordinate of the pixel in the ground image is 0. If the x value in the feature image is 1, then the horizontal coordinate of the pixel in the ground image is 1920.

[0171] In some embodiments, distortion correction can be performed based on the detection bounding boxes of the guide lines in the ground image. For example, the camera matrix and distortion coefficients of the image acquisition device are calibrated to convert data of detection bounding boxes that may contain distortion into data of ideal, distortion-free detection bounding boxes.

[0172] In some embodiments, the relevant information of the detection boxes in the vehicle coordinate system can be obtained by coordinate transformation based on the detection boxes in the distortion-free ground image.

[0173] In some embodiments of this specification, by projecting the detection box in the feature image onto the ground image, the accurate position of the guide line in the high-resolution image can be obtained, which is helpful for applications in high-precision target detection scenarios.

[0174] In some embodiments, the object detection model is trained through the following steps:

[0175] The sample ground image is input into the initial model to obtain multiple anchor frame regions of the sample ground image, where the anchor frame regions are candidate regions that may contain guide lines;

[0176] Based on multiple anchor box regions, an attention matrix is ​​obtained, which is used to describe the association strength between different anchor box regions;

[0177] Based on the attention matrix, the feature vector corresponding to each anchor box region, and the labeled data of the sample ground image, the loss value of the initial model is determined;

[0178] The initial model is trained based on the loss value to obtain the object detection model.

[0179] Anchor frame regions are image regions corresponding to preset anchor frames generated by the object detection model, used to predict the location and size of guide lines that may be present in the sample ground image.

[0180] The loss value is used to measure the difference between the detection results of the object detection model and the true label.

[0181] Annotated data refers to ground-based information used to train and evaluate the initial model. Annotated data can be done manually.

[0182] In some embodiments, the annotation data may include information such as the position of the guide line and the confidence score.

[0183] In some embodiments, the loss value comprises two parts: a classification loss value and a regression loss value. The classification loss value measures the accuracy of classifying the anchor box region; the regression loss value measures the deviation of the anchor box region's position and size from the true label. By minimizing the loss value, the initial model gradually improves its detection accuracy.

[0184] In some embodiments of this specification, training a target detection model to detect guide lines in ground images can improve the performance of the target detection model in practical applications, and help identify guide lines in complex environments and changing lighting conditions, so as to ensure that vehicles can accurately identify and track guide lines.

[0185] Each anchor frame region map corresponds to a feature vector on the feature image, and the feature vector can represent the local features of the anchor frame region.

[0186] In some embodiments, the feature vector of the anchor box region can be extracted from the feature image. For example, a fully connected layer (or a fully convolutional layer) is used to predict the confidence score of each candidate region, which represents the probability that the candidate region belongs to the guide line, and to predict the bounding box offset of each candidate region, which represents the center point translation and width and height scaling between the candidate region and the ground truth bounding box.

[0187] In some embodiments, cross-entropy loss can be calculated to obtain a classification loss value to measure the difference between the predicted confidence score and the true confidence score; and Smooth L1 loss can be calculated to obtain a regression loss value to measure the difference between the predicted bounding box offset and the true bounding box offset.

[0188] In some embodiments, the loss value of the initial model can be obtained in various ways based on the classification loss value and the regression loss value. For example, summation or weighted summation. The weights can be determined based on historical data or prior knowledge.

[0189] In some embodiments of this specification, through optimization of feature extraction and loss values, the initial model can more accurately identify and locate guide lines, which helps control vehicle movement.

[0190] In some embodiments, the similarity between different anchor box regions can be calculated, such as by the dot product or cosine similarity of feature vectors, to generate an attention matrix. The element Aij of the attention matrix represents the weight of the i-th anchor box region on the j-th anchor box region.

[0191] In some embodiments, the loss value of the initial model is determined based on the attention matrix, the feature vector corresponding to each anchor box region, and the labeled data of the sample ground image, including:

[0192] The feature vector of the anchor box region is fused with the attention matrix to obtain the fused feature vector;

[0193] The loss value of the initial model is determined based on the fused feature vectors and the labeled data of the sample ground images.

[0194] In some embodiments, the feature vector of each anchor box region is added to the weight value of the attention matrix to obtain a fused feature vector. For example, fused feature vector i = feature vector i + ΣAij × feature vector j.

[0195] The fused feature vectors reflect the global relationships between anchor frame regions.

[0196] In some embodiments, a fully connected layer (or a fully convolutional layer) can be used to predict the confidence score of each candidate region and the bounding box offset of each candidate region based on the fused feature vector. The classification loss value and the regression loss value of the bounding box offset are calculated based on the confidence score, the bounding box offset and the label data to determine the loss value of the initial model.

[0197] In some embodiments of this specification, by combining local features of the anchor box region with global attention information, the loss function can better utilize the relationship between the anchor box regions, thereby improving the performance of the detection guide line and helping to improve the adaptability of the target detection model to complex scenes.

[0198] Figure 5 This is an exemplary flowchart of a vehicle driving control method according to some embodiments of this specification. In some embodiments, process 500 may be executed based on electronic devices. Figure 5 As shown, process 500 includes the following steps.

[0199] Step 510: Determine the path planning information according to the aforementioned path planning method;

[0200] Step 520: Determine driving parameters based on route planning information and vehicle status parameters;

[0201] Step 530: Control the vehicle's movement according to the driving parameters.

[0202] Driving parameters refer to the command values ​​used to adjust the vehicle's motion state. Driving parameters can adjust the vehicle's speed, acceleration, steering angle, etc.

[0203] In some embodiments, vehicle status parameters can be monitored in real time, and the vehicle can be controlled according to the vehicle status parameters to ensure smooth vehicle operation.

[0204] Vehicle state parameters refer to various data used to describe the vehicle's driving state. For example, vehicle state parameters may include the vehicle's lateral deviation from the lane lines, lateral velocity, acceleration, and the vehicle's current position.

[0205] In some embodiments, driving parameters include the vehicle's steering angle and / or driving speed.

[0206] In some embodiments, the steering angle required to orient the vehicle toward the target point can be calculated based on the planned path information. For example, the lateral error and heading deviation angle of the vehicle are determined based on the path planning information and the driving trajectory; the vehicle's movement is then controlled based on the lateral error and heading deviation angle.

[0207] In some embodiments, the lateral error is the distance between the center position of the vehicle's front axle and the center trajectory point of the path planning information.

[0208] In some embodiments, the lateral error includes the distance from the center of the vehicle's front axle to the nearest trajectory reference point, which may be the point closest to the vehicle in the path planning information.

[0209] In this embodiment, the heading deviation angle of the vehicle is the angle required for the vehicle to resume straight-line driving.

[0210] In some embodiments, the steering angle required for the vehicle to return to the guide line and continue driving is determined based on the lateral error and the heading deviation angle; the vehicle is controlled to steer at this steering angle so that the vehicle returns to the guide line and continues driving.

[0211] In some embodiments, the steering wheel angle required for the vehicle to return to the guide line and continue driving is determined based on the lateral error and the heading deviation angle; the vehicle is controlled to drive according to the steering wheel angle so that the vehicle returns to the guide line and continues driving.

[0212] In some embodiments, the vehicle speed can be obtained by looking up a table based on the distance traveled and a preset mapping relationship. The preset mapping relationship is used to indicate the correspondence between multiple preset distances and preset speeds.

[0213] The preset mapping relationship can be determined based on experiments or experience. In some embodiments, the forward distance can be the Euclidean distance between the vehicle's current position and the target point.

[0214] In some embodiments, a steering control command is generated based on the calculated steering angle, and the vehicle is steered through the vehicle's steering system; a speed control command is generated based on the calculated forward distance, and the vehicle's speed is controlled through the vehicle's power system.

[0215] In some embodiments, driving parameters can be sent to the vehicle's chassis (e.g., steering system) to control the vehicle's movement.

[0216] In some embodiments of this specification, by accurately calculating the steering angle and other control parameters, the vehicle is ensured to travel along the predetermined planned path information, reducing the risk of deviation; the vehicle can accurately stop at the predetermined position, facilitating passenger boarding and alighting and improving the user experience.

[0217] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0218] Figure 6 This is a schematic diagram of the structure of a vehicle driving control system according to some embodiments of this specification.

[0219] like Figure 6 As shown, one or more embodiments of this specification also provide a structural schematic diagram of a vehicle driving control system. This vehicle driving control system may include:

[0220] Vision module 601 is used to acquire ground images of the driving area;

[0221] The control module 602 is used to control the driving of a vehicle according to any of the vehicle driving control methods described above.

[0222] The vision module 601 and the control module 602 can be used to execute the corresponding embodiments of the path planning method and vehicle driving control method described above. For the specific implementation methods of these modules and more details, please refer to the corresponding method section, which will not be elaborated here.

[0223] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0224] Figure 7 This is a schematic diagram of the structure of an electronic device according to some embodiments of this specification.

[0225] This application also provides an electronic device, which may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will understand that... Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0226] The processor 701 is the vehicle's driving control center, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 702, and calls data stored in the memory 702, to perform various functions and process data, thereby providing overall monitoring of the electronic device. It is understood that the processor 701 communicates with the controller via signal transmission. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 701.

[0227] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0228] In some embodiments of this application, the vehicle driving control system can be implemented as a computer program, and the computer program can be implemented as follows: Figure 7 The device operates on the electronic device shown. The memory of the electronic device can store various program modules that make up the vehicle driving control system. The computer program composed of the various program modules causes the processor to execute the steps in the path planning methods of the various embodiments of this application described in this specification.

[0229] The electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external electronic devices via a network connection. When the computer program is executed by the processor, it implements a path planning method provided in the above embodiment.

[0230] The electronic device also includes a power supply 703 that supplies power to the various components. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 703 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0231] The electronic device may also include an input unit 704, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0232] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 702 according to computer instructions, and the processor 701 runs the applications stored in the memory 702 to realize various functions, such as the path planning methods of the various embodiments of this application described in this specification.

[0233] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0234] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0235] It should be noted that, Figure 7 This is merely one implementation of the electronic device 700 provided in this application embodiment. In actual applications, the electronic device 700 may include more or fewer components, which is not limited here.

[0236] It should be understood that the various solutions in the embodiments of this application can be used in a reasonable combination, and the explanations or descriptions of the various terms appearing in the embodiments can be referenced or explained to each other in the various embodiments, without limitation.

[0237] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0238] Based on the above embodiments and the same concept, this application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method provided in the above embodiments.

[0239] Based on the above embodiments and the same concept, this application also provides a computer program product, which includes a computer program or instructions that, when run on a computer, cause the computer to perform the methods provided in the above embodiments.

[0240] Figure 8 This is an exemplary schematic diagram of a vehicle according to some embodiments of this specification.

[0241] like Figure 8 As shown in the illustration, this application also provides a vehicle, which includes the electronic equipment or the vehicle driving control system described in any embodiment, and a chassis, wherein the chassis is adapted to adjust the vehicle's driving state according to driving parameters. The vehicle may be a gasoline-powered vehicle, a plug-in hybrid electric vehicle, or a new energy vehicle, etc., and this specification does not specifically limit it.

[0242] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0243] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.

[0244] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although the descriptions of each embodiment in this application have different focuses, and the parts not described in detail in a certain embodiment can be referred to the relevant embodiments of other embodiments, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of this application without departing from the content of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A path planning method, characterized in that, The method includes: Based on ground images of the driving area, identify guide lines within the driving area; Key points are generated based on the guide lines; Select the target point from the key points; Path planning information is determined based on the target point.

2. The method according to claim 1, characterized in that, The step of generating key points based on the guide lines includes: The guide line is discretized to obtain multiple key points on the guide line.

3. The method according to claim 2, characterized in that, The target point is selected from the key points; Based on the target point, path planning information is determined, including: Select the key point closest to the current location from among the multiple key points as the target point; Based on the target point and the current location, the path planning information is determined.

4. The method according to claim 1, characterized in that, The identification of guide lines within the driving area based on the ground image of the driving area includes: Based on the ground image, the detection box of the guide line is determined using a target detection model.

5. The method according to claim 4, characterized in that, The step of determining the detection box of the guide line based on the ground image and using a target detection model includes: Based on the target detection model, multiple preset anchor boxes are generated for the ground image; Regression is performed on multiple preset anchor frames to obtain the detection frame of the guide line.

6. The method according to claim 5, characterized in that, The generation of multiple preset anchor boxes for the ground image based on the target detection model includes: Based on the preset parameters of the target detection model, multiple preset anchor frames are generated for the ground image. The preset parameters include at least one of the initial position, anchor frame angle, and anchor frame length.

7. The method according to claim 4, characterized in that, The step of determining the detection box of the guide line based on the ground image and using a target detection model includes: The ground image is downsampled to obtain a feature image of the ground image; Based on the feature image, the detection box of the guide line is determined using the target detection model.

8. The method according to claim 7, characterized in that, The step of determining the detection box of the guide line based on the feature image and the target detection model includes: Based on the feature image, the detection box of the guide line in the feature image is obtained through the target detection model; The detection box in the feature image is projected onto the ground image to obtain the detection box of the guide line in the ground image.

9. The method according to claim 8, characterized in that, The step of projecting the detection box in the feature image onto the ground image to obtain the detection box of the guide line in the ground image includes: The detection bounding box in the ground image is obtained based on the size information of the detection bounding box in the feature image and the ground image.

10. The method according to claim 4, characterized in that, The target detection model is trained through the following steps: The sample ground image is input into the initial model to obtain multiple anchor frame regions of the sample ground image, wherein the anchor frame regions are candidate regions that may contain guide lines; Based on the multiple anchor frame regions, an attention matrix is ​​obtained, which is used to describe the association strength between different anchor frame regions; Based on the attention matrix, the feature vector corresponding to each anchor box region, and the labeled data of the sample ground image, the loss value of the initial model is determined; The initial model is trained based on the loss value to obtain the target detection model.

11. A vehicle driving control method, characterized in that, The method includes: The path planning information is determined according to any one of claims 1-10; Based on the route planning information and vehicle status parameters, determine the driving parameters; The vehicle is controlled to drive according to the driving parameters.

12. The method according to claim 11, characterized in that, The driving parameters include steering angle and / or driving speed.

13. A vehicle driving control system, characterized in that, The system includes a vision module and a control module, wherein, The vision module is used to acquire ground images of the driving area; The control module is used to control vehicle movement according to the method according to any one of claims 11-12.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the path planning method according to any one of claims 1-10.

15. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the path planning method according to any one of claims 1-10.

16. A vehicle, characterized in that, Includes the electronic device of claim 15 or the vehicle driving control system of claim 13, and the chassis, wherein the chassis is adapted to adjust the vehicle driving state according to driving parameters.

Images