A path construction method and device, a terminal and a storage medium

Abstract

The application discloses a path construction method and device, a terminal and a storage medium, and the method comprises the following steps: acquiring vehicle driving state information and initial images of the surroundings of a preset driving path corresponding to the vehicle driving state information in real time when driving on the preset driving path; calculating a target overhead view corresponding to the initial images by a nonlinear difference correction algorithm according to the initial images; inputting the target overhead view into a preset deep learning model, classifying the pixel points of the target overhead view input into the preset deep learning model, and obtaining a partitioned image, which is an image obtained after the target overhead view is partitioned; scanning the partitioned image, and identifying a drivable area of the vehicle; and generating a path trajectory corresponding to the vehicle driving state information based on the drivable area, wherein the path trajectory is a path in the preset driving path; and the vehicle obtains the path trajectory through automatic learning, so that the vehicle can automatically plan a path in the subsequent automatic driving process.

Description

Technical Field

[0001] This invention relates to the field of self-learning technology for autonomous vehicles, and in particular to a path construction method, apparatus, terminal, and storage medium. Background Technology

[0002] With the development of automotive intelligence, autonomous driving is getting closer and closer to us. Among them, the "last mile" is the final link of autonomous driving. It is located in a low-speed and relatively closed environment with relatively low driving risks and high convenience and comfort for users. It is very likely to arrive ahead of schedule.

[0003] Currently, many solutions for automatically finding parking spaces and parking in underground garages involve preloading garage maps. This requires recording garage maps in advance, and only by creating a sufficient number of garage maps can it be used on a large scale, resulting in significant upfront costs. Another solution is to provide parking route planning through garage parking guidance and direction systems. This solution also requires the installation of recognition and communication equipment in a large number of garages, as well as the maintenance and upgrading of the equipment, which requires substantial manpower and material resources. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention discloses a path construction method. The vehicle automatically learns and acquires a path trajectory while driving along a pre-set driving path in an underground parking garage, enabling the vehicle to automatically plan paths during subsequent autonomous driving processes.

[0005] To achieve the above-mentioned objective, the present invention provides a path construction method, the method comprising:

[0006] The method includes:

[0007] While driving on a preset driving path, the vehicle driving status information and the initial image of the surrounding environment of the preset driving path corresponding to the vehicle driving status information are acquired in real time.

[0008] Based on the initial image, a target top view corresponding to the initial image is calculated using a nonlinear difference correction algorithm;

[0009] The target top view is input into a preset deep learning model, and the pixels of the target top view are classified to obtain partitioned images, which include images of drivable areas and images of non-drivable areas.

[0010] Scan the partitioned image to identify the drivable area of ​​the vehicle;

[0011] A path trajectory corresponding to the vehicle's driving status information is generated based on the drivable area, and the path trajectory is one of the preset driving paths.

[0012] In one embodiment, after generating the path trajectory corresponding to the vehicle driving status information based on the drivable area, the method further includes:

[0013] A map corresponding to the preset driving route is constructed based on the path trajectory.

[0014] In one embodiment, the real-time acquisition of vehicle driving status information and the initial image of the surrounding environment of the preset driving path corresponding to the vehicle driving status information includes:

[0015] The vehicle driving status information is acquired in real time as the vehicle travels along a preset driving path. The vehicle driving status information includes the driving strategy of the vehicle and the driver's driving habits during the journey along the preset driving path.

[0016] Based on the driving strategy and the driver's driving habits, the initial image of the environment around the preset driving path is acquired in real time during the vehicle's driving on the preset driving path.

[0017] In one embodiment, the drivable area includes drivable roads and drivable intersections, and obtaining the driving strategy of the vehicle while driving on the preset driving path includes:

[0018] Real-time acquisition of vehicle speed and steering wheel angle;

[0019] The vehicle's forward distance and heading angle are determined based on the vehicle's speed and the steering wheel angle.

[0020] The vehicle's driving strategy is determined based on the vehicle's forward distance and heading angle. The vehicle's driving strategy includes the forward distance on the drivable road and whether to turn at the drivable intersection.

[0021] In one embodiment, the drivable area includes drivable roads and drivable intersections; acquiring the driver's driving habits while the vehicle is traveling on a preset driving path includes:

[0022] Real-time acquisition of vehicle operation data during the driving process along a preset driving path;

[0023] The vehicle's operating data is preprocessed to obtain target operating data;

[0024] The target running data is input into a recurrent neural network, and features of the target running data are extracted from the recurrent neural network.

[0025] The features are input into a fully connected network to predict the driver's driving habits during the vehicle's journey along a preset driving path; the driving habits include the driving speed on drivable roads and the turning angle at drivable intersections.

[0026] In one embodiment, after generating the path trajectory corresponding to the vehicle's driving status information based on the drivable area, the method further includes: when driving again on the preset driving path...

[0027] Real-time acquisition of vehicle driving status information and initial images of the surrounding environment of the preset driving path corresponding to the vehicle driving status information.

[0028] Based on the initial image, a target top view corresponding to the initial image is calculated using a nonlinear difference correction algorithm;

[0029] The target top view is input into a preset deep learning model, and the pixels of the target top view input into the preset deep learning model are classified to obtain partitioned images, which include images of drivable areas and images of non-drivable areas.

[0030] Scan the partitioned image to identify the drivable area of ​​the vehicle;

[0031] Based on the drivable area, a current path trajectory corresponding to the vehicle's driving status information is generated, and the path trajectory is one of the preset driving paths.

[0032] In one embodiment, after generating the current path trajectory corresponding to the vehicle driving status information based on the drivable area, the method further includes:

[0033] The current path trajectory and the previously obtained path trajectory are fused to reconstruct a map corresponding to the preset driving route.

[0034] In one embodiment, before performing multi-track fusion on the current path trajectory and the previously obtained path trajectory to reconstruct the map corresponding to the preset driving route, the method further includes:

[0035] Determine whether the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to a preset first threshold.

[0036] If the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to the preset first threshold, then the current path trajectory and the previously obtained path trajectory are merged.

[0037] In one embodiment, it also includes:

[0038] If the overlap between the current path trajectory and the previously obtained path trajectory is less than the preset first threshold, then it is determined whether the matching degree between the current path trajectory and the preset driving path is less than the matching degree between the previously obtained path trajectory and the preset driving path.

[0039] If so, then regenerate the current path trajectory.

[0040] In one embodiment, calculating the target top view corresponding to the initial image using a nonlinear difference correction algorithm based on the initial image includes:

[0041] A target image is obtained based on the initial image, wherein the target image includes a top view of a region image that overlaps with the region where the target top view is located;

[0042] Obtain the number of times the target image appears;

[0043] Determine whether the number of times the target image appears is greater than or equal to a preset second threshold;

[0044] If so, then extract the feature points of the region image in each of the target images;

[0045] The feature points of each region image are matched to reconstruct the top view of the target.

[0046] In one embodiment, the step of calculating the target top view corresponding to the initial image using a nonlinear difference correction algorithm based on the initial image further includes:

[0047] The nonlinear difference correction algorithm is used to obtain the correspondence between the top view of the initial image and the initial image, and the correspondence includes the coordinate points between the top view of the initial image and the corresponding points between the initial images;

[0048] The target coordinate points are obtained from the initial image based on the correspondence.

[0049] Construct a top view of the target corresponding to the initial image based on the target coordinate points.

[0050] In one embodiment, the drivable area includes drivable roads and drivable intersections, and generating a path trajectory corresponding to the vehicle's driving status information based on the drivable area includes:

[0051] Based on the drivable area, determine the drivable roads, drivable intersections, and the distribution of the drivable roads and drivable intersections within the drivable area;

[0052] Based on the distribution of the drivable roads and intersections, a path trajectory corresponding to the vehicle's driving status information is generated.

[0053] In one embodiment, before identifying the drivable roads, drivable intersections, and the distribution of the drivable roads and drivable intersections in the drivable area, the method further includes:

[0054] The partitioned image is scanned using squares of a preset size to obtain the drivable area and the scanned area of ​​the vehicle;

[0055] Based on the scanned area, the drivable area is adjusted and reconstructed.

[0056] In one embodiment, adjusting the drivable area based on the scanned area and reconstructing the drivable area includes:

[0057] Based on the scanned area, the drivable area is expanded to obtain the expanded area;

[0058] Based on the scanned area, an erosion operation is performed on the expanded area to reconstruct the drivable area.

[0059] In one embodiment, identifying drivable roads, drivable intersections, and the distribution of the drivable roads and drivable intersections within the drivable area includes:

[0060] The drivable area is input into the road recognition model to identify the drivable roads in the drivable area and the information of the drivable roads, including the width and length of the drivable roads.

[0061] The drivable area is input into the intersection recognition model to identify the drivable intersections in the drivable area and the type of the drivable intersections;

[0062] Based on the information of the drivable area, the drivable roads within the drivable area, and the types of drivable intersections, the distribution of the drivable roads and drivable intersections within the drivable area is determined.

[0063] The present invention also provides a path construction apparatus, the apparatus comprising:

[0064] The first acquisition module is used to acquire vehicle driving status information and initial images of the surrounding environment of the preset driving path in real time when driving on a preset driving path.

[0065] The target top view acquisition module is used to calculate the target top view corresponding to the initial image based on the initial image using a nonlinear difference correction algorithm;

[0066] The partitioned image acquisition module is used to input the target top view into a preset deep learning model, classify the pixels of the target top view input into the preset deep learning model, and obtain partitioned images, wherein the partitioned images include drivable area images and non-drivable area images.

[0067] The recognition module is used to scan the partition image and identify the drivable area of ​​the vehicle;

[0068] The path trajectory generation module is used to generate a path trajectory corresponding to the vehicle driving status information based on the drivable area, wherein the path trajectory is one of the preset driving paths.

[0069] The present invention also provides a path building terminal, the terminal including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the path building method as described above.

[0070] The present invention also provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded by a processor and executed as described above in the path construction method.

[0071] Implementing the embodiments of the present invention has the following beneficial effects:

[0072] The path construction method disclosed in this invention allows a vehicle to automatically learn and obtain a path trajectory while driving on a preset driving path in an underground parking garage, so that the vehicle can automatically plan a path during subsequent autonomous driving. Attached Figure Description

[0073] To more clearly illustrate the path construction method, apparatus, system, and terminal of the present invention, the accompanying drawings required for the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 A flowchart illustrating a path construction method provided in an embodiment of the present invention;

[0075] Figure 2 This is a schematic diagram of a process for obtaining a vehicle driving strategy according to an embodiment of the present invention;

[0076] Figure 3 A schematic diagram of a process for obtaining a driver's driving habits provided in an embodiment of the present invention;

[0077] Figure 4 A schematic diagram of a process for obtaining a top view of a target provided by an embodiment of the present invention;

[0078] Figure 5 A schematic diagram of a top view for acquiring an initial image, provided as an embodiment of the present invention;

[0079] Figure 6 A schematic diagram illustrating another process for obtaining a target top view according to an embodiment of the present invention;

[0080] Figure 7 This is a schematic diagram of a structure for obtaining the location of an extreme point according to an embodiment of the present invention;

[0081] Figure 8 This is a schematic diagram illustrating the classification of pixels in an image according to an embodiment of the present invention.

[0082] Figure 9 This is a flowchart illustrating a method for identifying drivable roads and intersections in a drivable area, as provided in an embodiment of the present invention.

[0083] Figure 10 This is a schematic diagram illustrating the identification results of drivable roads and drivable intersections in a drivable area, provided by an embodiment of the present invention.

[0084] Figure 11 This is an example diagram of path trajectory fusion provided by an embodiment of the present invention;

[0085] Figure 12 This is a schematic diagram of a path construction device provided in an embodiment of the present invention;

[0086] Figure 13 This is a schematic diagram of a path construction terminal provided in an embodiment of the present invention. Detailed Implementation

[0087] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0088] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0089] The path construction method of this application is applied to the field of autonomous driving. Specifically, it involves a human driving a vehicle through an underground parking garage at least once, enabling the vehicle to automatically learn the path of the underground parking garage and then build an abstract path map so that the vehicle can subsequently perform autonomous driving based on the path map.

[0090] The following combination Figure 1 This invention introduces a path construction method based on the aforementioned system, which can be applied to path construction methods for autonomous vehicles. This invention can be applied, but is not limited to, sealed scenarios, such as underground parking garages, and is a method for constructing virtual maps of underground parking garages based on automatic path learning.

[0091] Please refer to Figure 1 The diagram illustrates a path construction method according to an embodiment of the present invention. This specification provides the operational steps described in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only execution order. The path construction method can be executed according to the order shown in the embodiments or drawings. Specifically, as shown... Figure 1 As shown, the method includes:

[0092] S101, When driving on a preset driving path, real-time vehicle driving status information and initial images of the surrounding environment of the preset driving path corresponding to the vehicle driving status information are acquired.

[0093] It should be noted that, in the embodiments of this specification, the autonomous vehicle can be driven manually by a driver on a preset driving path;

[0094] The preset driving route can be a drivable route that already exists in the preset driving area; for example, it can be at least one drivable road and drivable intersection that already exist in the underground parking garage.

[0095] In the embodiments described in this specification, the initial image of the surrounding environment corresponding to the preset driving path can be obtained in real time through the vehicle's front-view camera.

[0096] Specifically, the initial image can be a two-dimensional image;

[0097] Specifically, when the driver is driving the vehicle on a preset driving path, the vehicle automatically acquires the vehicle driving status information and the initial image of the surrounding environment of the preset driving path corresponding to the vehicle driving status information in real time.

[0098] In this embodiment of the specification, the real-time acquisition of vehicle driving status information and the initial image of the surrounding environment of the preset driving path corresponding to the vehicle driving status information includes the following steps:

[0099] Step 1: Acquire vehicle driving status information in real time while the vehicle is driving on the preset driving path. The vehicle driving status information includes the driving strategy of the vehicle and the driver's driving habits while driving on the preset driving path.

[0100] Specifically, in the embodiments of this specification, reference is made to... Figure 2 The diagram shown is a flowchart of obtaining a vehicle driving strategy according to an embodiment of the present invention.

[0101] In the embodiments described in this specification, specifically, when obtaining the vehicle driving strategy, the drivable area may include drivable roads and drivable intersections;

[0102] S201, real-time acquisition of vehicle speed and steering wheel angle;

[0103] In the embodiments described in this specification, the vehicle speed and steering wheel angle can be obtained based on the vehicle controller area network (CAN) signal;

[0104] S203, determine the vehicle's forward distance and heading angle based on the vehicle's speed and the steering wheel angle;

[0105] In the embodiments of this specification, the vehicle's running time can also be obtained. Specifically, the vehicle's forward distance can be calculated based on the vehicle's speed and running time; the vehicle's heading angle can be calculated based on the steering wheel angle.

[0106] S205, determine the vehicle's driving strategy based on the vehicle's forward distance and heading angle, the vehicle's driving strategy including the forward distance on the drivable road and whether to turn at the drivable intersection.

[0107] In the embodiments of this specification, the driving trend of the vehicle, that is, the driving strategy of the vehicle, can be determined based on the vehicle's forward distance and the heading angle.

[0108] Specifically, vehicle driving strategies may include vehicle driving data and driving needs, such as the distance traveled on the drivable road and whether to turn at the drivable intersection.

[0109] The method for obtaining the vehicle driving strategy in this application can accurately obtain the driving strategy of the vehicle when it is driving on the preset driving path, so as to obtain the initial image of the environment around the preset driving path when the vehicle is driving on the preset driving path according to the vehicle driving strategy.

[0110] Specifically, in the embodiments of this specification, reference is made to... Figure 3 The diagram shown is a flowchart of an embodiment of the present invention for obtaining a driver's driving habits.

[0111] In the embodiments of this specification, specifically, when acquiring the driver's driving habits, the drivable area may include drivable roads and drivable intersections;

[0112] S301, real-time acquisition of vehicle operation data during the driving process of a preset driving path;

[0113] In the embodiments of this specification, the vehicle's operating data during the driving process of the preset driving path may include the vehicle's steering angle, steering acceleration, vehicle speed, vehicle acceleration, accelerator pedal, and brakes, etc.

[0114] The vehicle's operational data during its journey along the preset driving path can also include driving video. Specifically, the vehicle's trajectory can be determined based on the driving video.

[0115] Specifically, when acquiring vehicle operation data, it is necessary to establish a time window and acquire vehicle operation data before and after changes in the vehicle's driving trajectory within the time window;

[0116] Specifically, the vehicle's operating data varies within different time windows;

[0117] S303, preprocess the vehicle's operating data to obtain target operating data;

[0118] In the embodiments of this specification, preprocessing the vehicle's operating data can be done by preprocessing the vehicle's operating data obtained within a time window. Specifically, it can be done by preprocessing data such as the vehicle's speed, acceleration, steering angle, and steering acceleration.

[0119] Specifically, the maximum, minimum, and average values ​​of vehicle speed, acceleration, steering angle, and steering acceleration can be obtained separately; the maximum, minimum, and average values ​​of each obtained operating data are the target operating data.

[0120] S305, Input the target running data into a recurrent neural network, and extract the features of the target running data from the recurrent neural network;

[0121] In the embodiments of this specification, features of the target running data can be extracted after the target running data is obtained;

[0122] Specifically, the target running data is input into a recurrent neural network, and features of the target running data are extracted from the recurrent neural network using a sequence-to-sequence model structure (seq2seq).

[0123] S307, Input the features into a fully connected network to predict the driver's driving habits during the vehicle's journey along a preset driving path; the driving habits include the driving speed on drivable roads and the turning angle at drivable intersections.

[0124] In the embodiments of this specification, after extracting the features of the target running data, the vehicle control features are preset according to the features. The control features may include the driving speed of the drivable road and the turning angle of the drivable intersection.

[0125] Specifically, the driver's driving habits can be obtained based on the vehicle's control characteristics during operation.

[0126] The method for obtaining the driver's driving habits in this application can effectively predict the driver's driving habits based on the vehicle's operating data while driving on a preset driving path, so as to obtain the initial image of the environment around the preset driving path during the vehicle's driving on the preset driving path based on the driver's driving habits.

[0127] Step 2: Based on the driving strategy and the driver's driving habits, acquire initial images of the environment surrounding the preset driving path in real time as the vehicle travels on the preset driving path.

[0128] In the embodiments of this specification, the initial image obtained by the vehicle's forward-facing camera during the acquisition of the initial image of the environment surrounding the preset driving path corresponds to the vehicle's driving strategy and the driver's driving habits.

[0129] When the vehicle's driving strategy and / or the driver's driving habits change, the number of initial images of the surrounding environment of the preset driving path, as well as the viewpoint and pixels of the images, will be different.

[0130] S103, Based on the initial image, a target top view corresponding to the initial image is calculated using a nonlinear difference correction algorithm;

[0131] In the embodiments of this specification, reference is made to Figure 4 The diagram shows a flowchart of obtaining a target top view according to an embodiment of the present invention; the details are as follows:

[0132] The step of calculating the target top view corresponding to the initial image using a nonlinear difference correction algorithm based on the initial image includes:

[0133] S401, Based on the nonlinear difference correction algorithm, obtain the correspondence between the top view of the initial image and the initial image, the correspondence including the coordinate points corresponding to the top view of the initial image and the initial image;

[0134] In the embodiments of this specification, before obtaining the correspondence between the top view of the initial image and the initial image, the method further includes: obtaining the top view of the initial image;

[0135] like Figure 5 The diagram shown is a top view of a method for obtaining an initial image.

[0136] The specific steps for obtaining the correspondence between the top view of the initial image and the initial image are as follows:

[0137] Perform a distortion correction operation on the initial image to obtain a distortion-corrected image;

[0138] Four points are selected in the distorted image to determine the top view of the initial image corresponding to the four points.

[0139] The coordinates of each point in the top view are identified, and the corresponding coordinates in the initial image are obtained through the perspective matrix, thereby obtaining the correspondence between the top view of the initial image and the initial image.

[0140] Specifically, the correspondence can be h(m,n)=f(i,j).

[0141] Specifically, the algorithm for obtaining the correspondence between the top view and the initial image by setting the perspective matrix is ​​as follows:

[0142] Let the perspective matrix be M, then the perspective transformation equation is: P = M·p

[0143] in, (x, y) are the coordinates of a point in the top view. (X / k, Y / k) are the coordinates of the corresponding point in the distortion-free image.

[0144] k = m 20 *x+m 21 *y+m 22 Substitute the four sets of points into the equation to obtain the perspective matrix M. Iterate through and identify the coordinates of each point in the top view, calculate the corresponding coordinates using the perspective matrix M, and then obtain the pixel information of the initial image coordinates according to h(m,n)=f(i,j).

[0145] In this embodiment of the specification, obtaining the correspondence between the top view of the initial image and the initial image based on the nonlinear difference correction algorithm includes:

[0146] Obtain the first pixel of the top view obtained above, and find the second pixel corresponding to the first pixel in the pixels of the initial image; the correspondence between the first pixel and the second pixel can be the correspondence between the top view of the initial image and the initial image.

[0147] Specifically, in the embodiments of this specification, in subsequent use, the correspondence between the top view of the initial image and the initial image can be directly obtained based on the nonlinear difference correction algorithm. The correspondence may include the coordinate points corresponding to the top view and the initial image. In this application, this method can quickly obtain the target top view of the initial image.

[0148] S403, Obtain the target coordinate point from the initial image based on the correspondence;

[0149] In the embodiments of this specification, based on the obtained initial image and the correspondence obtained above, the target coordinate point corresponding to the target top view can be directly found in the initial image.

[0150] S405, construct a top view of the target corresponding to the initial image based on the target coordinate points.

[0151] In the embodiments of this specification, a top view of the target corresponding to the initial image can be directly constructed based on the obtained target coordinate points.

[0152] In the embodiments of this specification, reference is made to Figure 6 The diagram illustrates another process for obtaining a target top view according to an embodiment of the present invention; the details are as follows:

[0153] S601, Obtain a target image based on the initial image, wherein the target image includes a top view of a region image that overlaps with the region where the target top view is located;

[0154] In the embodiments of this specification, the vehicle can obtain top-view images corresponding to several initial images during driving. During driving, the same area can include multiple top-view images from different perspectives of the forward-looking camera. The target image can be a top-view image that simultaneously covers the same object or the same area among multiple top-view images (specifically, it can include an image of an area that overlaps with the area where the target top-view is located).

[0155] Specifically, the region image of the target image coincides with the top view of the target.

[0156] S603, Obtain the number of times the target image appears;

[0157] In the embodiments of this specification, since the vehicle may acquire multiple images of a certain area or a certain image during the driving process, the target image may also appear multiple times;

[0158] S605, determine whether the number of times the target image appears is greater than or equal to a preset second threshold;

[0159] In the embodiments of this specification, the number of times the target image appears can be greater than or equal to a preset second threshold; the preset second threshold can be 50 times.

[0160] When the number of times a target image appears is less than a preset second threshold, the target image can be considered invalid; it can be abandoned or reacquired until the number of times the target top view image appears exceeds the preset second threshold.

[0161] S607, if so, extract the feature points of the region image in each of the target images;

[0162] In the embodiments of this specification, the Gaussian algorithm can be used to extract feature points of the region image;

[0163] Specifically, the target image can be first subjected to Gaussian blur, and then the different Gaussian blur results can be subtracted to obtain the difference operator (DoG):

[0164] Specifically, the algorithm for extracting feature points is as follows: L(x,y,σ)=G(x,y,σ)·I(x,y)

[0165]

[0166]

[0167] Where (x,y) represents spatial coordinates; I(x,y) represents the pixel value at (x,y);

[0168] L(x,y,σ) represents the size space definition of a two-dimensional image;

[0169] G(x,y,σ) represents a scale-variable Gaussian function;

[0170] σ represents the smoothness parameter of the image;

[0171] D(x,y,σ) represents the Gaussian difference scale space; k represents the scale coefficient.

[0172] The pixels of the DoG result of each layer are compared with the neighboring pixels. If they are extreme points, they are the feature points to be found. However, since these extreme points are discrete and some of them are singular points, it is necessary to relocate them to determine the location of the feature points.

[0173] In the embodiments of this specification, the method of re-locating and determining the position of feature points may include: curve fitting of the DoG function and Taylor series expansion to find the precise position;

[0174] Specifically, the algorithm for obtaining the location of feature points using Taylor series is as follows:

[0175] f(x)≈f(0)+f ′ (0)*x+f ″ (0)*x

[0176] Where x represents the position variable,

[0177] f ′ (0) represents the first derivative of f(x) at x = 0.

[0178] f ″ (0) represents the second derivative of f(x) at x = 0;

[0179] Once the location is determined, the size and orientation information of the target can be obtained, thus determining the location of the true extreme point.

[0180]

[0181]

[0182] Where m(x,y) represents the gradient value at (x,y).

[0183] θ(x,y) represents the gradient direction at (x,y).

[0184] L represents the scale space value of the key point coordinates.

[0185] For details, see Figure 7 The diagram shown is a schematic diagram of a structure for obtaining the location of extreme points.

[0186] The graph shows the true extreme points and the detected extreme points.

[0187] S609, Match the feature points of each region image to reconstruct the target top view;

[0188] In the embodiments of this specification, feature points of each region image are matched according to the method of co-coordinate of image feature points to obtain a new target top view; the target top view obtained by this method in this application is more accurate.

[0189] S105, the target top view is input into a preset deep learning model, and the pixels of the target top view input into the preset deep learning model are classified to obtain a partitioned image, the partitioned image including a drivable area image and a non-drivable area image.

[0190] The deep learning model pre-defined in this manual can be a fully convolutional network model;

[0191] In the embodiments of this specification, the preset deep learning model, such as a fully convolutional network model, can accept input images of any size, and then obtain upsampling of the same size as the input through deconvolution, that is, classifying each pixel.

[0192] Furthermore, in this application, after inputting the target top view into the preset deep learning model, the output result is still an image; that is, the preset deep learning model only segments the input image to achieve pixel-level classification and obtain partitioned images.

[0193] Specifically, such as Figure 8 The diagram shown is a schematic diagram of classifying pixels in an image.

[0194] Specifically, after inputting an image of size H×W (where H represents the height of the image and W represents the width of the image) into a pre-defined deep learning model, and performing operations such as convolution, pooling, and nonlinearity, the size of the first layer can be reduced to 1 / 4 of the input. 2 The size of the second-level result can be reduced to 1 / 8 of the input. 2 The size of the fifth layer result can be reduced to 1 / 16 of the input. 2 ..., the size of the eighth layer result can be reduced to 1 / 32 of the input. 2 As the number of convolutions and pooling operations increases, the image size decreases, with the smallest layer being 1 / 32 of the original image. 2 At this point, upsampling is required... The result is enlarged to the original image size H×W and the image is output (pixelwise output + loss). The final image will classify the pixels of each target top view based on the trained image.

[0195] In the embodiments of this specification, the obtained partitioned image may include a drivable area image and a non-drivable area image. The partitioned image may be an image obtained by partitioning the target top view. The partitioned image may include drivable areas, such as drivable roads and drivable intersections. The non-drivable area image may include parking lines, parking space areas, and other information.

[0196] S107, Scan the partition image to identify the drivable area of ​​the vehicle;

[0197] In the embodiments of this specification, information about each region in the scanned partition image is used to determine the drivable area of ​​the vehicle; specifically, the drivable area includes drivable roads and drivable intersections;

[0198] Specifically, a straight-line trend recognition module can be used to identify drivable roads in the drivable area, and an intersection trend recognition module can be used to identify drivable intersections in the drivable area.

[0199] In this embodiment of the specification, before identifying drivable roads and drivable intersections in the drivable area, the following steps are further included:

[0200] The partitioned image is scanned using squares of a preset size to obtain the drivable area and the scanned area of ​​the vehicle;

[0201] Based on the scanned area, the drivable area is adjusted and reconstructed.

[0202] In this embodiment of the specification, adjusting and reconstructing the drivable area based on the scanned area includes the following steps:

[0203] Based on the scanned area, the drivable area is expanded to obtain the expanded area;

[0204] Based on the scanned area, an erosion operation is performed on the expanded area to reconstruct the drivable area;

[0205] In the embodiments of this specification, the size of the squares can be selected according to the actual situation during the design process. In addition, this application uses a dilation and erosion operation on the image, which can effectively remove the situation of missing pixels or pixels not connected to the main body in the recognition result, making the obtained drivable area more accurate.

[0206] S109, Generate a path trajectory corresponding to the vehicle driving status information based on the drivable area, wherein the path trajectory is one of the preset driving paths.

[0207] In the embodiments of this specification, the preset driving path includes at least one drivable path, and specifically, it may include multiple drivable paths; the generated path trajectory may be one of the multiple drivable paths.

[0208] In this embodiment of the specification, generating a path trajectory corresponding to the vehicle's driving state information based on the drivable area may include the following steps:

[0209] Step 1: Based on the drivable area, determine the drivable roads, drivable intersections, and the distribution of the drivable roads and drivable intersections within the drivable area;

[0210] In the embodiments described in this specification, such as Figure 9 The diagram shows a flowchart of a method for identifying drivable roads and intersections within a drivable area; specifically, as follows:

[0211] S901, Input the drivable area into the road recognition model to identify the drivable roads in the drivable area and the information of the drivable roads, wherein the information of the drivable roads includes the width and length of the drivable roads;

[0212] In the embodiments of this specification, the road recognition model can be a road recognition algorithm that identifies road lines on a road and provides the location information of the road markings;

[0213] Specifically, this application may be a straight-line trend recognition algorithm for roads to identify information such as drivable straight lines in drivable areas.

[0214] Specifically, in the embodiments of this specification, the method for identifying drivable roads in a drivable area may include the following steps:

[0215] The road recognition results of size m*n are projected vertically to obtain the number h of road pixels in each column. i :

[0216] h i =h(i), i = 0, 1, 2, ..., n

[0217] Where m represents the height of the image and n represents the width of the image;

[0218] h i , where represents the number of road pixels in the i-th column.

[0219] h i The range is [0, m], then for h i We statistically analyze the values ​​h, where h ranges from [0, m], and count the number of times each value of h occurs, w. h For: w h =w(h)

[0220] Among them, w h , indicating the number of times different h values ​​occur.

[0221] w h The range is [0, n]. Find the value when w h When the maximum value is reached, record the size of h at this point as h. maxThat is, satisfying the threshold for a "column" to become a road, let

[0222] h i =h(i)>h max

[0223] Find the maximum value of i. max and minimum value i min This refers to the column positions of the two sides of the road in the image, which is also the width of the road.

[0224] S903, Input the drivable area into the intersection recognition model to identify the drivable intersections in the drivable area and the type of the drivable intersections;

[0225] In the embodiments of this specification, the intersection recognition model can be an algorithm for recognizing intersections in a road. Specifically, it can identify whether there is a drivable intersection on a drivable road and what type of intersection it is.

[0226] Specifically, in the embodiments of this specification, when identifying drivable intersections in a drivable area, based on the drivable roads identified above, not every intersection requires a turn during the vehicle's journey. When the vehicle reaches an intersection where a turn is not required, it is identified using a straight-line mode; if the vehicle needs to turn, it is indicated as a drivable intersection.

[0227] S905, based on the information of the drivable area, the drivable roads in the drivable area, and the types of drivable intersections, determine the distribution of the drivable roads and the drivable intersections in the drivable area.

[0228] In the embodiments of this specification, based on the above-described method for identifying drivable roads and drivable intersections, the distribution of drivable roads and drivable intersections in a drivable area can be accurately determined.

[0229] In another embodiment of this specification, drivable intersections and drivable roads in the drivable area can also be determined by the vehicle's mileage and heading angle.

[0230] Specifically, the method for identifying drivable intersections and drivable roads within a drivable area includes the following steps:

[0231] Real-time acquisition of vehicle speed and steering wheel angle;

[0232] In the embodiments described in this specification, the vehicle speed and steering wheel angle can be obtained based on the CAN signal;

[0233] The vehicle's forward distance and heading angle are determined based on the vehicle's speed and the steering wheel angle.

[0234] In the embodiments of this specification, the vehicle's running time can also be obtained. Specifically, the vehicle's forward distance can be calculated based on the vehicle's speed and running time; the vehicle's heading angle can be calculated based on the steering wheel angle.

[0235] The drivable intersections and drivable roads in the drivable area are determined based on the vehicle's forward distance and heading angle.

[0236] In the embodiments described in this specification, based on the vehicle's forward mileage, when the vehicle reaches the intersection area, the 5th column or the (n-6th)th column of pixels is selected according to whether it is a left turn or a right turn:

[0237] p i =p(i), i = 0, 1, 2, ..., m

[0238] When p(i) is 0 or 1, it indicates whether it is a road and determines the width of the road;

[0239] P i =p i -p i-1′ i = 1, 2, ..., m

[0240] Where, p i This indicates whether the i-th row in the 5th or (n-6th)th column is a road pixel.

[0241] p(i) is either 0 or 1, indicating whether it is a road.

[0242] P i This represents the relationship between the i-th row and the (i-1)-th row;

[0243] When P i When P >= 0, it indicates that the pixel road is continuous (i.e., it is a drivable road). i =-1 indicates that the pixel road is discontinuous, and P appears continuously. i The number of times the expression equals -1 is t1, and when t1 < T -1 Ignore discontinuities (where T) -1 (Pixel discontinuity threshold); processed continuously, P i Set to 0, and P appears consecutively. i The number of times the value equals 0 is t0. When t0 > T0 (where T0 is the intersection width threshold), an intersection appears in that frame of the image. The number of frames in which an intersection is continuously detected is t. When t > T0, an intersection appears in that frame. frame (where T) frame When a threshold (representing the number of consecutive frames detected at an intersection) is reached, an intersection is determined to have occurred, and the vehicle's direction θ at that moment is recorded. s And it begins to turn, with the vehicle's yaw angle changing by Δθ:

[0244] Δθ=abs(θ s -θ(t))

[0245] Where θ(t) represents the vehicle's yaw angle at time t.

[0246] abs represents the absolute value function;

[0247] When Δθ>0.8*θ i At that time, the vehicle completes the turn and enters the straight-line trend recognition module, θ i This refers to the intersection turning angle recorded during self-learning.

[0248] Based on the above method, drivable roads and intersections within a drivable area can also be identified, as shown in the specific diagram. Figure 10 As shown.

[0249] Step 2: Generate a path trajectory corresponding to the vehicle driving status information based on the distribution of the drivable roads and drivable intersections;

[0250] In the embodiments of this specification, the vehicle's travel route can be determined based on the distribution of the drivable roads and drivable intersections in the drivable area;

[0251] Based on the vehicle's driving route and driving status information, a path trajectory corresponding to the vehicle's driving status information is generated; this application uses this method to accurately obtain the path trajectory corresponding to the vehicle's driving status information.

[0252] In this embodiment of the specification, after generating the path trajectory corresponding to the vehicle driving status information based on the drivable area, the step may further include:

[0253] A map corresponding to the preset driving route is constructed based on the path trajectory.

[0254] In the embodiments of this specification, the map can be an underground parking garage map; specifically, the generated path trajectory can be processed using a trajectory abstraction algorithm to construct the underground parking garage map; the underground parking garage map constructed based on this method is an abstract path map; in this application, the map can be applied to any scenario, and can be used for autonomous driving in underground parking garages without the need for site-end equipment; and the underground parking garage map is a path planning map that conforms to the driver's driving habits.

[0255] In a specific embodiment of this specification, the path trajectory can be processed using the following method: specifically including:

[0256] Roads typically consist of five types: starting point, straight road, intersection, dead end, and ending point. Intersections are further divided into crossroads and T-junctions. When a vehicle approaches an intersection, it needs to make a turning decision to determine its path, with the current direction of travel serving as the default reference direction. An intersection structure is defined, including four parameters: intersection number (Node), distance (Dist), intersection turning information (Turn INF), and turning angle (Angle). Dist represents the distance between the current location and the starting point. Additionally, a separate "Pass Flag" is established, where "0" indicates continued travel and "1" indicates a dead end, prohibiting further travel.

[0257] The following table shows the driving situation in a specific embodiment:

[0258]

[0259] The table above shows a crossroads, where the default angle is a 90-degree left turn. When entering from direction ①, PassFlag = 0, indicating that you can continue. After entering, Turn INF is set to 1, and the intersection number Node is updated to the previous intersection number plus 1. Since a left turn is a dead end, the Pass Flag at this location is set to 1. When entering from direction ②, Pass Flag is set to 1, Turn INF is set to 2, and since it is the same intersection, Node remains unchanged. Since a left turn is a dead end, the Pass Flag at this location is set to 1 again. Entering from direction ③ is the same as ②. Entering from direction ④, which is a dead end, requires returning to the previous intersection. Finally, PassFlag = 1, and the intersection number Node is the previous intersection number.

[0260] In this embodiment of the specification, after generating the path trajectory corresponding to the vehicle driving status information based on the drivable area, the method further includes the following steps when driving on the preset driving path again:

[0261] Real-time acquisition of vehicle driving status information and initial images of the surrounding environment of the preset driving path corresponding to the vehicle driving status information.

[0262] In the embodiments described in this specification, the specific acquisition method is the same as described above;

[0263] Based on the initial image, a target top view corresponding to the initial image is calculated using a nonlinear difference correction algorithm;

[0264] The target top view is input into a preset deep learning model, and the pixels of the target top view input into the preset deep learning model are classified to obtain partitioned images, which include images of drivable areas and images of non-drivable areas.

[0265] Scan the partitioned image to identify the drivable area of ​​the vehicle;

[0266] Based on the drivable area, a current path trajectory corresponding to the vehicle driving status information is generated, and the path trajectory is one of the preset driving paths;

[0267] In the embodiments described in this specification, when the driver drives the vehicle on a preset driving path, the path trajectory obtained each time is different;

[0268] Specifically, when the vehicle travels on the preset driving path again, the same method for obtaining the path trajectory can be used to obtain the current path trajectory corresponding to the vehicle driving status information.

[0269] In this embodiment of the specification, after generating the current path trajectory corresponding to the vehicle driving status information based on the drivable area, the method further includes:

[0270] The current path trajectory and the previously obtained path trajectory are fused to reconstruct a map corresponding to the preset driving route;

[0271] In the embodiments of this specification, the spatial location of the same point in the current path trajectory and the previously obtained path trajectory are correlated to perform information fusion; a new path trajectory is obtained, and this method is used to verify the path trajectory to ensure that a more accurate abstract map is obtained, such as a parking garage map.

[0272] Preferably, in the embodiments of this specification, the preset driving route can be driven repeatedly to obtain at least three route trajectories;

[0273] The current path trajectory obtained each time is fused with the previously obtained path trajectory or the new path trajectory to obtain a more accurate map.

[0274] In a specific embodiment of this specification, the least squares method can be used to solve the problem and achieve multi-trajectory fusion; the details are as follows:

[0275] For example, the driving trajectory learned in the first learning is used as the reference point set X, and the driving trajectory learned in the second learning is used as the point set P to be merged. The reference point set X and the point set P to be merged are as follows:

[0276] X = {x1, x2, ..., x} n}

[0277] P = {p1, p2, ..., p} n}

[0278] Rotate and translate the point set P to obtain the target error function:

[0279]

[0280] Where E(R,t) represents the error function, and R represents the rotation matrix.

[0281] t represents the translation matrix, N p This represents the number of elements in the point set P.

[0282] Specifically, the centroids of the reference point set X and the point set P to be merged are:

[0283]

[0284]

[0285] Where, μ x N represents the centroid of the reference point set X. x --The number of elements in the reference point set X,

[0286] μ p Denotes the centroid of the point set P to be merged.

[0287] N p This represents the number of elements in the set P of points to be merged.

[0288] Therefore:

[0289] X′={x i -μ x}={x1-μ x ,x2-μ x ,……,x n -μ x}={x i ′}

[0290] P′={p i -μ p}={p1-μ p p2-μ p ,……,p n -μ p}={p i ′}

[0291] Where X′ represents the set of deviations between each element in the reference point set X and the centroid.

[0292] P′ represents the set of deviations between each element in the set of points to be merged and the centroid.

[0293] Find the optimal transformation using Singular Value Decomposition (SVD):

[0294]

[0295] Where W represents the real matrix to be decomposed, p i ′ T p i The transpose of ′

[0296] U and V are identity orthogonal matrices, and are called left and right singular matrices, respectively.

[0297] V T Let V denote the transpose of V, and σ1, σ2, σ3 be the singular values.

[0298] When rank(W) = 3, the optimal solution of E(R,t) is unique, and the values ​​of U and V can be solved.

[0299] Therefore, the rotation matrix R and the translation matrix t are respectively:

[0300] R = UV T

[0301] t = μ x -Rμ P

[0302] Substituting the rotation matrix R and translation matrix t into the target error function E(R,t), when the obtained target error function E(R,t) is sufficiently convergent, the fusion effect of the two point sets is as follows. Figure 11 As shown.

[0303] In this embodiment of the specification, before performing multi-track fusion of the current path trajectory and the previously obtained path trajectory to reconstruct the underground parking garage map, the method further includes:

[0304] H1, determine whether the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to a preset first threshold;

[0305] In the embodiments described in this specification, the preset first threshold can be 95%;

[0306] H2, if the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to a preset first threshold, then the current path trajectory and the previously obtained path trajectory are merged.

[0307] In the embodiments of this specification, the current path trajectory and the previously obtained path trajectory can be fused when the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to a preset first threshold.

[0308] H3, if the overlap between the current path trajectory and the previously obtained path trajectory is less than a preset first threshold, then determine whether the matching degree between the current path trajectory and the preset driving path is less than or greater than the matching degree between the previously obtained path trajectory and the preset driving path.

[0309] In this embodiment of the specification, if the overlap between the current path trajectory and the previously obtained path trajectory is less than a preset first threshold, then the path trajectory with fewer target top views obtained during the generation process of the current path trajectory and the previously obtained path trajectory can be abandoned.

[0310] Specifically, the matching degree between the current path trajectory and the preset driving path can be used to determine the path trajectory with fewer target top views obtained during the previous path trajectory generation process.

[0311] H4, if so, then regenerate the current path trajectory.

[0312] Specifically, in the embodiments of this specification, if the matching degree between the current path trajectory and the preset driving path is less than the matching degree between the previously obtained path trajectory and the preset driving path, the current path trajectory is abandoned, and the vehicle is driven on the preset driving path again to obtain a new current path trajectory; so that the new current path trajectory can be used to perform multi-trajectory fusion with the previously obtained path trajectory to reconstruct the path trajectory.

[0313] Specifically, the map corresponding to the preset driving route can be reconstructed based on the reconstructed path trajectory.

[0314] In another embodiment of this specification, if the matching degree between the current path trajectory and the preset driving path is greater than the matching degree between the previously obtained path trajectory and the preset driving path, then the previously obtained path trajectory is abandoned, and the vehicle is driven on the preset driving path again to obtain a new current path trajectory; so that the new current path trajectory can be used to perform multi-trajectory fusion with the previously obtained current path trajectory to reconstruct the path trajectory.

[0315] Specifically, the map corresponding to the preset driving route can be reconstructed based on the reconstructed path trajectory.

[0316] The path trajectory obtained by the above method in this application is closer to the actual driving trajectory; it can not only improve the smoothness of vehicle control in autonomous driving, but also reduce the risk of the vehicle deviating from the predetermined trajectory.

[0317] As can be seen from the embodiments of the path construction method, apparatus, terminal, and storage medium provided by the present invention, when driving on a preset driving path, the embodiments of the present invention acquire vehicle driving status information and an initial image of the surrounding environment of the preset driving path corresponding to the vehicle driving status information in real time; based on the initial image, a target top view corresponding to the initial image is calculated through a nonlinear difference correction algorithm; the target top view is input into a preset deep learning model, and the pixels of the target top view input into the preset deep learning model are classified to obtain partitioned images, the partitioned images including drivable area images and non-drivable area images; the partitioned images are scanned to identify the drivable area of ​​the vehicle; a path trajectory corresponding to the vehicle driving status information is generated based on the drivable area, the path trajectory being one of the preset driving paths; using the technical solution provided by the embodiments of this specification, the vehicle automatically learns to obtain a path trajectory during the process of driving on a preset driving path, so that the vehicle can automatically plan a path in the subsequent autonomous driving process.

[0318] This invention also provides a path construction apparatus, such as... Figure 12 As shown, this is a schematic diagram of a path construction device provided in an embodiment of the present invention; specifically, the device includes:

[0319] The first acquisition module 110 is used to acquire vehicle driving status information and an initial image of the surrounding environment of the preset driving path in real time when driving on a preset driving path.

[0320] The target top view acquisition module 120 is used to calculate the target top view corresponding to the initial image based on the initial image using a nonlinear difference correction algorithm.

[0321] The partitioned image acquisition module 130 is used to input the target top view into a preset deep learning model, classify the pixels of the target top view input into the preset deep learning model, and obtain partitioned images, wherein the partitioned images include drivable area images and non-drivable area images.

[0322] The identification module 140 is used to scan the partition image and identify the drivable area of ​​the vehicle;

[0323] The path trajectory generation module 150 is used to generate a path trajectory corresponding to the vehicle driving status information based on the drivable area, wherein the path trajectory is one of the preset driving paths.

[0324] In the embodiments of this specification, a map building module is also included, which is used to build a map corresponding to the preset driving route based on the path trajectory.

[0325] In this embodiment of the specification, the first acquisition module 110 includes:

[0326] The first acquisition unit is used to acquire vehicle driving status information in real time during the process of the vehicle driving on the preset driving path. The vehicle driving status information includes the driving strategy of the vehicle and the driver's driving habits during the process of the vehicle driving on the preset driving path.

[0327] The second acquisition unit is used to acquire, in real time, an initial image of the environment surrounding the preset driving path during the vehicle's driving on the preset driving path, based on the driving strategy and the driver's driving habits.

[0328] In this embodiment of the specification, the first acquisition unit includes:

[0329] The first acquisition subunit is used to acquire the vehicle speed and steering wheel angle in real time;

[0330] The first determining subunit is used to determine the vehicle's forward distance and heading angle based on the vehicle's speed and the steering wheel angle.

[0331] The second determining subunit is used to determine the vehicle's driving strategy based on the vehicle's forward mileage and the heading angle. The vehicle's driving strategy includes the forward mileage on the drivable road and whether to turn at the drivable intersection.

[0332] In this embodiment of the specification, the first acquisition unit further includes:

[0333] The second acquisition subunit is used to acquire real-time operating data of the vehicle during its journey along a preset driving path;

[0334] The third acquisition subunit is used to preprocess the vehicle's operating data to acquire target operating data;

[0335] A feature extraction subunit is used to input the target running data into a recurrent neural network and extract features from the target running data from the recurrent neural network.

[0336] The driver's driving habits determination subunit is used to input the features into a fully connected network to predict the driver's driving habits during the vehicle's journey along a preset driving path; the driving habits include the driving speed on drivable roads and the turning angle at drivable intersections.

[0337] In the embodiments described in this specification, the following are also included:

[0338] The second acquisition module is used to acquire vehicle driving status information and an initial image of the surrounding environment of the preset driving path corresponding to the vehicle driving status information in real time when driving on the preset driving path again.

[0339] The target top view acquisition module is used to calculate the target top view corresponding to the initial image based on the initial image using a nonlinear difference correction algorithm;

[0340] The partitioned image acquisition module is used to input the target top view into a preset deep learning model, classify the pixels of the target top view input into the preset deep learning model, and obtain partitioned images, wherein the partitioned images include drivable area images and non-drivable area images.

[0341] The recognition module is used to scan the partition image and identify the drivable area of ​​the vehicle;

[0342] The current path trajectory generation module is used to generate a current path trajectory corresponding to the vehicle driving status information based on the drivable area, wherein the path trajectory is one of the preset driving paths;

[0343] In the embodiments of this specification, it further includes: a map reconstruction module, used to perform multi-track fusion of the current path trajectory and the previously obtained path trajectory to reconstruct the underground parking garage map.

[0344] In the embodiments described in this specification, the following are also included:

[0345] The overlap determination module is used to determine whether the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to a preset first threshold.

[0346] The trajectory fusion module is used to fuse the current path trajectory with the previously obtained path trajectory if the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to the preset first threshold.

[0347] In the embodiments described in this specification, the following are also included:

[0348] The matching degree judgment module is used to determine whether the matching degree between the current path trajectory and the preset driving path is less than the matching degree between the previously obtained path trajectory and the preset driving path if the overlap between the current path trajectory and the previously obtained path trajectory is less than the preset first threshold.

[0349] The current path trajectory reconstruction module is used to regenerate the current path trajectory if the matching degree between the current path trajectory and the preset driving path is less than the matching degree between the previously obtained path trajectory and the preset driving path.

[0350] In this embodiment of the specification, the target top view acquisition module 120 includes:

[0351] A target image acquisition unit is used to obtain a target image based on the initial image, wherein the target image includes a top view of a region image that overlaps with the region where the target top view is located;

[0352] The frequency acquisition unit is used to acquire the number of times the target image appears;

[0353] The judgment unit is used to determine whether the number of times the target image appears is greater than or equal to a preset second threshold.

[0354] A feature point extraction unit is used to extract feature points of the region image in each of the target images if the condition is met.

[0355] The feature point matching unit is used to match the feature points of each region image to reconstruct the target top view.

[0356] In this embodiment of the specification, the target top view acquisition module 120 further includes:

[0357] The correspondence acquisition unit is used to acquire the correspondence between the top view of the initial image and the initial image based on the nonlinear difference correction algorithm. The correspondence includes the coordinate points corresponding to the top view of the initial image and the initial image.

[0358] A target coordinate point acquisition unit is used to acquire target coordinate points from the initial image based on the correspondence.

[0359] The target top view construction unit is used to construct a target top view corresponding to the initial image based on the target coordinate points.

[0360] In this embodiment of the specification, the path trajectory generation module 150 includes:

[0361] The first determining unit is configured to determine, based on the drivable area, the drivable roads, drivable intersections, and the distribution of the drivable roads and drivable intersections within the drivable area;

[0362] The path trajectory generation unit is used to generate a path trajectory corresponding to the vehicle driving status information based on the distribution of the drivable roads and the drivable intersections.

[0363] In the embodiments described in this specification, the following are also included:

[0364] The scanning unit is used to scan the partitioned image using a grid of preset size to obtain the drivable area and the scanning area of ​​the vehicle.

[0365] An adjustment unit is used to adjust the drivable area based on the scanned area and reconstruct the drivable area.

[0366] In the embodiments described in this specification, the adjustment unit includes:

[0367] The first adjustment subunit is used to perform an expansion operation on the drivable area based on the scanned area to obtain an expanded area;

[0368] The second adjustment subunit is used to perform an erosion operation on the expanded region based on the scanned region to reconstruct the drivable region.

[0369] In this embodiment of the specification, the first determining unit includes:

[0370] The first identification subunit is used to input the drivable area into the road identification model and identify the drivable roads in the drivable area and the information of the drivable roads, wherein the information of the drivable roads includes the width and length of the drivable roads;

[0371] The second identification subunit is used to input the drivable area into the intersection identification model and identify the drivable intersections in the drivable area and the type of the drivable intersections.

[0372] The third determining subunit is used to determine the distribution of the drivable roads and drivable intersections in the drivable area based on the information of the drivable area, the drivable roads in the drivable area, and the types of drivable intersections.

[0373] This invention provides a path construction terminal, which includes a processor and a memory. The memory stores at least one instruction or at least one program segment. The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the path construction method as described in the above method embodiments.

[0374] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0375] Figure 13This is a schematic diagram of a path building terminal provided in an embodiment of the present invention. The internal structure of the path building terminal may include, but is not limited to, a processor, a network interface, and a memory. The processor, network interface, and memory within the path building terminal can be connected via a bus or other means, as shown in the embodiments of this specification. Figure 13 Taking the example of a connection between China and Israel via a bus.

[0376] The processor (or CPU, Central Processing Unit) is the computing and control core of the path-building terminal. The network interface may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.). The memory is the storage device in the path-building terminal, used to store programs and data. It is understood that the memory here can be a high-speed RAM storage device, or a non-volatile memory device, such as at least one disk storage device; optionally, it can also be at least one storage device located remotely from the aforementioned processor. The memory provides storage space, which stores the operating system of the path-building terminal, including but not limited to: Windows (an operating system), Linux (an operating system), etc., which are not limited in this invention; and the storage space also stores one or more instructions suitable for loading and execution by the processor, which can be one or more computer programs (including program code). In the embodiments of this specification, the processor loads and executes one or more instructions stored in the memory to implement the path-building method provided in the above method embodiments.

[0377] Embodiments of the present invention also provide a computer-readable storage medium, which can be disposed in a path building terminal to store at least one instruction, at least one program, code set, or instruction set related to implementing a path building method in the method embodiments. The at least one instruction, the at least one program, the code set, or the instruction set can be loaded and executed by the processor of an electronic device to implement the path building method provided in the above method embodiments.

[0378] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0379] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0380] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and server embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0381] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0382] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A path construction method, characterized in that: The method includes: When driving on a preset driving path, the vehicle driving status information is acquired in real time. The vehicle driving status information includes the driving strategy of the vehicle and the driver's driving habits during the driving process on the preset driving path. The driving strategy includes the distance traveled on the drivable road and whether to turn at the drivable intersection. Based on the driving strategy and the driver's driving habits, the initial image of the environment around the preset driving path is acquired in real time during the vehicle's driving on the preset driving path, wherein the driving habits include the driving speed on the drivable road and the turning angle at the drivable intersection. Based on the initial image, a target top view corresponding to the initial image is calculated using a nonlinear difference correction algorithm; The target top view is input into a preset deep learning model, and the pixels of the target top view are classified to obtain a partitioned image. The partitioned image includes a drivable area image and a non-drivable area image, wherein the drivable area includes the drivable road and the drivable intersection. Scan the partitioned image to identify the drivable area of ​​the vehicle; A path trajectory corresponding to the vehicle's driving status information is generated based on the drivable area, and the path trajectory is one of the preset driving paths.

2. The path construction method according to claim 1, characterized in that: After generating the path trajectory corresponding to the vehicle driving status information based on the drivable area, the process further includes: A map corresponding to the preset driving route is constructed based on the path trajectory.

3. The path construction method according to claim 1, characterized in that: Obtaining the driving strategy of the vehicle while driving on the preset driving path includes: Real-time acquisition of vehicle speed and steering wheel angle; The vehicle's forward distance and heading angle are determined based on the vehicle's speed and the steering wheel angle. The vehicle's driving strategy is determined based on the vehicle's forward distance and heading angle.

4. The path construction method according to claim 1, characterized in that: The drivable area includes drivable roads and drivable intersections; Acquiring the driver's driving habits while the vehicle is traveling along a preset driving path includes: Real-time acquisition of vehicle operation data during the driving process along a preset driving path; The vehicle's operating data is preprocessed to obtain target operating data; The target running data is input into a recurrent neural network, and features of the target running data are extracted from the recurrent neural network. The features are input into a fully connected network to predict the driver's driving habits during the vehicle's journey along a preset driving path.

5. The path construction method according to claim 1, characterized in that: After generating the path trajectory corresponding to the vehicle's driving status information based on the drivable area, the process further includes: when driving again on the preset driving path... Real-time acquisition of vehicle driving status information and initial images of the surrounding environment of the preset driving path corresponding to the vehicle driving status information. Based on the initial image, a target top view corresponding to the initial image is calculated using a nonlinear difference correction algorithm; The target top view is input into a preset deep learning model, and the pixels of the target top view input into the preset deep learning model are classified to obtain partitioned images, which include images of drivable areas and images of non-drivable areas. Scan the partitioned image to identify the drivable area of ​​the vehicle; Based on the drivable area, a current path trajectory corresponding to the vehicle's driving status information is generated, and the path trajectory is one of the preset driving paths.

6. The path construction method according to claim 5, characterized in that: After generating the current path trajectory corresponding to the vehicle driving status information based on the drivable area, the process further includes: The current path trajectory and the previously obtained path trajectory are fused to reconstruct a map corresponding to the preset driving route.

7. The path construction method according to claim 6, characterized in that: Before performing multi-track fusion on the current path trajectory and the previously obtained path trajectory to reconstruct the map corresponding to the preset driving route, the method further includes: Determine whether the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to a preset first threshold. If the overlap between the current path trajectory and the previously obtained path trajectory is greater than or equal to a preset first threshold, then the current path trajectory and the previously obtained path trajectory are merged.

8. The path construction method according to claim 7, characterized in that: Also includes: If the overlap between the current path trajectory and the previously obtained path trajectory is less than a preset first threshold, then it is determined whether the matching degree between the current path trajectory and the preset driving path is less than the matching degree between the previously obtained path trajectory and the preset driving path. If so, then regenerate the current path trajectory.

9. The path construction method according to claim 1, characterized in that: The step of calculating the target top view corresponding to the initial image using a nonlinear difference correction algorithm based on the initial image includes: A target image is obtained based on the initial image, wherein the target image includes a top view of a region image that overlaps with the region where the target top view is located; Obtain the number of times the target image appears; Determine whether the number of times the target image appears is greater than or equal to a preset second threshold; If so, then extract the feature points of the region image in each of the target images; The feature points of each region image are matched to reconstruct the top view of the target.

10. The path construction method according to claim 1, characterized in that: The step of calculating the target top view corresponding to the initial image using a nonlinear difference correction algorithm based on the initial image further includes: The nonlinear difference correction algorithm is used to obtain the correspondence between the top view of the initial image and the initial image, and the correspondence includes the coordinate points between the top view of the initial image and the corresponding points between the initial images; The target coordinate points are obtained from the initial image based on the correspondence. Construct a top view of the target corresponding to the initial image based on the target coordinate points.

11. The path construction method according to claim 1, characterized in that: The step of generating a path trajectory corresponding to the vehicle's driving status information based on the drivable area includes: Based on the drivable area, determine the drivable roads, drivable intersections, and the distribution of the drivable roads and drivable intersections within the drivable area; Based on the distribution of the drivable roads and intersections, a path trajectory corresponding to the vehicle's driving status information is generated.

12. The path construction method according to claim 11, characterized in that: Before determining the drivable roads, drivable intersections, and their distribution within the drivable area, the method further includes: The partitioned image is scanned using squares of a preset size to obtain the drivable area and the scanned area of ​​the vehicle; Based on the scanned area, the drivable area is adjusted and reconstructed.

13. The path construction method according to claim 12, characterized in that: The step of adjusting and reconstructing the drivable area based on the scanned area includes: Based on the scanned area, the drivable area is expanded to obtain the expanded area; Based on the scanned area, an erosion operation is performed on the expanded area to reconstruct the drivable area.

14. The path construction method according to claim 11, characterized in that: Determining the drivable roads, drivable intersections, and their distribution within the drivable area includes: The drivable area is input into the road recognition model to identify the drivable roads in the drivable area and the information of the drivable roads, including the width and length of the drivable roads; The drivable area is input into the intersection recognition model to identify the drivable intersections in the drivable area and the type of the drivable intersections; Based on the information of the drivable area, the drivable roads within the drivable area, and the types of the drivable intersections, the distribution of the drivable roads and the drivable intersections within the drivable area is determined.

15. A path construction apparatus, characterized in that: The device includes: The first acquisition module is used to acquire, in real time, vehicle driving status information during the driving process on the preset driving path. The vehicle driving status information includes the vehicle's driving strategy and the driver's driving habits during the driving process on the preset driving path. The driving strategy includes the distance traveled on the drivable road and whether to turn at the drivable intersection. Based on the driving strategy and the driver's driving habits, the module acquires an initial image of the environment surrounding the preset driving path in real time during the driving process on the preset driving path. The driving habits include the driving speed on the drivable road and the turning angle at the drivable intersection. The target top view acquisition module is used to calculate the target top view corresponding to the initial image based on the initial image using a nonlinear difference correction algorithm; The partitioned image acquisition module is used to input the target top view into a preset deep learning model, classify the pixels of the target top view input into the preset deep learning model, and obtain partitioned images. The partitioned images include drivable area images and non-drivable area images, wherein the drivable area includes the drivable road and the drivable intersection. The recognition module is used to scan the partition image and identify the drivable area of ​​the vehicle; The path trajectory generation module is used to generate a path trajectory corresponding to the vehicle driving status information based on the drivable area, wherein the path trajectory is one of the preset driving paths.

16. A path construction terminal, characterized in that: The terminal includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the path construction method as described in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor using the path construction method as described in any one of claims 1 to 14.

Images