Lane positioning method and device, terminal equipment and computer readable storage medium

Abstract

The invention is suitable for the technical field of lane positioning, and provides a lane positioning method and device, terminal equipment and a computer readable storage medium, and the method comprises the steps: obtaining first information of a target road according to a first image, the first image being a shot image of a road on which a target vehicle currently travels, the first information comprises geometric information of a lane line of the target road, determining first lane information of a lane where the target vehicle is currently located in the target road according to the first image and preset information, and performing information fusion processing according to the first information, the first lane information and current driving data of the target vehicle, and according to the second information, re-determining second lane information of the lane where the target vehicle is located in the target road at present. By using the method, the accurate positioning of the lane can be improved under the condition of no high-precision map.

Description

Technical Field

[0001] The present application belongs to the technical field of lane positioning, and in particular, relates to a lane positioning method, apparatus, terminal equipment and computer-readable storage medium. Background Art

[0002] With the advancement of science in today's world, artificial intelligence is also developing rapidly. Autonomous driving technology is also becoming the mainstream research and development technology in the automotive industry. However, to achieve autonomous driving of cars, the accurate positioning performance of the vehicle must be improved.

[0003] The lane positioning method in the prior art generally relies on high-precision maps, for example, lane positioning is performed based on the geometric information of lane lines in high-precision maps. However, due to the large amount of data and high maintenance costs of high-precision maps, they cannot be applied on a large scale. Summary of the invention

[0004] The embodiments of the present application provide a lane positioning method, apparatus, terminal device, and computer-readable storage medium, which can improve lane positioning accuracy in the absence of a high-precision map.

[0005] In a first aspect, an embodiment of the present application provides a lane positioning method, including:

[0006] Obtaining first information of a target road according to a first image, wherein the target road is a road on which a target vehicle is currently traveling, the first image is a photographed image of the road on which the target vehicle is currently traveling, and the first information includes geometric information of all lane lines of the target road;

[0007] Determine first lane information of the lane where the target vehicle is currently located on the target road according to the first image and preset information, wherein the preset information includes the lane information of the target road;

[0008] Performing information fusion processing according to the first information, the first lane information, and the current driving data of the target vehicle to obtain second information after fusion processing;

[0009] The second lane information of the lane where the target vehicle is currently located in the target road is re-determined according to the second information.

[0010] In the embodiment of the present application, firstly, the geometric information (i.e., the first information) of all lane lines of the road where the target vehicle is located can be obtained according to the photographed image (i.e., the first image) of the road where the target vehicle is currently traveling. Since the first information is obtained according to the first image, rather than from the high-precision map, the method in the embodiment of the present application can be separated from the dependence on the high-precision map, reduce the amount of data processing, and reduce the maintenance cost; then, the first lane information of the lane where the target vehicle is currently located in the target road can be obtained according to the first information and the preset information. The first lane information includes the lane identification obtained by the lane number of the target vehicle, etc., which is equivalent to realizing the first lane positioning based on the image and the preset information. Secondly, information fusion processing is performed according to the first information, the first lane information, and the current driving data of the target vehicle (including vehicle speed, angular velocity, etc.) to obtain the second information after fusion processing, and finally the second lane information of the lane where the target vehicle is currently located is determined again according to the second information. The second lane information is fused according to multiple data to re-determine the current lane information of the target vehicle, which is equivalent to combining multiple data to perform lane positioning again on the basis of obtaining the first lane information only according to the image information and the preset information for the first time, thereby improving the accuracy of lane positioning to make up for the defect of low positioning accuracy caused by not using high-precision maps. The second lane positioning uses the geometric information of all lane lines of the road where the target vehicle is currently located and the vehicle's own driving data for positioning. It can achieve accurate positioning of the target vehicle without relying on high-precision maps.

[0011] In a possible implementation manner of the first aspect, determining first lane information of a lane in which the target vehicle is currently located on the target road according to the first image and preset information includes:

[0012] Segmenting the first image into a first sub-image and a second sub-image along a height direction of the first image, wherein the first sub-image includes information related to the lane in the target road, and the second sub-image does not include information related to the lane of the target road;

[0013] First lane information of a lane where the target vehicle is currently located on the target road is determined according to the first sub-image and the preset information.

[0014] In a possible implementation manner of the first aspect, determining first lane information of a lane in which the target vehicle is currently located on the target road according to the first sub-image and the preset information includes:

[0015] Performing image detection processing according to the first sub-image and the preset information to obtain a detection result, wherein the detection result includes at least one first lane number and a confidence level corresponding to each first lane number, and the confidence level indicates a probability that the lane number of the target vehicle currently located in the target road is the first lane number;

[0016] The first lane information is determined from at least one first lane sequence number of the detection result according to the confidence level in the detection result.

[0017] In a possible implementation manner of the first aspect, re-determining second lane information of a lane in which the target vehicle is currently located on the target road according to the second information includes:

[0018] Acquire historical information, where the historical information includes N historical positions before the current position of the target vehicle, where N is a positive integer;

[0019] The second lane information of the lane where the target vehicle is currently located in the target road is re-determined according to the historical information and the second information.

[0020] In a possible implementation manner of the first aspect, re-determining second lane information of a lane in which the target vehicle is currently located on the target road according to the historical information and the second information includes:

[0021] Obtain a first probability of a candidate number corresponding to each of the historical positions in the historical information, wherein the candidate number represents the lane number of the lane where the next candidate position of the historical position is located in the road, and the first probability represents the probability of the candidate number corresponding to the target vehicle at the historical position.

[0022] Obtaining a second probability between candidate serial numbers corresponding to each of two adjacent historical positions in the historical information, wherein the second probability represents a probability that the target vehicle transfers between lanes corresponding to the two candidate serial numbers;

[0023] Second lane information of a lane in which the target vehicle is currently located on the target road is determined according to the first probability and the second probability.

[0024] In a possible implementation manner of the first aspect, determining second lane information of a lane in which the target vehicle is currently located on the target road according to the first probability and the second probability includes:

[0025] According to each candidate position of each of the historical positions, M groups of routes are obtained by arranging and combining, wherein each group of routes includes the candidate positions of each of the N historical positions;

[0026] Calculate the third probability of each of the M groups of routes according to the first probability and the second probability, wherein the third probability of the i-th group of routes represents the probability that the target vehicle travels along the i-th group of routes, and i is a positive integer less than or equal to M;

[0027] The second lane information is determined according to the third probability.

[0028] In an implementation manner of the first aspect, determining the second lane information according to the third probability includes:

[0029] Determine a group of routes corresponding to the maximum value of the third probabilities of the M groups of routes as the target routes;

[0030] The second lane information is determined according to a candidate position of the last historical position in the target route.

[0031] In a second aspect, an embodiment of the present application provides a lane positioning device, including:

[0032] a first acquisition unit, configured to obtain first information of a target road according to a first image, wherein the target road is a road on which a target vehicle is currently traveling, the first image is a photographed image of the road on which the target vehicle is currently traveling, and the first information includes geometric information of all lane lines of the target road;

[0033] A first determining unit, configured to determine first lane information of a lane in which the target vehicle is currently located on the target road according to the first image and preset information, wherein the preset information includes the lane information of the target road;

[0034] A second acquisition unit is used to perform information fusion processing according to the first information, the first lane information and the current driving data of the target vehicle to obtain second information after fusion processing;

[0035] A second determining unit is used to re-determine second lane information of a lane where the target vehicle is currently located in the target road according to the second information.

[0036] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a lane positioning method as described in any one of the first aspects above is implemented.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the lane positioning method as described in any one of the above-mentioned first aspects is implemented.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the lane positioning method described in any one of the above-mentioned first aspects.

[0039] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 It is a system flow diagram of a lane positioning method provided by an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of lane numbers of a lane positioning method provided in an embodiment of the present application;

[0043] Figure 3 is a schematic diagram of a process for obtaining first lane information provided by an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a process for obtaining a segmented image provided by an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of a process for obtaining second lane information provided by an embodiment of the present application;

[0046] Figure 6 is a schematic diagram of lane positioning in a post-fusion module provided by an embodiment of the present application;

[0047] Figure 7 It is a block diagram of a lane positioning system based on an advanced navigation map provided in an embodiment of the present application;

[0048] Figure 8 is a structural block diagram of a lane positioning device based on advanced navigation provided in one embodiment of the present application;

[0049] Fig. 9 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0051] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0052] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0054] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0055] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0056] With the advancement of science in today's world, artificial intelligence is also developing rapidly. At present, the traditional automobile industry is relying on the development of artificial intelligence and is vigorously developing unmanned driving technology. Unmanned driving technology plays an outstanding role in reducing driving intensity and improving driving safety. Therefore, the automobile industry has also been committed to the research and development of unmanned driving technology.

[0057] On open roads in cities, to achieve automatic driving of cars, it is necessary to know the position (position and posture) of the vehicle in the world coordinate system (global GPS coordinate system or other coordinate systems that can uniquely represent any position in the world), that is, vehicle positioning technology. Lane positioning technology is divided into three parts: road positioning, lane positioning, and lane positioning. The existing commonly used lane positioning technology generally relies on high-precision maps. High-precision maps are a kind of digital maps with high precision and high detail. They are used to provide accurate geographic information and navigation guidance. Compared with traditional maps, high-precision maps are more accurate and detailed, and contain rich information such as roads, traffic signs, lanes, and intersection buildings. It not only provides geometric information of roads, but also includes the number, width, shape of lane lines, location and meaning of traffic signs, and even real-time changing information such as the status of traffic signals. However, the amount of data in high-precision maps is large, and the maintenance cost is high, which makes it impossible to apply them on a large scale.

[0058] In order to solve the above problems, an embodiment of the present application proposes a lane positioning technology. In the embodiment of the present application, first, the geometric information (i.e., the first information) of all lane lines of the road where the target vehicle is located can be obtained based on the captured image of the road in front of the target vehicle (i.e., the first image), and then the first lane information of the lane where the target vehicle is currently located in the target road can be obtained based on the first information and the preset information. Secondly, information fusion processing is performed based on the first information, the first lane information, and the current driving data of the target vehicle (including vehicle speed, angular velocity, etc.) to obtain the second information after fusion processing, and finally the second lane information of the lane where the target vehicle is currently located is determined again based on the second information. Through the above method, the target vehicle can be accurately positioned without relying on high-precision maps.

[0059] See also Figure 1 , is a system flow diagram of a lane positioning method provided by an embodiment of the present application, such as Figure 1 As shown, by way of example and not limitation, the method comprises the following steps:

[0060] Step S101, obtaining first information of a target road according to a first image, wherein the target road is a road on which a target vehicle is currently traveling, the first image is a photographed image of the road on which the target vehicle is currently traveling, and the first information includes geometric information of lane lines of the target road.

[0061] In the embodiments of the present application, the target vehicle refers to an autonomous driving vehicle, which can perform various driving operations and decisions without driver intervention. Its appearance is similar to that of an ordinary car, but it is equipped with various perceptions, computers and control systems inside to achieve the function of autonomous driving. In some application scenarios, the interior of the autonomous driving vehicle is usually equipped with one or more forward-looking cameras to obtain visual information in front of the vehicle to assist in environmental perception and driving decisions. The forward-looking camera is a high-resolution camera installed on the front of the vehicle, which can capture image data of targets such as the road ahead, traffic signs, vehicles, pedestrians, etc. The image data captured is the first image in this application. By recognizing the first image, the geometric information of all lane lines in front of the target vehicle can be obtained.

[0062] The geometric information of lane lines refers to the geometric properties and features of lane lines represented in images or sensor data. This information can help the autonomous driving system to identify roads, keep lanes, and plan paths. For example, the geometric information of lane lines may include line type, line segment length, line segment width, intersection relationship between line segments, and intersection point location. The autonomous driving system can analyze this information to perform operations such as lane detection, lane keeping, and path planning to achieve accurate autonomous driving functions.

[0063] Step S102: determining first lane information of a lane in which the target vehicle is currently located on the target road according to the first image and preset information, wherein the preset information includes the lane information of the target road.

[0064] In the embodiments of the present application, the navigation map refers to a map that provides road-level navigation. It mainly includes road geometry information (such as the length, width, and intersection information of the road edge line), road topology, traffic lights, and other information. The preset information in this application refers to the information contained in the advanced navigation map. The advanced navigation map is a map that adds road change point information on the basis of the navigation map. The road change point information contains the high-precision location of the change point, such as the number of lanes, lane width, lane topology, etc.

[0065] It is worth noting that the difference between the advanced navigation map and the high-precision navigation map is that the advanced navigation map only contains the geometric information of the road, but not the geometric information of the lane lines marked on the road. In other words, the accuracy of the advanced navigation map is lower than that of the high-precision navigation map, but the data processing required for the advanced navigation map is less than that required for the high-precision navigation map.

[0066] In the embodiment of the present application, obtaining the geometric information of the lane line from the first image instead of using a high-precision navigation map to obtain the geometric information of the lane line can effectively reduce the amount of data processing, thereby helping to improve the processing efficiency of lane positioning.

[0067] In the embodiment of the present application, lane information may include lane direction, lane number, lane type, lane sequence number, lane width, lane restriction, etc. This information is very important for the autonomous driving system and can help determine the position of the vehicle on the road, select the appropriate lane, and plan the driving path.

[0068] Lane numbers describe the order of each lane on the road in the same driving direction so that the navigation system can accurately select and indicate the lane the vehicle is in. Lane numbers are usually identified by numbers, increasing from left to right. For example, the leftmost side of the road may be marked 1, and then gradually increase to the right.

[0069] For example, refer to Figure 2 , is a schematic diagram of the lane number of the lane positioning method provided by an embodiment of the present application, such as Figure 2 As shown, the road has four lanes, which can be divided into lane line 21 and lane line 22 according to the type of lane line. Lane line 21 is a solid line used to distinguish the driving direction; lane line 22 is a dotted line used to distinguish different lanes driving in the same direction. Figure 2 For each driving direction, each lane can be numbered in the order from left to right in the driving direction. Figure 2 As shown, for the two lanes on the right, the lane numbers of the two lanes are 1 and 2 in the order of their driving directions from left to right. For the two lanes on the left, the lane numbers of the two lanes are 1 and 2 in the order of their driving directions from left to right.

[0070] It should be noted that Figure 2 This is just an example of lane numbering. In other application scenarios, all lanes in different driving directions can also be numbered in sequence. For example, Figure 2 There are 4 lanes in the two driving directions shown, which are numbered 1, 2, 3 and 4 in order from west to east. The numbering rules of the lane numbers are not specifically limited in the embodiments of the present application.

[0071] In one embodiment, see Figure 3 , is a schematic diagram of a process for obtaining first lane information provided by an embodiment of the present application, such as Figure 3 As shown, step S102 includes:

[0072] Step S301: segment the first image into a first sub-image and a second sub-image along a height direction of the first image, wherein the first sub-image includes information related to the lanes in the target road, and the second sub-image does not include information related to the lanes of the target road.

[0073] One implementation method for obtaining the first lane information is to directly recognize the first image without performing any processing.

[0074] In an embodiment of the present application, another implementation method for obtaining the first lane information is to crop the first image. Specifically, since the first image obtained by the front-view camera contains information of multiple scenes such as roads and lanes, vehicles and pedestrians, and building obstacles, in order to improve the positioning accuracy of the road, the first image can be segmented, retaining the geometric information closest to the road lane line, and removing information such as buildings that are irrelevant to lane positioning. Through correlation analysis, the image containing lane lines, boundary lines, etc. that are more relevant to the target road is determined as the first sub-image, and the rest (such as the image part containing buildings, people, trees, etc. that are less relevant to the target road) is determined as the second sub-image.

[0075] See also Figure 4 , is a flow chart of obtaining a segmented image provided by an embodiment of the present application. Figure 4 As shown in (a), the first image includes the lane line in the near distance and the building in the far distance. Among them, the building part has little correlation with the lane line, and the building may affect the accuracy of lane positioning in the lane positioning algorithm. Therefore, in order to improve the accuracy of lane positioning, the first image can be divided into two sub-images along the height direction. Figure 4 As shown in (b) in FIG, the second sub-image does not contain information related to the lane line, but only contains distant buildings; Figure 4 As shown in (c) in FIG. 5 , the first sub-image contains information related to lane lines, but information such as buildings is filtered out.

[0076] The first lane information can be obtained using the preset information, but since the first image contains information irrelevant to the lane, such as various buildings, during the image recognition process, the need to identify irrelevant features such as buildings not only leads to low work efficiency, but also may lead to inaccurate positioning of the first lane. Therefore, cropping the first image can not only improve the efficiency of image recognition, but also improve the accuracy of lane positioning.

[0077] Step S302: determining first lane information of a lane in which the target vehicle is currently located on the target road according to the first sub-image and the preset information.

[0078] In one embodiment, a method for implementing step S302 includes:

[0079] Image detection processing is performed according to the first sub-image and the preset information to obtain a detection result, wherein the detection result includes at least one first lane number and a confidence level corresponding to each first lane number, the confidence level represents a probability that the lane number of the target vehicle currently located in the target road is the first lane number, and according to the confidence level in the detection result, the first lane information is determined from at least one first lane number in the detection result.

[0080] In an embodiment of the present application, an image detection model can be used for image detection processing. Among them, the image detection model includes but is not limited to a convolutional neural network module (Convolutional Neural Network, CNN). CNN is a deep learning model commonly used in image recognition and computer vision tasks. In the field of autonomous driving, convolutional neural network models are commonly used for vehicle detection and target recognition. The first sub-image and preset information are input into a convolutional neural network, which is used to extract lane line-related features such as edges, textures, and colors in the image, and the extracted lane line-related features are then feature matched according to the preset information to determine the first lane information of the lane where the target vehicle is currently located.

[0081] For example, the convolutional neural network model can be the Restne18 model, and its structure can be divided into three layers, namely, the convolutional layer, the pooling layer, and the fully connected layer. Among them, the convolutional layer is mainly used to extract features from the image, the pooling layer is mainly used to sample the data processed by the convolutional layer to obtain local significant features, and the fully connected layer is used to classify the data features extracted by the convolutional layer and the fully connected layer.

[0082] The first sub-image and the preset information are input into the convolutional neural network, and after a series of convolutional layers and pooling layers, the feature information in the first image can be extracted. This feature information can capture the shape, texture, and edge characteristics of the lane line. Finally, the fully connected layer of the convolutional neural network matches these features with the preset information and outputs the detection result of the first lane line.

[0083] Among them, in the process of lane line detection and recognition tasks, confidence is an indicator used to measure the degree of certainty or confidence in the predicted results, which indicates a measure of the correct probability or reliability of the predicted results.

[0084] In an embodiment of the present application, the lane number may be determined according to the confidence of the detection result by determining the lane number corresponding to the detection result with the highest confidence in descending order of confidence.

[0085] Using the image detection model to obtain the first lane information can reduce the computational complexity of the model and effectively improve the running speed of the model.

[0086] Step S103: performing information fusion processing according to the first information, the first lane information and the current driving data of the target vehicle to obtain second information after fusion processing.

[0087] The first information, the first lane information and all the current driving data of the target vehicle obtained above are fused. The fusion process first requires data preprocessing of the above information. For example, data preprocessing may include filtering noise, correcting deviations or performing data normalization on the information obtained by each sensor. Secondly, feature extraction is performed on the above information. According to different application requirements, key features can be extracted from each data source, for example, feature information such as the position of the vehicle is extracted from the first image, feature information such as the lane number of the target vehicle is obtained from the first lane information, and feature information such as the speed and acceleration of the target vehicle is extracted from the driving data. Then the extracted feature information is data-associated to determine the corresponding relationship between them. For example, the vehicle position captured in the first image is associated with the sensor data such as the speed and acceleration corresponding to the moment when the first image is collected. The above-obtained associated information is recorded as the second information.

[0088] Step S104: re-determine the second lane information of the lane where the target vehicle is currently located in the target road according to the second information.

[0089] In an embodiment of the present application, the first lane information is obtained by using a neural network algorithm using a first image and preset information. However, since the use of a neural network in the process of predicting the lane number may cause the lane number to jump and lead to inaccurate positioning, the present application also proposes a post-fusion technology to improve the accuracy of the positioning algorithm.

[0090] The driving data of the target vehicle in this application includes the Global Positioning System (GPS), the Inertial Measurement Unit (IMU) and the vehicle speed, etc. Among them, GPS is a satellite navigation system that can determine the exact position of the vehicle by receiving satellite signals and provide positioning information such as longitude and latitude. IMU is a device that integrates sensors such as accelerometers and gyroscopes, and is used to measure and record the acceleration, angular velocity and attitude information of the vehicle. The driving speed of the target vehicle can be obtained through a speed sensor. Many vehicles are equipped with speed sensors. These sensors are usually installed in the vehicle's transmission system, which can measure the speed of wheel rotation and convert it into the vehicle's driving speed. The output of the speed sensor can be transmitted to the automatic driving system through the vehicle's CAN bus (controller area network) and other methods.

[0091] In one implementation, steps S103 and S104 may be steps performed by a post-fusion module. In other words, the first information, the first lane information, and the current driving data of the target vehicle are input into the post-fusion module, and the second lane information is output.

[0092] After acquiring the data information of the target vehicle, the first information, the first lane information and the driving data of the target vehicle are input and output to the post-fusion module for data fusion. Optionally, the post-fusion module can use a hidden Markov model (HMM), which can model and predict the driving speed of the vehicle and is used to describe the probability conversion relationship between the implicit state and the observable state.

[0093] Optionally, the post-fusion technology can also use a distributed architecture fusion framework. The idea is that each sensor uses its internal filtering and tracking algorithm, and the post-fusion module effectively combines the filtering results of multiple sensors. Since the hidden Markov model is simpler to calculate than the distributed architecture fusion framework, this application uses the hidden Markov model as the post-fusion module for information fusion. Since its calculation steps are relatively simple, it can speed up the algorithm's running time and improve the speed of the lane positioning algorithm.

[0094] In an embodiment of the present application, the second lane information includes the lane number of the target vehicle re-determined according to the second information. Specifically, multiple possible driving routes of the target vehicle can be obtained according to the second information, the probability of each driving route can be calculated, and then the second lane information of the target vehicle can be determined according to the calculated probability.

[0095] Among them, the first lane information and the second lane information may be the same or different. In some application scenarios, the lane number changes under specific circumstances. For example, after a traffic accident, it is necessary to temporarily close or re-plan the lane, which may cause the lane number to change. In the embodiment of the present application, the above situation is referred to as lane number jump. If there is no problem of lane number jump in the process of locating the first lane information using the image detection model, the first lane information and the second lane information are the same. If there is a problem of lane number jump in the process of locating the first lane information using the image detection model, the second lane information obtained by performing lane positioning again is different from the second lane information, and the second lane information is more accurate.

[0096] In one embodiment, step S104 includes:

[0097] Acquire historical information, where the historical information includes N historical positions before the current position of the target vehicle, where N is a positive integer, and re-determine second lane information of a lane where the target vehicle is currently located in the target road based on the historical information and the second information.

[0098] like Figure 6 As shown, if the current position of the target vehicle is at position 5, positions 1 to 4 are the historical positions of the target vehicle.

[0099] In some implementations, the historical location of the target vehicle may be obtained by accessing a database. The historical location of the target vehicle may be stored in different databases, such as the driving record of the target vehicle, a database provided by a traffic monitoring system or a vehicle positioning service, etc.

[0100] In one embodiment, see Figure 5 , Figure 5 is a schematic diagram of a process for obtaining a second lane provided in an embodiment of the present application, such as Figure 5 As shown, determining the final sequence number of the target lane according to the historical information and the second information includes:

[0101] Step S501, obtaining a first probability of a candidate number corresponding to each of the historical positions in the historical information, wherein the candidate number represents the lane number of the lane where the next candidate position of the historical position is located in the road, and the first probability represents the probability of the candidate number corresponding to the target vehicle when it is at the historical position.

[0102] In the embodiments of the present application, Figure 6 As shown in FIG. 1 , it is a schematic diagram of lane positioning in the post-fusion module provided by an embodiment of the present application. The serial numbers in the circle represent the positions at 5 moments. HMM has two important concepts: observation probability and transition probability. Observation probability refers to the probability that a certain historical position of the target vehicle is a candidate lane. For example, if the current position of the target vehicle is Figure 6 For position 1, its possible candidate states are lane A1, lane A2, outside the left lane, and outside the right lane. Therefore, lane A1, lane A2, outside the left lane, and outside the right lane are candidate lanes for the target vehicle to drive to the next position at position 1. In this system, the HMM model can be used to obtain the first probability of each historical position.

[0103] Exemplarily, the historical position data is used as the observation sequence, and the candidate sequence number data is used as the hidden state sequence. For each historical position, the HMM model can be used for inference to calculate the probability of the target vehicle corresponding to the candidate sequence number at the historical position, and the forward or backward algorithm can be used for calculation. For each historical position, the candidate sequence number with the highest probability can be selected as the first probability according to the calculated candidate sequence number probability.

[0104] Step S502: obtaining a second probability between candidate serial numbers corresponding to each two adjacent historical positions in the historical information, wherein the second probability represents a probability of the target vehicle transferring between lanes corresponding to two candidate serial numbers.

[0105] In the embodiment of the present application, the probability of transition refers to the probability of a change between candidate states corresponding to two adjacent moments. Figure 6 In the figure, the candidate lane A1 at position 4 and the candidate lane B1 at position 5 are connected (the connection relationship can be obtained from the advanced navigation map), so the transfer probability is high, while the candidate lane A1 at position 4 and the candidate lane B3 at position 5 are not connected, so the transfer probability is low. Figure 6 As shown in , the transfer probability between the candidate lanes corresponding to historical positions 1 and 2, the transfer probability between the candidate lanes corresponding to historical positions 2 and 3, and so on are calculated respectively, and the second probability between the candidate positions corresponding to each two adjacent historical positions in the historical data can be calculated.

[0106] One way to calculate the transition probability may include: obtaining a training data set, the training data set including multiple samples, each sample recording information about a lane change event of a target vehicle, including the lane number before the lane change and the lane number after the lane change. Counting the number of occurrences of each type of lane change event in the training data set, wherein the same type of lane change event refers to the same lane number before the lane change and the same lane number after the lane change. Based on the number of occurrences of each type of lane change event obtained by counting, calculate the transition probability of each type of lane change event. For example, the transition probability of lane change event A is the number of occurrences of lane change event A divided by the total number of lane change events in the training data set.

[0107] Step S503: Determine second lane information of the lane where the target vehicle is located on the target road according to the first probability and the second probability.

[0108] In the embodiment of the present application, the operating probability of the target vehicle on a certain driving route can be obtained according to the first probability and the second probability. Figure 6 As shown, the method for determining the second lane information corresponding to the previous historical position 4 from the current position 5 to the current position may include multiplying the first probability of the historical position 4 and the transfer probability (second probability) from the historical position 4 to the current position 5 to obtain the operation probability of the above-mentioned motion trajectory (4 to 5), and based on its operation probability, the current position 5 can be estimated to determine the second lane information.

[0109] Through the above method, the observation data and transfer data of all historical data of the target vehicle can be obtained, and the optimal lane number can be accurately determined based on the observation data and transfer data.

[0110] In one embodiment, step S503 includes:

[0111] According to each candidate position of each of the historical positions, M groups of routes are obtained by permutation and combination, wherein each group of routes includes the candidate positions of the N historical positions. According to the first probability and the second probability, the third probability of each of the M groups of routes is calculated, wherein the third probability of the i-th group of routes represents the probability that the target vehicle travels along the i-th group of routes, and i is a positive integer less than or equal to M; and the final serial number of the target lane is determined according to the third probability.

[0112] In the embodiment of the present application, it is necessary to obtain all possible driving routes of the target vehicle, that is, the candidate lanes of each historical position can be arranged and combined to obtain M routes. Exemplary, the lane driving routes may be: 1 (A1) -> 2 (A1) -> 3 (A1) -> 4 (A1) -> 5 (B1); 1 (A2) -> 2 (A1) -> 3 (A1) -> 4 (A1) -> 5 (B1); 1 (outside the left lane) -> 2 (A1) -> 3 (A1) -> 4 (A1) -> 5 (B1) ... M, etc. As mentioned above Figure 6 As shown in , the historical data of the target vehicle includes five positions, so each route contains candidate positions corresponding to the five historical positions. For example, the candidate positions corresponding to the five historical positions in the route 1(A1)->2(A1)->3(A1)->4(A1)->5(B1) are A1, A1, A1, A1, and B1.

[0113] In the embodiment of the present application, the probability of the current driving route, i.e., the third probability, is obtained by multiplying the first probability and the second probability in each driving route. After obtaining the third probabilities of each driving route, the maximum probability in the third probabilities is determined, and the candidate of the last position in the driving route with the maximum probability is the optimal lane number at the current moment. For example, if in the above driving routes 1(A1)->2(A1)->3(A1)->4(A1)->5(B1), if the observation probabilities of historical positions 1-5 are a1, a2, a3, a4, and a5 respectively, and their transition probabilities are b1, b2, b3, b4, and b5 respectively, then the observation probability in the above driving route is c1=a1×a2×a3×a4×a5, and the transition probability in the driving route is c2=b1×b2×b3×b4×b5, then the third probability of the above driving route is d=c1×c2. By the above method, the third probabilities of M driving routes can be obtained and the sizes of all M third probabilities can be compared.

[0114] In the embodiment of the present application, the final sequence number is determined based on the third probability obtained above, that is, the candidate at the last position of the chain with the maximum probability is taken as the final lane sequence number of the target vehicle at the current moment.

[0115] Exemplarily, if the third probability in the driving route 1 (A1) -> 2 (A1) -> 3 (A1) -> 4 (A1) -> 5 (B1) is the largest, the lane number B1, which is the candidate for the last position in the driving route, is determined as the lane number of the current position of the target vehicle.

[0116] By using the above method, all possible driving routes of the target vehicle can be obtained, and the third probability of each route can be calculated. The optimal lane of the target vehicle can be determined more objectively by using the third probability.

[0117] This application provides a lane positioning method, referring to Figure 7 , is a block diagram of a lane positioning system based on an advanced navigation map provided by an embodiment of the present application, such as Figure 7 As shown, the front view image (first image) is input into the perception module (image processing module) and the lane positioning network module (convolutional neural network module). The perception module outputs geometric information such as lane lines and road boundary lines. The lane positioning network module outputs the first lane information, which is used to perceive lane lines, boundary lines, lane numbers, GPS, imu, vehicle speed and other information. The first lane information output by the lane positioning network module is used as the input signal of the post-fusion module. The post-fusion module uses these signals and information such as the number of lanes provided by the advanced navigation map to obtain the second lane information. Through the above method, the problem of achieving lane positioning with high accuracy and strong generalization performance can be solved without high-precision maps.

[0118] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0119] Corresponding to the lane positioning method described in the above embodiment, Figure 8 This is a structural block diagram of a lane positioning device provided by an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0120] Reference Figure 8 , the device comprises:

[0121] A first acquisition unit 81 is used to obtain first information of a target road according to a first image, wherein the target road is a road on which a target vehicle is currently traveling, the first image is a photographed image of the road on which the target vehicle is currently traveling, and the first information includes geometric information of all lane lines of the target road;

[0122] A first determining unit 82 is used to determine first lane information of a lane in which the target vehicle is currently located on the target road according to the first image and preset information, wherein the preset information includes the lane information of the target road;

[0123] A second acquisition unit 83 is used to perform information fusion processing according to the first information, the first lane information and the current driving data of the target vehicle to obtain second information after fusion processing;

[0124] The second determining unit 84 is configured to re-determine second lane information of the lane where the target vehicle is currently located on the target road according to the second information.

[0125] Optionally, the first determining unit 82 is further configured to:

[0126] Segmenting the first image into a first sub-image and a second sub-image along a height direction of the first image, wherein the first sub-image includes information related to the lane in the target road, and the second sub-image does not include information related to the lane of the target road;

[0127] First lane information of a lane where the target vehicle is currently located on the target road is determined according to the first sub-image and the preset information.

[0128] Optionally, the first determining unit 82 is further configured to:

[0129] Performing image detection processing according to the first sub-image and the preset information to obtain a detection result, wherein the detection result includes at least one first lane number and a confidence level corresponding to each first lane number, and the confidence level indicates a probability that the lane number of the target vehicle currently located in the target road is the first lane number;

[0130] The first lane information is determined from at least one first lane sequence number of the detection result according to the confidence level in the detection result.

[0131] Optionally, the second determining unit 84 is further configured to:

[0132] Acquire historical information, where the historical information includes N historical positions before the current position of the target vehicle, where N is a positive integer;

[0133] The second lane information of the lane where the target vehicle is currently located in the target road is re-determined according to the historical information and the second information.

[0134] Optionally, the second determining unit 84 is further configured to:

[0135] Obtain a first probability of a candidate number corresponding to each of the historical positions in the historical information, wherein the candidate number represents the lane number of the lane where the next candidate position of the historical position is located in the road, and the first probability represents the probability of the candidate number corresponding to the target vehicle at the historical position.

[0136] Obtaining a second probability between candidate serial numbers corresponding to each of two adjacent historical positions in the historical information, wherein the second probability represents a probability that the target vehicle transfers between lanes corresponding to the two candidate serial numbers;

[0137] Second lane information of a lane in which the target vehicle is currently located on the target road is determined according to the first probability and the second probability.

[0138] Optionally, the second determining unit 84 is further configured to:

[0139] According to each candidate position of each of the historical positions, M groups of routes are obtained by arranging and combining, wherein each group of routes includes the candidate positions of each of the N historical positions;

[0140] Calculate the third probability of each of the M groups of routes according to the first probability and the second probability, wherein the third probability of the i-th group of routes represents the probability that the target vehicle travels along the i-th group of routes, and i is a positive integer less than or equal to M;

[0141] The second lane information is determined according to the third probability.

[0142] Optionally, the second determining unit 84 is further configured to:

[0143] Determine a group of routes corresponding to the maximum value of the third probabilities of the M groups of routes as the target routes;

[0144] The second lane information is determined according to a candidate position of the last historical position in the target route.

[0145] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0146] in addition, Figure 8 The lane positioning device shown may be a software unit, a hardware unit, or a combination of software and hardware, which is built into an existing terminal device, or may be integrated into the terminal device as an independent pendant, or may exist as an independent terminal device.

[0147] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0148] Fig. 9 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Fig. 9 As shown, the terminal device 9 of this embodiment includes: at least one processor 90 ( Fig. 9 Only one is shown in the figure) a processor, a memory 91, and a computer program 92 stored in the memory 91 and executable on the at least one processor 90, wherein when the processor 90 executes the computer program 92, the steps in any of the above lane positioning method embodiments are implemented.

[0149] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Fig. 9 It is only an example of the terminal device 9 and does not constitute a limitation on the terminal device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0150] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0151] In some embodiments, the memory 91 may be an internal storage unit of the terminal device 9, such as a hard disk or memory of the terminal device 9. In other embodiments, the memory 91 may also be an external storage device of the terminal device 9, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 9. Further, the memory 91 may also include both an internal storage unit of the terminal device 9 and an external storage device. The memory 91 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program, etc. The memory 91 may also be used to temporarily store data that has been output or is to be output.

[0152] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0153] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0155] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0156] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0157] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A lane positioning method, characterized in that: include: Obtaining first information of a target road according to a first image, wherein the target road is a road on which a target vehicle is currently traveling, the first image is a photographed image of the road on which the target vehicle is currently traveling, and the first information includes geometric information of all lane lines of the target road; Determine first lane information of the lane where the target vehicle is currently located on the target road according to the first image and preset information, wherein the preset information includes the lane information of the target road; Performing information fusion processing according to the first information, the first lane information, and the current driving data of the target vehicle to obtain second information after fusion processing; The second lane information of the lane where the target vehicle is currently located in the target road is re-determined according to the second information.

2. The lane positioning method according to claim 1, characterized in that: The determining, according to the first image and preset information, first lane information of the lane in which the target vehicle is currently located on the target road includes: Segmenting the first image into a first sub-image and a second sub-image along a height direction of the first image, wherein the first sub-image includes information related to the lane in the target road, and the second sub-image does not include information related to the lane of the target road; First lane information of a lane where the target vehicle is currently located on the target road is determined according to the first sub-image and the preset information.

3. The lane positioning method according to claim 2, characterized in that: The determining, according to the first sub-image and the preset information, first lane information of the lane where the target vehicle is currently located on the target road includes: Performing image detection processing according to the first sub-image and the preset information to obtain a detection result, wherein the detection result includes at least one first lane number and a confidence level corresponding to each first lane number, and the confidence level indicates a probability that the lane number of the target vehicle currently located in the target road is the first lane number; The first lane information is determined from at least one first lane sequence number of the detection result according to the confidence level in the detection result.

4. The lane positioning method according to claim 1, characterized in that: The re-determining second lane information of the lane where the target vehicle is currently located on the target road according to the second information includes: Acquire historical information, where the historical information includes N historical positions before the current position of the target vehicle, where N is a positive integer; The second lane information of the lane where the target vehicle is currently located in the target road is re-determined according to the historical information and the second information.

5. The lane positioning method according to claim 4, characterized in that: The re-determining the second lane information of the lane where the target vehicle is currently located on the target road according to the historical information and the second information includes: Obtaining a first probability of a candidate sequence number corresponding to each of the historical positions in the historical information, wherein the candidate sequence number represents the lane sequence number of the lane where the next candidate position of the historical position is located on the road, and the first probability represents the probability of the candidate sequence number corresponding to the target vehicle at the historical position; Obtaining a second probability between candidate serial numbers corresponding to each of two adjacent historical positions in the historical information, wherein the second probability represents a probability that the target vehicle transfers between lanes corresponding to the two candidate serial numbers; Second lane information of a lane in which the target vehicle is currently located on the target road is determined according to the first probability and the second probability.

6. The lane positioning method according to claim 5, wherein determining the second lane information of the lane where the target vehicle is currently located in the target road according to the first probability and the second probability comprises: According to each candidate position of each of the historical positions, M groups of routes are obtained by arranging and combining, wherein each group of routes includes the candidate positions of each of the N historical positions; Calculate the third probability of each of the M groups of routes according to the first probability and the second probability, wherein the third probability of the i-th group of routes represents the probability that the target vehicle travels along the i-th group of routes, and i is a positive integer less than or equal to M; The second lane information is determined according to the third probability.

7. The lane positioning method according to claim 6, characterized in that: The determining the second lane information according to the third probability includes: Determine a group of routes corresponding to the maximum value of the third probabilities of the M groups of routes as the target routes; The second lane information is determined according to a candidate position of the last historical position in the target route.

8. A lane positioning device, characterized in that: include: a first acquisition unit, configured to obtain first information of a target road according to a first image, wherein the target road is a road on which a target vehicle is currently traveling, the first image is a photographed image of the road on which the target vehicle is currently traveling, and the first information includes geometric information of all lane lines of the target road; A first determining unit, configured to determine first lane information of a lane in which the target vehicle is currently located on the target road according to the first image and preset information, wherein the preset information includes the lane information of the target road; A second acquisition unit is used to perform information fusion processing according to the first information, the first lane information and the current driving data of the target vehicle to obtain second information after fusion processing; A second determining unit is used to re-determine second lane information of a lane where the target vehicle is currently located in the target road according to the second information.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images