Positioning method, device, electronic device and computer storage medium

Abstract

The embodiments of the present application provide a positioning method, electronic device, and computer storage medium. The positioning method includes: obtaining a position observation value of a target object at the Kth moment, wherein the position observation value includes: observation data obtained based on visual semantic information corresponding to an environmental image of the target object's location; obtaining N predicted positions of the target object at the Kth moment; determining the lane weights of the target object at each of the N predicted positions at the Kth moment based on the position observation value at the Kth moment and a high-precision map; and obtaining the position information of the target object at the Kth moment based on the lane weights of the target object at each of the N predicted positions at the Kth moment. This method has higher positioning accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of positioning technology, and in particular to a positioning method, device, electronic device, and computer storage medium. Background Art

[0002] With advances in artificial intelligence algorithms and network technologies, intelligent driving is becoming increasingly possible. As one of the foundational and core components of intelligent driving systems, the positioning unit's output accuracy directly impacts the control accuracy and timeliness of the intelligent driving system.

[0003] Taking satellite-based positioning units as an example, these units can achieve an accuracy of 3-5 meters in open environments. However, in urban areas with tall buildings, the accuracy can drop to tens or even hundreds of meters due to obstruction of satellite signals and multipath effects. The inventors of this application discovered during their research on related technologies that meter-level accuracy cannot meet the requirements of intelligent driving. Therefore, improving positioning accuracy to sub-meter or even centimeter levels is a core issue that positioning units need to address. Summary of the Invention

[0004] In view of this, an embodiment of the present application provides a positioning solution to at least partially solve the above-mentioned problem.

[0005] According to a first aspect of an embodiment of the present application, a positioning method is provided, including: obtaining a position observation value of a target object at the Kth moment, the position observation value including: observation data obtained based on visual semantic information corresponding to an environmental image of the target object's location; obtaining N predicted positions of the target object at the Kth moment; determining, based on the position observation value at the Kth moment and a high-precision map, the lane weights of the target object at the N predicted positions at the Kth moment; and obtaining the position information of the target object at the Kth moment based on the lane weights of the target object at the N predicted positions at the Kth moment.

[0006] According to a second aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the positioning method described in the first aspect.

[0007] According to a third aspect of the embodiments of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the positioning method as described in the first aspect is implemented.

[0008] According to a third aspect of an embodiment of the present application, a computer program product is provided, which implements the positioning method as described above when executed by a processor.

[0009] The positioning solution provided in the embodiment of the present application utilizes visual recognition of the environmental image of the target object's location to obtain a position observation, and based on the position observation and the high-precision map, determines the lane weights of N different predicted positions at the Kth moment, and then determines the position information of the target object based on the lane weights of the N different predicted positions. Since the position observation and the high-precision map obtained by visual recognition are utilized, the positioning accuracy is improved. Since the lane weights of the N different predicted positions are calculated separately, the correlation between the positioning accuracy and the matching accuracy of the position observation and the high-precision map is reduced, thereby ensuring that a higher positioning accuracy can be achieved in more scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0011] Figure 1A This is a flowchart of the steps of a positioning method according to the first embodiment of the present application;

[0012] Figure 1B This is a schematic diagram of a usage scenario according to the first embodiment of the present application;

[0013] Figure 2A This is a flowchart of the steps of a positioning method according to the second embodiment of the present application;

[0014] Figure 2B is a schematic diagram of particle degradation in the prior art;

[0015] Figure 2C Schematic diagram of the predicted position at the Kth moment according to a positioning method according to the second embodiment of the present application;

[0016] Figure 2D This is a flowchart of the steps of the positioning method in the usage scenario according to the second embodiment of the present application;

[0017] Figure 2E This is a schematic diagram of the initialization phase of the usage scenario according to the second embodiment of the present application;

[0018] Figure 2F This is a schematic diagram of a single positioning according to the usage scenario of the second embodiment of the present application;

[0019] Figure 2G This is a schematic diagram of a state where the position observation value obtained at the Kth moment is satellite positioning data in the usage scenario of the second embodiment of the present application;

[0020] Figure 2H Schematic diagram of N predicted positions at the Kth moment after the satellite positioning position is updated in the usage scenario of the second embodiment of the present application;

[0021] Figure 3 This is a structural block diagram of a positioning device according to the third embodiment of the present application;

[0022] Figure 4 This is a structural diagram of an electronic device according to the fourth embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present application.

[0024] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0025] Example 1

[0026] Reference Figure 1A , shows a step flow chart of a positioning method according to embodiment 1 of the present application.

[0027] The positioning method in this embodiment can be applied to scenarios such as navigation and intelligent driving, achieving precise positioning of target objects (such as people and vehicles) in navigation or intelligent driving scenarios, such as lane-level positioning or sub-meter or centimeter-level positioning. For example, in an intelligent driving scenario, if the target object is an unmanned truck, the positioning method can be used to determine the lane and position of the unmanned truck within the lane, and then the unmanned truck can be controlled in a timely manner to perform actions such as merging and turning to ensure driving safety.

[0028] The positioning method of this embodiment is described below. The method includes the following steps:

[0029] Step S102: Obtain the position observation of the target object at the Kth moment.

[0030] The position observation is obtained based on the visual semantic information corresponding to the environment image of the target object or obtained through satellite positioning. The position observation includes the relative position information between the target object and the reference object in the environment, the confidence level corresponding to the relative position information, and satellite positioning data.

[0031] For example, an image capture device on the target object captures an image of the environment where the target object is located, and then visual recognition is performed on the environment image to obtain corresponding visual semantic information, such as a reference object in the environment and the aforementioned position observations. The position observations include the aforementioned relative position information, which includes but is not limited to the orientation of the target object relative to the reference object and the distance between the two.

[0032] The reference object may be a lane line, or a traffic facility (such as a traffic light, a traffic sign), etc.

[0033] In one case, if the reference object is a lane line, the relative position information in the position observation can be: the distance relative to the left solid lane line is 1.5M, or: the distance relative to the right dotted lane line is 1.2M, etc.

[0034] In another case, if the reference object is a traffic light or a traffic sign, the relative position information in the position observation can be: the distance from the traffic light in front is 3M, or the relative position information can be the distance from the traffic sign on the left is 4M, or the relative position information can be the distance from the traffic sign on the right is 1M, etc.

[0035] Step S104: Obtain N predicted positions of the target object at the Kth moment.

[0036] In one scenario, if the Kth moment is the first moment, a rough estimated position of the target object is determined based on the position information detected by the target object's satellite positioning system, and the road on which the target object is located is determined based on the rough estimated position. N predicted positions are evenly distributed across the cross section of the road as the N predicted positions at the first moment, and the lane weights of the N predicted positions at the first moment are configured as 1 / N. N can be a positive integer.

[0037] In another case, if the Kth moment is a moment other than the first, the predicted position at the Kth moment can be determined based on the position information at the K-1th moment.

[0038] Step S106: Determine the lane weights of the target object at the N predicted positions at the Kth moment based on the position observation at the Kth moment and the high-precision map.

[0039] Based on the position observation at time K, the target object's location information can be determined in conjunction with a high-precision map. In this embodiment, a particle filter is used to estimate the target object's location information to address the problem in the prior art where the Kalman filter only outputs a single location, resulting in accurate estimation of the target object's location information only when the reference object is correctly associated with the calibration object in the high-precision map. If the association is incorrect, positioning accuracy is drastically reduced.

[0040] In the particle filtering approach, each predicted position at time K is considered a particle. It is assumed that the target object can appear at any predicted position at time K, but the probability of appearing at different predicted positions varies, meaning that each predicted position at time K has a different lane weight. This particle filtering approach ensures a relatively accurate estimate of the target object's position, even if the reference object is not correctly associated with the calibrated object in the high-precision map.

[0041] Step S108: Obtaining the position information of the target object at the Kth moment according to the lane weights of the N predicted positions of the target object at the Kth moment.

[0042] For the position observations corresponding to different reference objects, the probability can be calculated in an appropriate manner.

[0043] For example, if the position observation includes an intercept L of 1.5 meters relative to a reference lane line (e.g., the left solid lane line), and the confidence level of the intercept is 0.8, then for each of the N predicted positions at time K, the actual distance between each predicted position and the left demarcated lane line is determined based on the high-precision map. The lane weight for that predicted position is determined based on the difference between the actual distance and the intercept L. For example, if a road has three lanes, designated lane 1, lane 2, and lane 3, and the N predicted positions are predicted positions 1 through N, with predicted position 1 located within lane 1, predicted position 2 located within lane 2, and predicted positions 3 through N located within lane 3. For each predicted position 1, the actual distance between predicted position 1 and the left demarcated lane line of lane 1 can be determined based on the high-precision map. The lane weight for predicted position 1 is determined based on the intercept L, the confidence level of the intercept, and the actual distance. For predicted position 2, the actual distance between predicted position 2 and the marked lane line on the left side of lane 2 is determined based on the high-precision map. The lane weight for predicted position 2 is determined based on the intercept L, the confidence level of the intercept, and the actual distance. The process for predicted positions 3 to N is similar and will not be repeated here.

[0044] For another example, the position observation includes a reference distance of 3M between the target object and the reference traffic facility (such as traffic sign A on the left), and a confidence level of 0.9 for the reference distance. For each predicted position, based on a high-precision map, the actual distance between each predicted position and the calibrated traffic sign on the left is determined, and then the lane weight of each predicted position is determined based on the actual distance, the reference distance, and the confidence level of the reference distance.

[0045] In this way, the lane weight corresponding to each predicted position can be determined—that is, the probability that the target object is located at that predicted position at time K. During this process, even if there are errors in the association between the reference object and the high-precision map's calibration features, these errors can be corrected with successive iterations. This results in predicted positions closer to the target object's true location receiving larger lane weights, thereby improving adaptability and ensuring positioning accuracy in complex environments.

[0046] Subsequently, the position information of the target object at the Kth moment can be determined based on the calculated lane weights of the predicted positions at each Kth moment. The position information can be a single position or multiple positions.

[0047] The following describes the implementation of this method using a specific usage scenario:

[0048] like Figure 1B As shown, in this use case, the target object is an unmanned truck (hereinafter referred to as the truck). The truck is equipped with at least one image acquisition device, and may also be equipped with a satellite positioning system, a data processing chip with data processing capabilities, and a communication unit. The positioning method can be executed by the data processing chip on the truck or by a cloud or server connected to the truck, and the execution results are fed back to the truck.

[0049] During the driving process of the truck, its rough estimated position can be determined based on the satellite positioning system, thereby achieving road-level positioning (such as on Road A), and then the number of lanes contained in Road A can be determined based on the high-precision map. In the usage scenario, Road A includes 3 lanes, which are recorded as Lane 1, Lane 2 and Lane 3 as an example.

[0050] At the Kth moment, the position observation is obtained by visually recognizing the environment image of the truck's location. For example, the intercept L between the truck and the left solid lane line is 1.5M, and the intercept R between the truck and the right dotted lane line is 1.5M. In addition, based on the N predicted positions at the K-1th moment ( Figure 1B The predicted positions A1~AN) and the corresponding lane weights are shown in , and the N predicted positions at the Kth moment are obtained ( Figure 1B The predicted positions B1 to BN are shown in FIG.

[0051] For the predicted position B1 at time K, the actual distance L between the predicted position B1 and the left calibrated lane line and the actual distance R between the predicted position B1 and the right calibrated lane line are determined based on the high-precision map. The probability P1L of the predicted position B1 is determined based on the intercept L and the actual distance L. The probability P1R of the predicted position B1 is determined based on the intercept R and the actual distance R. The lane weight of the predicted position B1 is then determined based on the product of the probabilities P1L and P1R.

[0052] The lane weights of the predicted positions B2 to BN can be determined in a similar manner. The lane weight calculation method when the reference object in the position observation is a traffic facility is similar and will not be repeated here.

[0053] After determining the lane weights of each predicted position at each K-th moment, if it is necessary to determine the position of a unique target object at the K-th moment, the N predicted positions can be weighted and summed to estimate the final position at the K-th moment.

[0054] Alternatively, if it is allowed to output the position information of multiple K-th moments, these N predicted positions can be clustered according to the number of lanes contained in road A, thereby outputting the position information of the target object in different lanes, so that the position of the target object can be accurately located in combination with other observations in the future.

[0055] Through this embodiment, the position observation is obtained by visual recognition of the environmental image of the target object's location. Based on the position observation and the high-precision map, the lane weights of N different predicted positions at the Kth moment are determined, and then the position information of the target object is determined according to the lane weights of the N different predicted positions. Since the position observation and the high-precision map obtained by visual recognition are used, the positioning accuracy is improved. Since the lane weights of N different predicted positions are calculated separately, the correlation between the positioning accuracy and the matching accuracy of the calibration objects in the high-precision map is reduced, thereby ensuring that higher positioning accuracy can be achieved in more scenarios.

[0056] The positioning method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a mobile phone, a PAD, etc.) and a PC.

[0057] Example 2

[0058] Reference Figure 2A , shows a schematic flow chart of the steps of the positioning method according to an embodiment of the present application.

[0059] In this embodiment, the positioning method includes the following steps:

[0060] Step S202: Obtain an initialized lane clustering group.

[0061] At the first moment, a rough estimated position of the target object is determined based on the position information detected by the satellite positioning system of the target object, and the road where the target object is located and the number of lanes included in the road are determined based on the rough estimated position.

[0062] N predicted positions are evenly distributed on the cross section of the road as the N predicted positions at the first moment, and the lane weights of the N predicted positions at the first moment are configured as 1 / N.

[0063] At the second moment, the inter-frame displacement between the first and second moments is determined based on the position change and random error output by the IMU (Inertial Measurement Unit) configured on the target object. Furthermore, based on the N predicted positions at the first moment and their corresponding lane weights, the number of predicted positions at the second moment corresponding to each predicted position at the first moment is determined, and the N predicted positions at the second moment are determined in combination with the inter-frame displacement.

[0064] Since the lane weights of the N predicted positions at the first moment are equal, each predicted position at the first moment corresponds to a predicted position at the second moment. The predicted position at the second moment is determined by the sum of its corresponding predicted position at the first moment, the inter-frame displacement, and the random error. For example, if the predicted position P11 at the first moment splits out the predicted position P21 at the second moment, the position of the predicted position P21 at the second moment is the predicted position P11 at the first moment + the inter-frame displacement + the random error. Among them, the random error obeys the normal distribution (i.e., Gaussian distribution).

[0065] After obtaining the N predicted positions at the second moment, the lane weight of each predicted position at the second moment can be calculated based on the position observations obtained at the second moment. The N predicted positions at the second moment are then clustered according to the number of lanes on the road to obtain lane cluster groups corresponding to the number of lanes.

[0066] For example, if the road includes three lanes, the N predicted positions at the second moment are clustered into three lane cluster groups.

[0067] In a feasible clustering method, the N predicted positions at the second moment are clustered based on the lane line type. If the reference object obtained at the second moment is a lane line, and it is a solid lane line on the left and a dashed lane line on the right, then based on the high-precision map, a set of predicted positions with two adjacent lane line types matching (denoted as set A), a set of predicted positions with only one lane line type matching (denoted as set B), and a set of predicted positions with two lane line types not matching (denoted as set C) are determined from the N predicted positions at the second moment, and different inter-group weights are configured for sets A to C respectively. The specific inter-group weights can be determined as needed, such as based on the number of matched lane lines and the confidence of visual recognition. In this embodiment, the inter-group weights of sets A to C gradually decrease.

[0068] Another feasible clustering method is to cluster the N predicted positions at the second moment based on the collected positions. For example, based on the predicted positions at the second moment and the high-precision map, three lane-corresponding sets are determined, denoted as Set A, Set B, and Set C. The weights between the three sets can be determined based on the position information obtained at the second moment (such as satellite positioning data or radar / visual point cloud positioning data).

[0069] Taking satellite positioning data as an example, the probability of the target object being at each predicted position at the second moment is determined based on the satellite positioning data, which can be expressed as:

[0070]

[0071] Among them, x GNSS is the satellite positioning data, z ki represents the I-th predicted position in the K-th set, K belongs to one of the sets A, B and C, σ GNSS is the confidence level output by the satellite positioning system, P KI is the coordinate of the I-th predicted position of the K-th set determined based on the high-precision map, p GNSS Indicates the position indicated by satellite positioning data, and EXP{} is an exponential function.

[0072] After calculating the probability of each predicted position in the set, the probabilities of all predicted positions in the set can be summed, which is expressed as:

[0073]

[0074] The cumulative probability of the Kth set at the second moment is expressed as:

[0075]

[0076] Among them, T is the time number, that is, the second time, T-1 is the previous time, that is, the first time, represents the normalized probability of the Kth set at time T-1, and the normalized probability of the Kth set at time T is:

[0077] The normalized probability at time T is the inter-group weight of the Kth set.

[0078] In this way, the N predicted positions at the second moment can be clustered to obtain lane cluster groups. The inter-group weights of the lane cluster groups are used to represent the probability of the target object appearing in a certain lane. This can avoid the particle degradation problem caused by the iterative process at multiple moments.

[0079] The following is a specific example to illustrate particle degradation:

[0080] like Figure 2B There are three lanes on the current road, among which the cross-sectional width of the lane where Z0 is located is 3.1M, the cross-sectional width of the lane where Z1 is located is 3.0M, and the cross-sectional width of the lane where Z2 is located is 3.2M. If the obtained position observation indicates that the intercept of the target object from the left solid lane line is 1.5M and the intercept from the right dotted lane line is 1.5M, then without grouping the particles (that is, the N predicted positions), if the lane weights of the N predicted positions at a certain moment are determined simply based on the position observation, the lane weight of the predicted position in the lane where Z1 is located will be increased. In the iterative process, since the predicted position at the next moment is determined based on the predicted position and its lane weight at the previous moment, this error will continue to accumulate. After a short iteration time (usually a few seconds or more than ten seconds), the lane weight of the predicted position in the lane where Z1 is located will be much larger than the lane weights of the predicted positions in the lanes where Z0 and Z2 are located, causing the predicted positions in the lanes where Z0 and Z2 to degenerate, and the lane where Z1 is located will be mistakenly regarded as the correct position, resulting in positioning error.

[0081] Lane clustering groups are obtained by clustering the N predicted positions at the second moment according to the lanes they belong to. This ensures that the lane width only affects the intra-group weights of different predicted positions within a lane clustering group, without affecting the inter-group weights of the lane clustering group. This avoids the problem of particle degradation caused by error accumulation.

[0082] During the movement of the target object, each position observation obtained (this position observation can be not only the position observation obtained by visual recognition of the environment image, but also the satellite positioning data obtained based on the satellite positioning system) can be used as a moment to update the lane weights and intra-group weights of the N predicted positions. In some cases (such as when the reference object is a lane line), the inter-group weights can also be updated. In this way, as more and more position observations are obtained in the iteration, the lane weights of the predicted positions that are closer to the actual position of the target object in the N predicted positions can become larger and larger, thereby achieving convergence. Since the position observations in the iterative process can be not only the observation data obtained based on visual semantic information, but also satellite positioning data or other data, the data obtained by visual semantic recognition, high-precision maps and multi-source sensors are fused to improve positioning accuracy.

[0083] Step S204: Obtain the position observation of the target object at the Kth moment.

[0084] This implementation process is similar to the implementation process of step S102 described above, so it will not be repeated here. In one example, the position observation may include satellite positioning data and its confidence level. Alternatively, the position observation may include a reference distance of the target object relative to a reference traffic facility and a confidence level for the reference distance. Alternatively, the position observation may include an intercept of the target object relative to a reference lane line and a confidence level for the intercept.

[0085] Step S206: Obtain N predicted positions of the target object at the Kth moment.

[0086] When the position observation is received at the Kth time, the N predicted positions at the Kth time can be determined based on the N predicted positions at the K-1th time. Figure 2C As shown, the center of the circle is the predicted position at the K-1th moment, and the diameter of the circle represents the lane weight of the predicted position at the K-1th moment.

[0087] The acquisition of the N predicted positions of the target object at the Kth moment is achieved through the following sub-steps (not shown in the figure):

[0088] Sub-step S2061: Obtain the N predicted positions of the target object at the K-1th moment and the lane weight corresponding to the predicted position at the K-1th moment.

[0089] The predicted position at the K-1th moment is determined by fusing position observations collected by multi-source sensors at multiple historical moments before the K-1th moment.

[0090] Sub-step S2062: Determine N predicted positions at the Kth moment corresponding to the predicted position at the K-1th moment based on the lane weights corresponding to the predicted position at the K-1th moment.

[0091] In one feasible approach, sub-step S2062 includes the following process:

[0092] Process A1: According to the lane weights corresponding to the predicted positions at the K-1th time, determine the number of predicted positions at the K-1th time split from the predicted positions at the K-1th time, and the number is less than or equal to N.

[0093] The number of predicted positions at the Kth moment split from each predicted position at the K-1th moment can be determined by multiplying the corresponding lane weight by the total number of predicted positions at the K-1th moment (ie, N).

[0094] For example, for Figure 2C The first predicted position at the K-1th time is split into two predicted positions at the Kth time based on the product of its corresponding lane weight and N. The second predicted position at the K-1th time is split into one predicted position at the Kth time. If the lane weight is too small, resulting in the product of its weight and N being less than 1, the predicted position at the K-1th time cannot be split into a predicted position at the Kth time. This naturally filters out predicted positions with too small lane weights.

[0095] Process B1: According to the predicted positions at the K-1th moment and the inter-frame displacement from the K-1th moment to the Kth moment, determine the N predicted positions at the Kth moment split from all the predicted positions at the K-1th moment.

[0096] The inter-frame displacement is determined according to output data of an inertial measurement unit of the target object and a random error.

[0097] For example, the predicted position at the Kth moment can be determined by the following transition probability model. The transition probability model can be expressed as:

[0098] z k =z k-1 +(Δx IMU , Δy IMU ) T +w k

[0099] Among them, Z k is the predicted position at the Kth moment, Z k-1 The predicted position at the K-1th moment is a known quantity, (Δx IMU , Δy IMU ) Tis the transposition of the position change between two adjacent moments output by the inertial measurement unit (i.e., the inter-frame displacement), w n is the estimated noise of the inertial measurement unit (IMU unit), which is assumed to be Gaussian distributed. The predicted positions at different K-th moments are randomly taken (i.e., random errors), so that the values ​​of the predicted positions at the K-th moment split from the same predicted position at the K-1-th moment are different.

[0100] In this way, N predicted positions at the Kth moment can be obtained, such as Figure 2C This achieves the prediction of N predicted positions at the Kth moment, and the corresponding weights can be updated based on the position observations.

[0101] Step S208: Determine the lane weights of the target object at the N predicted positions at the Kth moment based on the position observation at the Kth moment and the high-precision map.

[0102] In one feasible embodiment, step S208 includes the following sub-steps (not shown in the figure):

[0103] Sub-step S2081: Determine the position weights of the target object when it is located at the N predicted positions at the Kth moment based on the position observation at the Kth moment and the high-precision map.

[0104] In the first case, the position observation includes the intercept of the target object relative to the reference lane line and the confidence level of the intercept. Accordingly, the position weight can be determined by: determining the actual distance of the target object relative to the reference lane line when located at each of the N predicted positions based on the high-precision map; and determining the position weights corresponding to the N predicted positions based on the intercept of the target object relative to the reference lane line, the confidence level of the intercept, and each actual distance included in the position observation.

[0105] For example, for the I-th predicted position among the N predicted positions at the K-th time, the position weight is calculated as follows:

[0106]

[0107] Among them, μ(x vision |z) is the probability (that is, the position weight), σ vision is the confidence of the intercept output by the visual recognition model, c vision is the intercept relative to the lane line (can be the left lane line or the right lane line), c map is the actual distance between the I-th predicted position determined based on the high-precision map and the adjacent left lane line or right lane line. It should be noted that c vision and c mapIt should be the lane line on the same side, i.e. c vision is relative to the left lane line, c map It should also be relative to the left lane line. The calculated p(x vision |z) can be used as the position weight of the I-th predicted position.

[0108] In the second scenario, the position observation includes the reference distance of the target object relative to the reference transportation facility and the confidence level of the reference distance. Accordingly, the position weight can be determined by: determining, based on a high-precision map, the actual distance of the target object relative to the reference transportation facility when the target object is at each of the N predicted locations; and determining the position weights corresponding to the N predicted locations based on the position observation, including the reference distance of the target object relative to the reference transportation facility, the confidence level of the reference distance, and each actual distance.

[0109] For example, for the I-th predicted position among the N predicted positions at the K-th time, the position weight is calculated as follows:

[0110]

[0111] Among them, p(o|z) is the probability (i.e., position weight), 0 represents the identified reference transportation facility, σ o is the confidence (i.e. standard deviation) of the reference distance output by the visual recognition model, (p o -z) is the reference distance relative to the transportation facility, p′ o represents the position of the calibrated traffic facility associated with the identified traffic facility, as determined based on the high-precision map (that is, the actual distance between the first predicted position and the calibrated traffic facility in the high-precision map). The calculated p(o|z) is the position weight corresponding to the first predicted position.

[0112] In the third scenario, the position observation includes satellite positioning data and a confidence level corresponding to the satellite positioning data. Accordingly, the position weight may be determined by determining a position weight corresponding to each predicted position based on the position observation including the satellite positioning data and the confidence level corresponding to the satellite positioning data and the N predicted positions.

[0113] For example, for the I-th predicted position among the N predicted positions at the K-th time, the position weight is calculated as follows:

[0114]

[0115] Among them, p(x GNSS |z) is the probability (i.e., position weight), σ GNSSis the confidence level output by the satellite positioning system (i.e., the standard deviation of GNSS positioning), p GNSS is the satellite positioning data, p z The position indicated by the first predicted position. The calculated p(x GNSS |z) is the position weight of the I-th predicted position.

[0116] In this way, the position weights corresponding to the N predicted positions at the Kth moment can be obtained. In order to prevent particle degradation, the intra-group weights and / or inter-group weights of the lane clustering groups corresponding to the N predicted positions at the Kth moment can be updated to determine the position information of the target object at the Kth moment.

[0117] Sub-step S2082: Determine the lane cluster group to which the N predicted positions at the K-th moment belong.

[0118] The number of lane cluster groups is determined based on the number of lanes on the road corresponding to the target object. Lane cluster groups correspond one-to-one to lanes and can also be referred to as particle groups.

[0119] The process of obtaining lane cluster groups has been described in detail above. In this sub-step, for a certain predicted position, it is only necessary to determine the lane cluster group to which it belongs based on its position.

[0120] Sub-step S2083: Determine the intra-group weights of the N predicted positions in the lane cluster group to which they belong based on the position weights of the N predicted positions.

[0121] Because the intra-group weight of a lane cluster group indicates the probability of the target object being at a certain predicted position within the lane corresponding to the lane cluster group, updating the intra-group weight based on the position weight can be achieved by normalizing the position weights of the predicted positions in the current lane cluster group for each lane cluster group; and determining the intra-group weights of the N predicted positions within the lane cluster group based on the normalization results.

[0122] For example, predicted positions 1 to 5 belong to lane cluster group 1. The total weight is obtained by summing the position weights of predicted positions 1 to 5, and then normalization is achieved by calculating the ratio of the position weight of each predicted position to the total weight. The normalized ratio is the in-group weight of each predicted position.

[0123] Sub-step S2084: Determine the inter-group weights between the lane clustering groups.

[0124] The inter-group weight of the lane clustering group indicates the probability that the target object is in the corresponding lane. The inter-group weight can prevent particle degradation. Therefore, the inter-group weight is updated when the reference object is a lane line or satellite positioning data. If the reference object is a traffic facility, the inter-group weight does not need to be updated and the original inter-group weight is directly used as the inter-group weight at the Kth moment.

[0125] Updating the inter-group weights can be achieved as follows: determining the sum of the position weights of the predicted positions within each lane clustering group as the initial weight of the lane clustering group; performing inter-group normalization on the initial weights of multiple lane clustering groups; and determining the inter-group weights of each lane clustering group based on the results of the inter-group normalization.

[0126] The specific updating process is the process of updating the inter-group weights based on the satellite positioning data in the aforementioned step S202, which will not be described in detail here.

[0127] Sub-step S2085: Determine the lane weights corresponding to the N predicted positions based on the intra-group weights and the inter-group weights.

[0128] For a certain predicted position, the intra-group weight and the inter-group weight are weighted summed to determine the lane weight corresponding to the predicted position.

[0129] Step S210: Obtaining the position information of the target object at the Kth moment according to the lane weights of the N predicted positions of the target object at the Kth moment.

[0130] In one feasible approach, if only one position is allowed to be output as the position information, the predicted position with the largest lane weight can be selected as the output position. Alternatively, if multiple positions are allowed to be output, the predicted position with the largest lane weight within each lane can be output.

[0131] The following combination Figures 2D to 2H The positioning method of this embodiment is described as follows using a specific usage scenario as an example:

[0132] Figure 2E to Figure 2H The results of visualizing the particle filter calculation process are shown in Figure 2. The steps of the positioning method are as follows: Figure 2D As shown, it includes:

[0133] Step S1: Determine the current road.

[0134] Figure 2E Corresponding to the state at the time of initialization, the current road where the target object is located can be determined based on the satellite positioning data of the target object.

[0135] Step S2: Particle initialization and grouping.

[0136] Since the lane where the target object is located cannot be located more accurately based on satellite positioning data, the particles are evenly distributed on the cross section of the entire road to form N predicted positions at the first moment, and the lane weights of the N predicted positions at the first moment are all equal to 1 / N.

[0137] According to the lanes in which the particles are located, they can be divided into three particle groups, that is, three lane clustering groups.

[0138] Step S3: Based on the position observations, determine the N predicted positions and corresponding lane weights at the Kth moment.

[0139] Figure 2F This is the state after observing lane marking recognition results and updating them. Visual recognition results indicate that there is a lane marking at 1.73 meters on the left and another at 2.18 meters on the right, both of which are solid. However, based on the high-precision map, all three lanes on the current road are solid. Therefore, the target lane cannot be determined, and only the weights within the group can be updated.

[0140] Specifically, step S3 includes the following sub-steps:

[0141] Sub-step S31: using the transition probability to predict N predicted positions at the Kth moment.

[0142] Based on the known N predicted positions at the K-1th time and their corresponding lane weights, the number of predicted positions at the Kth time split from the N predicted positions at the K-1th time and their corresponding relationship can be determined.

[0143] Each predicted position at the Kth moment is equal to the sum of its corresponding predicted position at the K-1th moment, the inter-frame displacement, and the random error.

[0144] Sub-step S32: Using the position observation and the high-precision map, determine the lane weights of the N predicted positions at the Kth moment.

[0145] If the reference object in the position observation is a lane line, the position weight of the predicted position is updated based on the lane line observation probability model. The lane line observation probability model is as follows:

[0146]

[0147] If the reference object in the position observation is a transportation facility, the position weight of the predicted position is updated based on the transportation facility observation probability model. The transportation facility observation probability model is as follows:

[0148]

[0149] If the position observation is satellite positioning data, the position weight is updated based on the satellite positioning observation probability model. The satellite positioning observation probability model is as follows:

[0150]

[0151] The meaning of the above observation probability model has been explained in detail in the previous steps, so it will not be repeated here.

[0152] Based on the obtained position weight, each lane cluster group can be normalized within the group, so as to determine the group weight of the predicted position at the Kth moment in each lane cluster group.

[0153] In addition, if the position observation is the intercept of the lane line or satellite positioning data, the inter-group weights of the lane clustering groups can be updated. The updating process is as described above, so it will not be repeated here.

[0154] Step S4: Determine the position information at the Kth moment based on the lane weight.

[0155] By iterating in this way, the position of the target object can be updated.

[0156] Figure 2G It is the state when the position observation is obtained as satellite positioning data at the Kth moment. Figure 2G The black dot in the middle represents the satellite positioning position. The inter-group weights of the three lane clustering groups at time K are all 1 / 3. Based on the satellite positioning position, the predicted position at time K and its corresponding lane weight can be determined, and then the inter-group weights and intra-group weights can be determined.

[0157] Figure 2H A schematic diagram showing N predicted positions at the Kth moment after the satellite positioning position is updated is shown.

[0158] In this example, because the observed positioning is RTK (real-time kinematic) with a fixed solution, the positioning accuracy is very high (error < 5 cm). Based on this, it can be determined that there is a high probability that the target object is in the second lane, and all particles converge to a nearby position.

[0159] Through the above process, the position weights of the N predicted positions at the Kth moment are updated using the position observations, and then the position weights are normalized to obtain the intra-group weights, so that the sum of the intra-group weights of the predicted positions within the same lane cluster group is 1. In addition, the inter-group weights of lane cluster groups can be updated based on the lane line type and satellite positioning data, so that the sum of the inter-group weights of all lane cluster groups is 1.

[0160] After updating the inter-group weights and intra-group weights, we get an approximate representation of the posterior distribution of the N predicted positions at time K. By repeating the above process, we get an approximate representation of the posterior distribution of the position observations at each moment.

[0161] This meets the high demands for positioning accuracy and reliability in scenarios such as electronic navigation and intelligent driving. The implementation method uses multi-source sensor fusion based on visual semantic recognition results, combined with high-precision maps, to output high-precision, stable and reliable positioning results. Furthermore, this positioning method fully utilizes the advantage of high-precision maps, which contain richer map element information than traditional maps (such as lane geometry and type, ground markers, and various traffic facilities such as traffic signs and traffic lights).

[0162] Image acquisition devices mounted on target objects can obtain a wealth of environmental information (such as pedestrians, vehicles, lane markings, and various traffic facilities). By matching and fusing the elements identified by the image acquisition device with those in the high-precision map, the relative position of the target object relative to the identified elements is combined with the high-precision map to determine the global position of the target object. Integrating this global position into the positioning system can further improve positioning accuracy and reliability.

[0163] Because the iterative process can incorporate information from multiple reference objects (such as poles and lane markings), the poor adaptability of a single reference type is avoided. A multi-state hypothesis model based on particle filtering fundamentally solves the lane association problem, removing the strong constraint that the initial positioning error must be less than half the lane width, and expanding the model's applicability.

[0164] Building on the traditional low-cost inertial navigation fusion positioning solution, the addition of visual semantic recognition input enables continuous high-precision positioning in scenarios with HD map data. This system can adapt to a variety of signal schemes during different iterations. For example, for satellite positioning, it can use both consumer-grade satellite positioning and the more accurate RTK positioning. For visual semantic recognition, high-precision positioning can be achieved with only lane marking recognition, or even without lane marking line shape recognition. Using RTK positioning or richer visual semantic recognition results can improve positioning recall and stability.

[0165] The positioning effect of this method can achieve: the lateral position error is less than 0.2M, accounting for more than 95%; the longitudinal position error is less than 0.5M, accounting for more than 95%; the heading error is less than 0.5 degrees, accounting for more than 95%.

[0166] In summary, this solution proposes a unified fusion positioning framework with good scalability. It can not only perform positioning based on poles, lane lines, and traffic facilities based on visual recognition information, but also integrate any other input that can provide information about its own position (for example, using lidar or visual point cloud to estimate the vehicle's position and posture). In one update, only one input can be used for update, achieving the purpose of separate modeling and updating. The updated predicted position carries the information of all historical inputs, realizing the fusion of multi-source information.

[0167] By abandoning the classic Kalman filter's assumption of a single most probable state, a particle filter-based multi-state hypothesis model is adopted, broadening the model's applicability and significantly improving accuracy and recall. In addition to particle filtering, other analytical models that support multi-state hypotheses can also be used to model state quantities (such as mixture Gaussian models) and combined with the Kalman filter framework for filter estimation.

[0168] The positioning method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a mobile phone, a PAD, etc.) and a PC.

[0169] Example 3

[0170] Reference Figure 3 , shows a structural block diagram of a positioning device according to Example 3 of the present application.

[0171] The positioning device of this embodiment includes:

[0172] A first acquisition module 302 is configured to acquire a position observation value of the target object at the Kth moment, wherein the position observation value includes observation data obtained based on visual semantic information corresponding to an environment image at the location of the target object;

[0173] The second acquisition module 304 is used to obtain N predicted positions of the target object at the Kth moment;

[0174] A calculation module 306 is configured to determine lane weights of the target object at each of the N predicted positions at the Kth moment based on the position observation at the Kth moment and the high-precision map;

[0175] The determination module 308 is configured to obtain the position information of the target object at the Kth moment based on the lane weights of the N predicted positions of the target object at the Kth moment.

[0176] Optionally, the second acquisition module 304 is used to obtain the N predicted positions of the target object at the K-1th moment and the lane weight corresponding to the predicted position at the K-1th moment; and determine the N predicted positions at the Kth moment corresponding to the predicted position at the K-1th moment based on the lane weight corresponding to the predicted position at the K-1th moment.

[0177] Optionally, the second acquisition module 304 is used to determine the N predicted positions at the Kth moment corresponding to the predicted position at the K-1th moment according to the lane weights corresponding to the predicted positions at the K-1th moment, and determine the number of predicted positions at the Kth moment split from each predicted position at the K-1th moment according to the lane weights corresponding to each predicted position at the K-1th moment, and the number is less than or equal to N; determine the N predicted positions at the Kth moment split from all predicted positions at the K-1th moment according to each predicted position at the K-1th moment and the inter-frame displacement from the K-1th moment to the Kth moment, wherein the inter-frame displacement is determined based on the output data and random error of the inertial measurement unit of the target object.

[0178] Optionally, the calculation module 306 is used to determine the position weights of the target object when it is located at the N predicted positions at the Kth moment based on the position observation value and the high-precision map at the Kth moment; determine the lane clustering groups to which the N predicted positions at the Kth moment belong, wherein the number of the lane clustering groups is determined according to the number of lanes contained in the road corresponding to the position of the target object; determine the intra-group weights of the N predicted positions in the lane clustering groups to which they belong based on the position weights of the N predicted positions; determine the inter-group weights between the lane clustering groups; and determine the lane weights corresponding to the N predicted positions based on the intra-group weights and the inter-group weights.

[0179] Optionally, the calculation module 306 is used to determine the position weights of the target object when it is located at the N predicted positions at the Kth moment based on the position observation value at the Kth moment and the high-precision map, and to determine the actual distances of the target object relative to the reference lane line when it is located at the N predicted positions based on the high-precision map; and to determine the position weights corresponding to the N predicted positions based on the intercept of the target object relative to the reference lane line included in the position observation value, the confidence of the intercept, and each actual distance.

[0180] Optionally, the calculation module 306 is used to determine the position weights of the target object when it is located at the N predicted positions at the Kth moment based on the position observation quantity at the Kth moment and the high-precision map, and to determine the actual distance of the target object relative to the reference traffic facility when the target object is located at the N predicted positions based on the high-precision map; and determine the position weights corresponding to the N predicted positions based on the position observation quantity including the reference distance of the target object relative to the reference traffic facility, the confidence of the reference distance, and each actual distance.

[0181] Optionally, the calculation module 306 is used to determine the position weights of the target object when it is located at the N predicted positions at the Kth moment based on the position observation quantity and the high-precision map at the Kth moment, and determine the position weights corresponding to each of the predicted positions based on the position observation quantity including satellite positioning data and the confidence corresponding to the satellite positioning data and the N predicted positions.

[0182] Optionally, the calculation module 306 is used to normalize the position weights of the predicted positions in the current lane clustering group for each lane clustering group when determining the intra-group weights of the N predicted positions in the lane clustering group to which they belong based on the position weights of the N predicted positions; and determine the intra-group weights of the N predicted positions in the lane clustering group to which they belong based on the normalization result.

[0183] Optionally, the calculation module 306 is used to determine the sum of the position weights of the predicted positions within each lane clustering group as the initial weight of the lane clustering group when determining the inter-group weights between the lane clustering groups; perform inter-group normalization on the initial weights of multiple lane clustering groups; and determine the inter-group weights of each lane clustering group based on the results of the inter-group normalization.

[0184] The positioning device of this embodiment is used to implement the corresponding positioning methods in the aforementioned multiple method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the positioning device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be described in detail here.

[0185] Example 4

[0186] Reference Figure 4 , shows a structural diagram of an electronic device according to the fourth embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device.

[0187] like Figure 4As shown, the electronic device may include: a processor (PROCESSOR) 402 , a communication interface (COMMUNICATIONS INTERFACE) 404 , a memory (MEMORY) 406 , and a communication bus 408 .

[0188] in:

[0189] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0190] The communication interface 404 is used to communicate with other electronic devices or servers.

[0191] The processor 402 is configured to execute the program 410 , and specifically may execute the relevant steps in the above positioning method embodiment.

[0192] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0193] Processor 402 may be a CPU, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0194] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (NON-VOLATILE MEMORY), such as at least one disk memory.

[0195] The program 410 may be specifically configured to enable the processor 402 to execute the corresponding steps of the aforementioned positioning method.

[0196] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0197] The above-described method according to the embodiment of the present application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium downloaded via a network and stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, processor or hardware, the positioning method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the positioning method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the positioning method shown herein.

[0198] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of this application.

[0199] The above implementation methods are only used to illustrate the embodiments of the present application, and are not intended to limit the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present application, and the scope of patent protection of the embodiments of the present application should be defined by the claims.

Claims

1. A positioning method, comprising: Obtaining a position observation value of the target object at the Kth moment, the position observation value comprising: observation data obtained based on visual semantic information corresponding to an environment image at the location of the target object; Obtaining N predicted positions of the target object at the Kth moment; Determining, based on the position observation at the Kth moment and the high-precision map, the position weights of the target object at the N predicted positions at the Kth moment; Determining the lane cluster groups to which the N predicted positions at the K-th moment belong, wherein the number of the lane cluster groups is determined according to the number of lanes included in the road corresponding to the position of the target object; Determining, based on the position weights of the N predicted positions, the in-group weights of the N predicted positions in the lane cluster group to which they belong; Determining inter-group weights between the lane cluster groups; Determining lane weights corresponding to the N predicted positions based on the intra-group weights and the inter-group weights; The position information of the target object at the Kth moment is obtained according to the lane weights of the N predicted positions of the target object at the Kth moment.

2. The method according to claim 1, wherein The obtaining of N predicted positions of the target object at the Kth moment includes: Obtaining N predicted positions of the target object at the K-1th moment and lane weights corresponding to the predicted positions at the K-1th moment; Determine N predicted positions at the Kth moment corresponding to the predicted position at the K-1th moment based on the lane weights corresponding to the predicted position at the K-1th moment.

3. The method according to claim 2, wherein: The determining, based on lane weights corresponding to the predicted position at the K-1th moment, N predicted positions at the K-th moment corresponding to the predicted position at the K-1th moment, includes: Determining, based on lane weights corresponding to the respective predicted positions at the K-1th time, a number of predicted positions at the K-1th time split from the respective predicted positions at the K-1th time, the number being less than or equal to N; Based on the predicted positions at the K-1th moment and the inter-frame displacement from the K-1th moment to the Kth moment, determine the N predicted positions at the Kth moment split from all the predicted positions at the K-1th moment, wherein the inter-frame displacement is determined based on the output data and random error of the inertial measurement unit of the target object.

4. The method according to claim 1, wherein The determining, based on the position observation at the K-th moment and the high-precision map, the position weights of the target object when it is located at the N predicted positions at the K-th moment includes: Determining, based on the high-precision map, actual distances of the target object relative to a reference lane line when the target object is located at each of the N predicted positions; The position weights corresponding to the N predicted positions are determined based on the intercept of the target object relative to the reference lane line, the confidence of the intercept, and each actual distance included in the position observation.

5. The method according to claim 1, wherein Determining the position weights of the target object at each of the N predicted positions at the Kth moment based on the position observation at the Kth moment and the high-precision map includes: determining, based on the high-precision map, the actual distance of the target object relative to the reference transportation facility when the target object is at each of the N predicted positions; The position weights corresponding to the N predicted positions are determined based on the position observations including the reference distance of the target object relative to the reference traffic facility, the confidence level of the reference distance, and each actual distance.

6. The method according to claim 1, wherein The determining, based on the position observation at the K-th moment and the high-precision map, the position weights of the target object when it is located at the N predicted positions at the K-th moment includes: According to the position observation quantity including satellite positioning data and the confidence corresponding to the satellite positioning data and the N predicted positions, a position weight corresponding to each predicted position is determined.

7. The method according to claim 1, wherein Determining, based on the position weights of the N predicted positions, the in-group weights of the N predicted positions in the lane cluster group to which they belong includes: For each lane cluster group, normalizing the position weight of the predicted position in the current lane cluster group; According to the normalization result, the intra-group weights of the N predicted positions in the lane cluster group to which they belong are determined.

8. The method according to claim 1, wherein Determining the inter-group weights between the lane clustering groups includes: Determining a sum of the position weights of the predicted positions in each lane cluster group as an initial weight of the lane cluster group; performing inter-group normalization on initial weights of the plurality of lane clustering groups; According to the result of inter-group normalization, the inter-group weights of the lane clustering groups are determined.

9. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the positioning method according to any one of claims 1 to 8 is implemented.

10. A computer program product, which implements the positioning method according to any one of claims 1 to 8 when executed by a processor.

Images