Method and apparatus for determining confidence, electronic device, and storage medium

Abstract

Embodiments of the present application provide a method and device for determining confidence, electronic equipment and storage medium. The method for determining confidence comprises: determining a plurality of feature points of current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor; obtaining a three-dimensional residual confidence and a two-dimensional likelihood confidence of the pose information according to the plurality of feature points and the pose information; and determining the confidence of the pose information corresponding to the current frame point cloud data after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence. Since the three-dimensional residual confidence can reduce mutual interference between dimensions, improve detection of positioning divergence, and the two-dimensional likelihood confidence can improve detection of positioning divergence in an open scene, the confidence of the pose information is reliably determined.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, and particularly relate to a method and device for determining confidence, an electronic device and a storage medium. BACKGROUND

[0002] In an autonomous driving system, a positioning module provides real-time global pose information for an unmanned vehicle. Laser positioning is a very important sub-module of the positioning module. Laser positioning needs to determine whether the output pose of the unmanned vehicle is accurate, so as to discover the divergence of laser positioning in time and ensure the safety requirements of the unmanned vehicle.

[0003] However, there is no reliable scheme for determining whether the pose is accurate in the prior art. SUMMARY

[0004] Therefore, embodiments of the present application provide a method and device for determining confidence, an electronic device and a storage medium to solve or alleviate the above problems.

[0005] According to a first aspect of embodiments of the present application, a method for determining confidence is provided, comprising: determining a plurality of feature points of current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor; obtaining a three-dimensional residual confidence and a two-dimensional likelihood confidence of the pose information according to the plurality of feature points and the pose information; and determining the confidence of the pose information corresponding to the current frame point cloud data after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence.

[0006] According to a second aspect of embodiments of the present application, a device for determining confidence is provided, comprising: a first determination module configured to determine a plurality of feature points of current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor; an obtaining module configured to obtain a three-dimensional residual confidence and a two-dimensional likelihood confidence of the pose information according to the plurality of feature points and the pose information; and a second determination module configured to determine the confidence of the pose information corresponding to the current frame point cloud data after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence.

[0007] According to a third aspect of embodiments of the present application, an electronic device is provided, which comprises a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the method of the first aspect.

[0008] According to a fourth aspect of the embodiments of the present application, a storage medium is provided, which has stored thereon a computer program, and the computer program is executed by a processor to implement the method according to the first aspect.

[0009] In the scheme of the embodiments of the present application, a plurality of feature points of the current frame point cloud data are determined, and the sensor collects the pose information corresponding to the current frame point cloud data; according to the plurality of feature points and the pose information, a three-dimensional residual confidence of the pose information and a two-dimensional likelihood confidence are obtained; after the three-dimensional residual confidence and the two-dimensional likelihood confidence are fused, the confidence of the pose information corresponding to the current frame point cloud data is determined. Since the three-dimensional residual confidence can reduce the mutual interference between the dimensions, improve the detection of positioning divergence, and the two-dimensional likelihood confidence can improve the detection of positioning divergence in an open scene, the confidence of the pose information is reliably determined. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0011] Figure 1 The schematic flow chart of the method for determining the confidence of an embodiment of the present application;

[0012] Figure 2 The schematic flow chart of the method for determining the confidence of another embodiment of the present application;

[0013] Figure 3 The schematic flow chart of the method for determining the confidence of another embodiment of the present application;

[0014] Figure 4A The schematic flow chart of the method for determining the confidence of another embodiment of the present application;

[0015] Figure 4B The schematic flow chart of the method for determining the confidence of another embodiment of the present application;

[0016] Figure 5 The schematic block diagram of the device for determining the confidence of another embodiment of the present application; and

[0017] Figure 6 The hardware structure of the electronic device of another embodiment of the present application. DETAILED DESCRIPTION

[0018] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art shall fall within the scope of protection of the present application.

[0019] The specific implementation of the embodiments of the present application will be further described below in conjunction with the drawings of the embodiments of the present application.

[0020] Figure 1 The schematic flowchart of the method for determining confidence of an embodiment of the present application. Figure 1 The method for determining confidence can be applied to any appropriate electronic device with data processing capability, including but not limited to: servers, car machines, autonomous driving systems, mobile terminals (such as mobile phones, PADs, etc.), and PC machines, sensors with data processing capability, etc. The method comprises:

[0021] 110: determining a plurality of feature points of the current frame point cloud data and collecting pose information corresponding to the current frame point cloud data by the sensor.

[0022] It should be understood that the environmental information can be perceived by the sensor, for example, a camera, a laser radar, a millimeter wave radar, and an industrial camera are used to obtain the environmental information. The set of feature points of the current frame point cloud data can be a set of feature points after motion compensation. The current frame point cloud data is collected. The global positioning system (GPS) or the like can be used to obtain the pose information. The current frame point cloud data can be collected by a sensor such as a laser radar. For example, the sensors in the present application can all be laser sensors, wherein the number of sensors can be one or more. The sensor can be installed obliquely, horizontally, or laterally. Preferably, the sensor is installed horizontally. For example, a plurality of sensors are arranged at different positions of a vehicle such as an autonomous driving vehicle. One or more lasers on the vehicle can be used to generate point cloud information of the surrounding environment at a certain frequency.

[0023] 120: obtaining a three-dimensional residual confidence of the pose information and a two-dimensional likelihood confidence according to the plurality of feature points and the pose information.

[0024] It should be understood that in one example, a set of feature points can be transformed from the vehicle body coordinate system to the global coordinate system according to the pose information, and the likelihood probability of the transformed set of feature points based on the global coordinate system is determined. For example, the global coordinate system-based map can be a three-dimensional point cloud map or a two-dimensional occupancy grid map. In another example, a set of feature points can be transformed from the global coordinate system to the vehicle body coordinate system according to the pose information, and the residual confidence of the set of feature points based on the vehicle body coordinate system is determined.

[0025] It should also be understood that the set of feature points can be feature points that meet the matching rate. For example, among the point surface vectors corresponding to the plurality of feature points, the selected point surface vector that meets the matching rate threshold corresponds to the set of feature points.

[0026] 130: After fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence, the confidence of the pose information corresponding to the current frame point cloud data is determined.

[0027] It should be understood that the confidence in the text can be represented as a probability value between 0 and 1, or the above-mentioned probability value can be binarized and determined as the confidence. The accuracy of the confidence can represent the divergence of the pose positioning, which can be reduced by setting an accuracy range, calculating the posterior probability of the likelihood probability based on the accuracy range as the confidence, and reducing the divergence of the pose positioning.

[0028] In the scheme of the embodiments of the present application, a plurality of feature points of the current frame point cloud data and the pose information corresponding to the current frame point cloud data collected by the sensor are determined; the three-dimensional residual confidence and the two-dimensional likelihood confidence of the pose information are obtained according to the plurality of feature points and the pose information; and after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence, the confidence of the pose information corresponding to the current frame point cloud data is determined. Since the three-dimensional residual confidence can reduce the mutual interference between the dimensions, improve the detection of the positioning divergence, and the two-dimensional likelihood confidence can improve the detection of the positioning divergence in the open scene, the confidence of the pose information is reliably determined.

[0029] Of course, according to the different types of autonomous vehicles, the above-mentioned algorithm modules will also be different. For example, different algorithm modules will be involved for logistics vehicles, public service vehicles, medical service vehicles, and terminal service vehicles. The algorithm modules will be illustrated below for the four kinds of autonomous vehicles respectively:

[0030] Among them, the logistics vehicle refers to the vehicle used in the logistics scene, for example, it can be a logistics vehicle with automatic sorting function, a logistics vehicle with cold storage function, and a logistics vehicle with measurement function. These logistics vehicles will involve different algorithm modules.

[0031] For example, for a logistics vehicle, the above-mentioned algorithm can be executed with a positioning module, which can automatically take out and carry, sort and store the goods accurately during the driving of the logistics vehicle and after reaching the destination. This involves a positioning module for goods sorting, which mainly implements positioning control during the processes of goods taking out, carrying, sorting and storing, and driving, etc.

[0032] For example, in most logistics scenarios, charges are made according to the volume and / or weight of the package, and the number of logistics packages is very large. Simply relying on the courier to measure the volume and / or weight of the package is very inefficient and has high labor costs. Therefore, in some scenarios, a logistics vehicle is used to perform the above-mentioned operation. This involves a positioning module for a logistics vehicle, which is mainly used to determine the pose and determine the confidence of the pose information.

[0033] Among them, the public service vehicle refers to a vehicle that provides a certain public service, which can be a fire truck, an ice removal vehicle, a water truck, a snow plow, a garbage disposal vehicle, a traffic control vehicle, etc. These public service vehicles will involve different algorithm modules.

[0034] For example, for an automatic driving fire truck, its main task is to perform a reasonable fire extinguishing task on the fire scene. In order to better complete the above-mentioned task, this involves a positioning module for a fire extinguishing task, which is at least used to determine the pose and determine the confidence of the pose information.

[0035] For example, for an ice removal vehicle, its main task is to remove the ice and snow on the road surface. This involves an ice removal positioning module, which at least needs to realize the positioning recognition of the ice removal vehicle on the road surface, which can be combined with the automatic control logic of other ice removal schemes to realize an efficient and reliable ice removal process.

[0036] Among them, the medical service vehicle refers to an automatic driving vehicle that can provide one or more medical services. This vehicle can provide disinfection, temperature measurement, medicine dispensing, isolation and other medical services. This involves a positioning module that cooperates with various self-service medical services, which mainly implements the determination of the pose and the determination of the confidence of the pose information to realize accurate positioning control. It should be understood that the positioning module can also cooperate with other modules to realize the recognition of disinfection requirements and the control of disinfection devices to disinfect patients, or the recognition of patient positions to control temperature measurement devices to automatically approach the positions such as the forehead of the patient to measure the temperature of the patient, or to realize the judgment of the disease, give a prescription according to the judgment result, and realize the recognition of the medicine / medicine container and the control of the medicine taking robot to pick up the medicine for the patient according to the prescription, etc.

[0037] The terminal service vehicle refers to a self-service automatic driving vehicle that can provide certain convenient services to users instead of some terminal devices, for example, the vehicle can provide users with printing, attendance, scanning, unlocking, payment, retail and other services.

[0038] For example, in some application scenarios, users often need to go to a specific location to print or scan documents, which is time-consuming and laborious. Therefore, a terminal service vehicle that can provide printing / scanning services for users appears, which has a positioning module that mainly implements the determination of the execution pose and the determination of the confidence of the pose information to achieve accurate positioning control. It should be understood that the positioning module can also cooperate with other modules to achieve various functions and purposes, for example, interconnection with a user terminal device, the user issues a printing instruction through the terminal device, the service vehicle responds to the printing instruction, automatically prints the document required by the user and can automatically deliver the printed document to the user's location, the user does not need to go to the printer to queue, which can greatly improve the printing efficiency. Alternatively, the service vehicle can move to the user's location in response to the scanning instruction issued by the user through the terminal device, the user places the document to be scanned on the scanning tool of the service vehicle to complete the scanning, and does not need to go to the printer / scanner to queue, which saves time and effort. This involves an algorithm module for providing printing / scanning services, which at least needs to identify the interconnection with the user terminal device, the response of the printing / scanning instruction, the positioning of the user's location and the travel control, etc.

[0039] For example, with the development of new retail service scenarios, more and more e-commerce companies use vending machines to sell goods in various office buildings and public areas. However, these vending machines are fixed and cannot be moved, and users need to be close to the vending machine to purchase the desired goods, which is not very convenient. Therefore, a self-driving vehicle providing retail services is provided. The vehicle has a positioning module that mainly determines the pose and the confidence of the pose information to achieve accurate positioning control. It should be understood that the positioning module can also cooperate with other modules to achieve various functions and purposes, such as automatically moving and carrying goods and providing a corresponding self-service shopping APP or shopping portal. Users can use a terminal such as a mobile phone to place an order through the APP or shopping portal, which includes the name and quantity of the goods to be purchased and the user's location. After receiving the order, the vehicle can determine whether it has the goods and sufficient quantity to meet the user's purchase. If it does, it can automatically move to the user's location and provide the goods to the user, further improving the convenience of shopping and saving the user's time so that the user can use the time for more important things. This involves an algorithm module that provides retail services. These algorithm modules mainly implement logic such as responding to user order requests, order processing, goods information maintenance, user location positioning, and payment management.

[0040] In another implementation of the present application, the three-dimensional residual confidence of the pose information is obtained according to the plurality of feature points and the pose information, comprising: in the world coordinate system in which the pose information and the current frame point cloud data are located, the plurality of feature points are matched with the point cloud data by an iterative nearest point algorithm to obtain a plurality of point-plane vectors; the plurality of point-plane vectors are transformed to a vehicle body coordinate system indicated by the pose information; based on three dimensions of the vehicle body coordinate system, the plurality of transformed point-plane vectors are decomposed to determine statistical information of three groups of vector components; and the three-dimensional residual confidence of the pose information is calculated as a first confidence according to the statistical information of the three groups of vector components.

[0041] Since the point cloud map is in the world coordinate system, the plurality of point-plane vectors are obtained by matching in the world coordinate system, and accurate plurality of point-plane vectors are obtained by a small amount of calculation. In addition, the statistical information of the three groups of vector components of the plurality of point-plane vectors is determined based on the three dimensions of the vehicle body coordinate system, which is more conducive to reducing the interference between the vector components of different dimensions, and thus reliably judging the accuracy of the pose information.

[0042] In other words, the residual information of the point cloud iterative nearest point matching is dimensionally decomposed, and the pose confidence between different dimensions is calculated respectively, which reduces the interference between different dimensions and further improves the detection rate of pose divergence.

[0043] In addition, compared with a scheme in which mutual influences between different dimensions are not considered when using point cloud matching information, the scheme of the embodiment of the application adopts a dimension decomposition manner, improves the judgment accuracy of the confidence of the pose, and improves the detection rate of the positioning divergence.

[0044] In one example, the set of feature points includes feature points of at least one of a ground, a road shoulder, a wall surface, and a rod-shaped object.

[0045] In another implementation of the application, based on three dimensions of the vehicle body coordinate system, the transformed plurality of point surface vectors are decomposed to determine statistical information of the corresponding three groups of vector components, including: based on the three dimensions of the vehicle body coordinate system respectively, the transformed plurality of point surface vectors are decomposed to obtain the weights of the corresponding three dimensions, and finally the residual three-dimension weighted mean and variance are counted, wherein the three dimensions include a lateral direction, a longitudinal direction, and a height direction.

[0046] Since the lateral direction, the longitudinal direction, and the height direction can reflect the different accuracies of the vehicle body coordinate system when moving, the interference between the above different dimensions is reduced. In addition, the residual mean and variance are reliable statistical indicators, and they can improve the efficiency of the calculation.

[0047] In another implementation of the application, based on the three dimensions of the vehicle body coordinate system respectively, the transformed plurality of point surface vectors are decomposed to obtain the weights of the corresponding three dimensions, and finally the residual three-dimension weighted mean and variance are counted, including: based on the three dimensions of the vehicle body coordinate system respectively, the transformed plurality of point surface vectors are decomposed to determine the respective residuals of the three groups of vector components; according to the respective weight information of the transformed plurality of point surface vectors in the three dimensions of the vehicle body coordinate system and the respective residuals of the three groups of vector components, the respective residual mean and variance of the three groups of vector components are calculated.

[0048] Since the respective weight information of the transformed plurality of point surface vectors in the three dimensions of the vehicle body coordinate system can reflect the influence degree of each dimension on the pose, the respective residual mean and variance of the three groups of vector components are determined according to the weight information, so that the calculation accuracy of the respective residual mean and variance of the three groups of vector components is improved.

[0049] Specifically, Figure 2A method of determining residual confidence of another embodiment of the present application is shown. As shown, a set of point-plane vectors can be determined based on a matching rate threshold. For example, the matching rate of a plurality of point-plane vectors is higher than or not lower than the matching rate threshold. The plurality of point-plane vectors are decomposed into three dimensions of lateral, longitudinal and height directions to obtain a set of vector lateral components, a set of vector longitudinal components and a set of height direction components. Residuals and variances of the three sets of vector components are calculated and input into a normalized confidence function to obtain a residual confidence as a first confidence of the pose information.

[0050] In one example, the dimension decomposition of the point-plane vectors on N matches is projected to the vehicle body coordinate system by the pose, which can be calculated by the following formula (1):

[0051]

[0052] wherein n i g represents the point-plane vector in the world coordinate system; n i b represents the point-plane vector in the vehicle body coordinate system corresponding to the pose information; R g b represents the transformation matrix from the world coordinate system to the vehicle body coordinate system determined based on the pose information.

[0053] In addition, the weight of each feature point in the lateral (x), longitudinal (y) and height (z) directions can be calculated by the following formula (2):

[0054]

[0055] wherein p(i, x) represents the probability of the point-plane vector n i b in the lateral direction; p(i, y) represents the probability of the point-plane vector n i b in the longitudinal direction; p(i, z) represents the probability of the point-plane vector n i b in the height direction; ω(i, x) represents the weight of the probability p(i, x) in the sum of probabilities of all point-plane vectors; ω(i, y) represents the weight of the probability p(i, y) in the sum of probabilities of all point-plane vectors; ω(i, z) represents the weight of the probability p(i, z) in the sum of probabilities of all point-plane vectors.

[0056] In addition, the mean and variance of the residuals of the three dimensions can be calculated by the following formula (3):

[0057]

[0058] where, mean x represents the residual d i corresponding to the lateral direction; std x represents the residual d i corresponding to the lateral direction; mean y represents the residual d i corresponding to the longitudinal direction; std y represents the residual d i corresponding to the longitudinal direction; mean i represents the residual d i corresponding to the height direction; std

[0059] In addition, the normalized confidence function can be calculated by the following formula (4):

[0060]

[0061] where, s(x) and q(x) represent the normalization functions; score represents the matching residual normalized function s(x); q represents the match rate function normalized function. For the calculation of the score of each dimension, it can be calculated by the following formula (5):

[0062] score x = s(mean x ) * s(std x ) * q(ratio xy )

[0063] score y = s(mean y ) * s(std y ) * q(ratio xy )

[0064] score2 = s(mean z ) * s(std z ) * q(ratio z ) (5)

[0065] where, score x represents the confidence score of the lateral direction; score y represents the confidence score of the longitudinal direction; score z represents the confidence score of the height direction.

[0066] In another implementation of the present application, the two-dimensional likelihood confidence of the pose information is obtained according to the plurality of feature points and the pose information, comprising: projecting the plurality of feature points into a two-dimensional coordinate system in which the current occupancy grid map of the point cloud data is located, to obtain a plurality of two-dimensional feature points; determining the likelihood probability of the plurality of two-dimensional feature points corresponding to the current occupancy grid map according to a likelihood domain model; and determining the two-dimensional likelihood confidence of the pose information as the second confidence according to the likelihood probability.

[0067] Since the current occupancy grid map is in a two-dimensional coordinate system, the likelihood probability of the set of two-dimensional feature points determined according to the likelihood domain model reduces the influence of environmental changes, and thus an accurate confidence of the pose information is obtained.

[0068] In addition, compared with the scheme of only considering the coincidence rate when using a two-dimensional grid map, the scheme of the embodiments of the present application uses a likelihood probability model of the occupancy grid map, improves the judgment accuracy of the confidence of the pose, and improves the detection rate of the localization divergence.

[0069] In one example, the set of feature points includes feature points of at least one of a road shoulder, a wall surface, and a rod-shaped object.

[0070] In another implementation of the present application, the method further comprises: determining the point cloud data of the historical key frame according to the movement distance and the angle change of the sensor; and updating the current occupancy grid map according to the logarithmic geometric state of the point cloud data of the historical key frame.

[0071] Since the historical point cloud key frame determined according to the movement distance and the angle change of the sensor can more accurately reflect the changes of the point cloud map, the current occupancy grid map is reliably updated using the historical point cloud key frame. In addition, the logarithmic geometric state is conducive to efficient calculation, and thus the current occupancy grid map is updated using the logarithmic geometric state of the historical point cloud key frame, improving the calculation efficiency. In addition, updating the current occupancy grid map maintains the occupancy grid map in real time, more flexibly adjusts the parameters, and detects the localization divergence.

[0072] In addition, compared with the scheme of performing a prior likelihood probability calculation on the occupancy grid map and then judging whether the localization diverges by comparing the proportional relationship between the real-time pose likelihood probability and the prior calculated likelihood probability, in the scheme of the embodiments of the present application, the likelihood probability is updated accordingly with the update of the occupancy grid map, improving the overall stability of the system.

[0073] In another implementation of the present application, the two-dimensional likelihood confidence of the pose information is obtained according to the likelihood probability, comprising: binarizing the likelihood probability according to the accuracy range of the likelihood probability to obtain the two-dimensional likelihood confidence of the pose information as the second confidence.

[0074] Since the accuracy range of the likelihood probability can reflect the accuracy of the confidence, the second confidence of the pose information is determined according to the accuracy range of the likelihood probability, and the reliability of the obtained confidence is improved.

[0075] Specifically, Figure 3 A method for determining the confidence of the likelihood of another embodiment of the application is shown. As shown, a set of feature points is transformed into a world coordinate system to obtain a set of transformed feature points. The transformed feature points are projected onto the XY plane and matched with the occupancy grid map, or can be directly projected into the occupancy grid map and input into the divergence detection module. According to the accuracy range of the likelihood probability, the likelihood probability is binarized to obtain the second confidence of the pose information.

[0076] In one example, the historical frame information of the pose passing the confidence check is maintained and updated in the form of key frames using the logarithmic geometric state update algorithm, which can be calculated using the following formula (6):

[0077]

[0078] wherein, l t,i represents the logarithmic geometric state of the key frame corresponding to the grid i at time t.

[0079] In addition, the likelihood probability corresponding to the occupancy grid map can be calculated using the following formula (7):

[0080]

[0081] wherein, P occupied represents the update probability corresponding to the end point in the occupancy grid map; P free represents the update probability corresponding to the intermediate point in the occupancy grid map; P unknown represents the update probability corresponding to other positions in the occupancy grid map.

[0082] In addition, the likelihood probability value of the domain occupancy grid map can be calculated using the likelihood domain model. It can be calculated using the following formula (8):

[0083]

[0084] wherein, the likelihood probability model p(z t k | x t , m) represents the likelihood probability value of the feature point k in the frame point cloud data collected at the pose x t corresponding to the map m at time t.

[0085] In addition, the divergence detection module (divergence judgment module) judges whether the positioning exceeds the accuracy range according to the size of the likelihood probability value of the current frame. For example, the following formula (9) can be used for empirical parameter adjustment to determine the threshold value:

[0086]

[0087] wherein p th represents the accuracy range of the confidence degree; P fai lure represents the final confidence degree.

[0088] In another implementation of the present application, the confidence degree of the pose information is determined according to the transformed set of feature points, comprising: determining the residual confidence degree of the pose information according to the transformed set of feature points by the iterative nearest point algorithm, and determining the likelihood difference confidence degree of the pose information according to the transformed set of feature points by the current occupancy grid map; determining the confidence degree of the pose information according to the residual confidence degree and the likelihood confidence degree, wherein if at least one of the residual confidence degree and the likelihood confidence degree indicates that the pose information is positioning divergence, the confidence degree of the pose information indicates that the pose information is positioning divergence.

[0089] Since the residual confidence degree can improve the efficiency of the calculation, and the likelihood confidence degree more flexibly adjusts the parameters, the confidence degree of the pose information is determined according to the residual confidence degree and the likelihood confidence degree, which greatly improves the accuracy of the confidence degree of the pose.

[0090] Figure 4A The schematic flowchart of the method for determining the confidence degree of another embodiment of the present application. Figure 4A The method for determining the confidence degree can be applied to any appropriate electronic device with data processing capability, including but not limited to: servers, car machines, autonomous driving systems, mobile terminals (such as mobile phones, PADs, etc.), PC machines, sensors with data processing capability, etc. The method comprises:

[0091] 410: determining a set of feature points of the current frame point cloud data and the pose information when the current frame point cloud data is collected. This step can be used as an example of determining a plurality of feature points of the current frame point cloud data and the pose information corresponding to the sensor collecting the current frame point cloud data.

[0092] 420: performing coordinate transformation on the set of feature points according to the pose information. This step can be used as the first step of obtaining the three-dimensional residual confidence degree and the two-dimensional likelihood confidence degree of the pose information according to the plurality of feature points and the pose information.

[0093] 430: determining a residual confidence of the pose information according to the transformed set of feature points by the iterative closest point algorithm, and determining a likelihood confidence of the pose information according to the transformed set of feature points by the current occupancy grid map. This step can be as a second step of obtaining a three-dimensional residual confidence and a two-dimensional likelihood confidence of the pose information according to the set of feature points and the pose information.

[0094] 440: determining a confidence of the pose information according to the residual confidence and the likelihood confidence, wherein if at least one of the residual confidence and the likelihood confidence indicates that the pose information is localization divergent, the confidence of the pose information indicates that the pose information is localization divergent. This step can be as an example of determining the confidence of the pose information corresponding to the current frame point cloud data after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence.

[0095] Specifically, Figure 4B A method of fusing the confidence is shown for another embodiment of the present application. As shown, a set of feature points and pose information are processed in an iterative closest point residual confidence processing module based on a vehicle body coordinate system to obtain a residual confidence. The set of feature points and the pose information are processed in an occupancy grid map likelihood confidence processing module based on a world coordinate system to obtain a likelihood confidence. The residual confidence and the likelihood confidence are input to a fusion confidence processing module for processing (e.g., logical AND processing), and if at least one of the residual confidence and the likelihood confidence indicates that the pose information is localization divergent, the confidence of the pose information indicates that the pose information is localization divergent.

[0096] In one example, determining the residual confidence of the pose information according to the transformed set of feature points by the iterative closest point algorithm includes: matching the transformed set of feature points with a point cloud map in a world coordinate system in which the pose information and the point cloud map are located by the iterative closest point algorithm to obtain a plurality of point-plane vectors; transforming the plurality of point-plane vectors to a vehicle body coordinate system indicated by the pose information; decomposing the transformed plurality of point-plane vectors based on three dimensions of the vehicle body coordinate system to determine statistical information of three corresponding sets of vector components; and calculating a first confidence of the pose information according to the statistical information of the three sets of vector components.

[0097] In another example, determining the likelihood confidence of the pose information according to the transformed set of feature points by the current occupancy grid map includes: projecting the transformed set of feature points into a two-dimensional coordinate system in which a current occupancy grid map of the point cloud map is located to obtain a set of two-dimensional feature points; determining a likelihood probability that the set of two-dimensional feature points corresponds to the current occupancy grid map according to a likelihood domain model; and determining a second confidence of the pose information according to the likelihood probability.

[0098] Figure 5A schematic block diagram of a device for determining confidence according to another embodiment of the present application. Figure 5 The device for determining confidence can be applied to any appropriate electronic device with data processing capability, including but not limited to: servers, car machines, autonomous driving systems, mobile terminals (such as mobile phones, PADs, etc.), PC machines, sensors with data processing capability, etc. The device comprises:

[0099] A first determination module 510 is configured to determine a plurality of feature points of current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor.

[0100] An acquisition module 520 is configured to obtain a three-dimensional residual confidence of the pose information and a two-dimensional likelihood confidence according to the plurality of feature points and the pose information.

[0101] A second determination module 530 is configured to determine a confidence of the pose information corresponding to the current frame point cloud data after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence.

[0102] In the scheme of the embodiments of the present application, a plurality of feature points of current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor are determined; a three-dimensional residual confidence of the pose information and a two-dimensional likelihood confidence are obtained according to the plurality of feature points and the pose information; and a confidence of the pose information corresponding to the current frame point cloud data is determined after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence. Since the three-dimensional residual confidence can reduce mutual interference between dimensions, improve detection of positioning divergence, and the two-dimensional likelihood confidence can improve detection of positioning divergence in an open scene, the confidence of the pose information is reliably determined.

[0103] In another implementation manner of the present application, the second determination module is specifically configured to: in a world coordinate system in which the pose information and the current frame point cloud data are located, match the plurality of feature points with the point cloud data through an iterative nearest point algorithm to obtain a plurality of point-surface vectors; transform the plurality of point-surface vectors to a vehicle body coordinate system indicated by the pose information through the pose information; decompose the plurality of transformed point-surface vectors based on three dimensions of the vehicle body coordinate system to determine statistical information of three groups of vector components; and calculate the three-dimensional residual confidence of the pose information as a first confidence according to the statistical information of the three groups of vector components.

[0104] Since the point cloud map is in the world coordinate system, the plurality of point-surface vectors are obtained by matching in the world coordinate system, and accurate plurality of point-surface vectors are obtained through a small amount of calculation. In addition, the statistical information of the three groups of vector components of the plurality of point-surface vectors is determined based on the three dimensions of the vehicle body coordinate system, which is more conducive to reducing the interference of vector components between different dimensions, and thus the accuracy of the pose information is reliably determined.

[0105] In other words, the residual information of the point cloud iterative nearest neighbor matching is dimensionally decomposed, and the pose confidence in different dimensions is calculated respectively, thereby reducing the interference between different dimensions and improving the detection rate of pose divergence.

[0106] In another implementation of the present application, the second determination module is specifically configured to: decompose the plurality of transformed point-plane vectors based on three dimensions of the vehicle body coordinate system respectively, to obtain the weights of the three dimensions respectively, and finally to calculate the three-dimension weighted mean and variance of the residual, wherein the three dimensions include the lateral direction, the longitudinal direction and the height direction.

[0107] Since the lateral direction, the longitudinal direction and the height direction can reflect the different accuracies of the vehicle body coordinate system in motion, the interference between the above different dimensions is reduced. In addition, the residual mean and variance are reliable statistical indicators, and they can improve the calculation efficiency.

[0108] In another implementation of the present application, the second determination module is specifically configured to: decompose the plurality of transformed point-plane vectors based on three dimensions of the vehicle body coordinate system respectively, to determine the respective residuals of the three groups of vector components; and calculate the mean and variance of the respective residuals of the three groups of vector components according to the respective weight information of the plurality of transformed point-plane vectors in the three dimensions of the vehicle body coordinate system and the respective residuals of the three groups of vector components.

[0109] Since the respective weight information of the plurality of transformed point-plane vectors in the three dimensions of the vehicle body coordinate system can reflect the influence degree of each dimension on the pose, the mean and variance of the respective residuals of the three groups of vector components are determined according to the weight information, thereby improving the calculation accuracy of the mean and variance of the respective residuals of the three groups of vector components.

[0110] In another implementation of the present application, the second determination module is specifically configured to: project the plurality of feature points into a two-dimensional coordinate system in which a current occupancy grid map of the point cloud data is located, to obtain a plurality of two-dimensional feature points; determine the likelihood probability of the plurality of two-dimensional feature points corresponding to the current occupancy grid map according to a likelihood domain model; and determine the two-dimensional likelihood confidence of the pose information as the second confidence according to the likelihood probability.

[0111] Since the current occupancy grid map is in a two-dimensional coordinate system, the likelihood probability of a group of two-dimensional feature points determined according to the likelihood domain model reduces the influence of environmental changes, and thus an accurate confidence of the pose information is obtained.

[0112] In another implementation of the present application, the device further comprises: an updating module configured to determine the point cloud data of a historical key frame according to the moving distance and the angle change of the sensor; and update the current occupancy grid map according to the logarithmic geometric state of the point cloud data of the historical key frame.

[0113] Since the historical point cloud key frame determined according to the moving distance and the angle change of the sensor can more accurately reflect the change of the point cloud map, the current occupancy grid map is reliably updated by using the historical point cloud key frame. In addition, the logarithmic geometric state is conducive to efficient calculation, so that the current occupancy grid map is updated by using the logarithmic geometric state of the historical point cloud key frame, and the calculation efficiency is improved. In addition, updating the current occupancy grid map maintains the occupancy grid map in real time, and more flexibly adjusts the parameters to detect the divergence of the pose.

[0114] In another implementation manner of the present application, the second determination module is specifically configured to: according to the accuracy range of the likelihood probability, binarize the likelihood probability to obtain a two-dimensional likelihood confidence of the pose information as the second confidence.

[0115] Since the accuracy range of the likelihood probability can reflect the accuracy of the confidence, the second confidence of the pose information is determined according to the accuracy range of the likelihood probability, and the reliability of the obtained confidence is improved.

[0116] In another implementation manner of the present application, the second determination module is specifically configured to: determine a residual confidence of the pose information according to the transformed set of feature points by an iterative nearest point algorithm, and determine a likelihood difference confidence of the pose information according to the transformed set of feature points by the current occupancy grid map; determine the confidence of the pose information according to the residual confidence and the likelihood confidence, wherein if at least one of the residual confidence and the likelihood confidence indicates that the pose information is localization divergence, the confidence of the pose information indicates that the pose information is localization divergence.

[0117] The device of the embodiment is used to implement the corresponding method in the plurality of method embodiments described above, and has the beneficial effects of the corresponding method embodiments, which will not be described here. In addition, the function implementation of each module in the device of the embodiment can be referred to the description of the corresponding part in the method embodiments described above, which will not be described here either.

[0118] Figure 6 The hardware structure of the electronic device of another embodiment of the present application is shown in FIG. 6, which can include a processor 601, a communication interface 602, a storage medium 603 and a communication bus 604. Figure 6

[0119] The processor 601, the communication interface 602 and the storage medium 603 can complete mutual communication through the communication bus 604.

[0120] Optionally, the communication interface 602 can be the interface of the communication module.

[0121] ​The processor 601 can be specifically configured to determine a plurality of feature points of current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor, obtain a three-dimensional residual confidence and a two-dimensional likelihood confidence of the pose information according to the plurality of feature points and the pose information, and determine a confidence of the pose information corresponding to the current frame point cloud data after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence.

[0122] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general processor can be a microprocessor or the processor can also be any conventional processor.

[0123] The storage medium described above can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0124] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a storage medium, the computer program containing program code configured to perform the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication section, and / or installed from a detachable medium. When the computer program is executed by a central processing unit (CPU), the above-described functions defined in the methods of the present application are performed. It should be noted that the storage medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The storage medium may, for example, but not limited to, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, in which a computer readable program code is carried. Such a propagated data signal can take many forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium can also be any medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the storage medium can be transmitted by any suitable medium, including but not limited to, wireless, wire line, optical fiber, RF, or any suitable combination of the above.

[0125] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0126] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for performing the specified functions. The specific implementations described in the detailed description have been provided for the purpose of illustration only and the application is not limited to these specific implementations.

[0127] The modules involved in the embodiments described in the present application can be implemented in software or in hardware. The names of these modules do not constitute a limitation on the modules themselves in some cases.

[0128] As another aspect, the present application also provides a storage medium having stored thereon a computer program, which, when executed by a processor, implements the method described in the above embodiments.

[0129] As another aspect, the present application also provides a storage medium, which can be included in the apparatus described in the above embodiments, or can exist independently without being assembled into the apparatus. The storage medium carries one or more programs, which, when executed by the apparatus, cause the apparatus to: determine a plurality of feature points of current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor; obtain a three-dimensional residual confidence and a two-dimensional likelihood confidence of the pose information according to the plurality of feature points and the pose information; and determine a confidence of the pose information corresponding to the current frame point cloud data after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence.

[0130] The expressions “first”, “second”, “the first” or “the second” used in various embodiments of the present disclosure can modify various components regardless of order and / or importance, but the expressions do not limit the corresponding components. The above expressions are configured only for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element can be called a second element, and similarly, a second element can be called a first element without departing from the scope of the present disclosure.

[0131] When an element (for example, a first element) is referred to as being “(operatively or communicatively) coupled with / to” or “connected to” another element (for example, a second element), it should be understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via a yet another element (for example, a third element). In contrast, it will be understood that when an element (for example, a first element) is referred to as being “directly connected” or “directly coupled” to another element (a second element), no element (for example, a third element) is interposed therebetween.

[0132] The above description is merely that of the preferred embodiments and the best mode of carrying out the application and the technical principles of the application. It should be understood by those skilled in the art that the scope of the application involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the arbitrary combinations of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present application (but not limited to) having similar functions.

Claims

1. A method for determining confidence, comprising: determining a plurality of feature points of a current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor; matching the plurality of feature points with the point cloud data in a world coordinate system by an iterative nearest neighbor algorithm, and projecting the plurality of feature points to a two-dimensional coordinate system in which a current occupancy grid map of the point cloud data is located, to obtain a plurality of two-dimensional feature points; transforming a plurality of point-plane vectors matched to the vehicle body coordinate system indicated by the pose information, and decomposing the plurality of transformed point-plane vectors based on three dimensions of the vehicle body coordinate system to determine statistical information of three groups of vector components; calculating a three-dimensional residual confidence of the pose information according to the statistical information of the three groups of vector components, determining a likelihood probability of the plurality of two-dimensional feature points corresponding to the current occupancy grid map, and determining a two-dimensional likelihood confidence of the pose information according to the likelihood probability; fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence to determine a confidence of the pose information corresponding to the current frame point cloud data.

2. The method of claim 1, wherein, The decomposing the plurality of transformed point-plane vectors based on three dimensions of the vehicle body coordinate system to determine statistical information of three groups of vector components comprises: decomposing the plurality of transformed point-plane vectors based on three dimensions of the vehicle body coordinate system respectively to obtain weights of the three dimensions, and finally calculating a weighted mean and variance of the three dimensions of the residual, wherein the three dimensions include a lateral direction, a longitudinal direction, and a height direction.

3. The method of claim 2, wherein, The decomposing the plurality of transformed point-plane vectors based on three dimensions of the vehicle body coordinate system respectively to obtain weights of the three dimensions, and finally calculating a weighted mean and variance of the three dimensions of the residual comprises: decomposing the plurality of transformed point-plane vectors based on three dimensions of the vehicle body coordinate system respectively to determine residual components of the three dimensions respectively; calculating a weighted mean and variance of the three dimensions of the residual according to the weight information of the plurality of transformed point-plane vectors in the three dimensions of the vehicle body coordinate system and the residual of the three groups of vector components respectively.

4. The method of claim 1, wherein, The method further comprises: determining point cloud data of a historical key frame according to a moving distance and an angle change of the sensor; updating the current occupancy grid map according to a logarithmic geometric state of the point cloud data of the historical key frame.

5. The method of claim 1, wherein, The determining a two-dimensional likelihood confidence of the pose information according to the likelihood probability comprises: binarizing the likelihood probability according to an accuracy range of the likelihood probability to obtain the two-dimensional likelihood confidence of the pose information as a second confidence.

6. An apparatus for determining confidence, comprising: a first determining module configured to determine a plurality of feature points of a current frame point cloud data and pose information corresponding to the current frame point cloud data collected by a sensor; The acquisition module matches the plurality of feature points with the point cloud data in a world coordinate system through an iterative nearest point algorithm, and projects the plurality of feature points to a two-dimensional coordinate system in which a current occupancy grid map of the point cloud data is located, to obtain a plurality of two-dimensional feature points; transforms the plurality of point-plane vectors obtained through matching to a vehicle body coordinate system indicated by the pose information, and decomposes the transformed plurality of point-plane vectors based on three dimensions of the vehicle body coordinate system, to determine statistical information of three groups of vector components; According to the statistical information of the three groups of vector components, a three-dimensional residual confidence of the pose information is calculated, a likelihood probability of the plurality of two-dimensional feature points corresponding to the current occupancy grid map is determined, and a two-dimensional likelihood confidence of the pose information is determined according to the likelihood probability; The second determination module determines a confidence of the pose information corresponding to the current frame of point cloud data after fusing the three-dimensional residual confidence and the two-dimensional likelihood confidence. 7.An electronic device, comprising: a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction makes the processor execute the corresponding operation of the method in any one of claims 1-5. 8.A storage medium having a computer program stored thereon, the program being executed by a processor to implement the method of any one of claims 1-5.

Images