Positioning method, device, vehicle, medium and program product

Abstract

The invention is applied to the field of vehicle positioning, and provides a positioning method and device, a vehicle, a medium and a program product, and the method comprises the steps that under the condition that a vehicle machine detects a request for positioning the vehicle, the vehicle machine can obtain the positioning information of the vehicle by obtaining the driving distance and angle of the vehicle within a period of time in real time; and identifying to obtain the track pose of the vehicle in the time period. In addition, the vehicle machine can collect real-time feature information in the vehicle driving process. And then, the vehicle machine also determines environment feature information matched with the collected feature information from a plurality of pieces of environment feature information of the parking lot map, and takes a pose corresponding to the matched environment feature information as a feature pose. And finally, the vehicle determines the pose of the vehicle in combination with the trajectory pose and the feature pose. Therefore, accurate vehicle pose information can be obtained, and functions of parking based on vehicle positioning and the like can be completed.

Description

Technical Field

[0001] The present application relates to the field of vehicle positioning, and in particular to a positioning method, device, vehicle, medium and program product. Background Art

[0002] The vehicle computer has automatic parking functions such as automated valet parking (AVP). The vehicle computer needs to obtain the position and / or angle (hereinafter collectively referred to as posture) of the vehicle in the parking lot, and control the vehicle to the target position in combination with the existing parking lot map, for example, to control the vehicle to drive to a designated parking space.

[0003] However, due to poor signal in the parking lot, for example, when the vehicle is in an underground parking lot, the vehicle computer cannot obtain accurate positioning information in the parking lot. Since it cannot accurately obtain the real-time positioning information of the vehicle, the vehicle computer may drive the vehicle to the wrong position during the automatic parking process, or even cause a parking accident. Summary of the invention

[0004] In view of this, the present application provides a positioning method, device, vehicle, medium and program product.

[0005] In a first aspect, a positioning method is provided, which is applied to a vehicle-mounted device in a vehicle, and the method includes: detecting a positioning request, the positioning request is used to request obtaining the posture information of the vehicle at a target time; obtaining multiple trajectory postures of the vehicle in a first time period, and environmental feature information collected at each trajectory posture; determining a feature posture corresponding to the environmental feature information of each trajectory posture from an environmental feature map, the environmental feature map including environmental feature information corresponding to different postures; determining multiple groups of posture pairs for calculating a target trajectory based on multiple trajectory postures and multiple feature postures, wherein a first posture pair in the multiple groups of posture pairs includes a first trajectory posture and a first feature posture collected at a first moment; fitting the target trajectory of the vehicle in the first time period based on the multiple groups of posture pairs; obtaining the posture information of the vehicle at the target moment based on the target trajectory, the trajectory posture of the vehicle at the target moment, and the feature posture at the target moment.

[0006] In the above scheme, when the vehicle computer detects a request for positioning the vehicle, for example, when a request for automatic parking is detected, the vehicle computer can obtain the trajectory posture corresponding to each moment in the vehicle's driving trajectory within a period of time. In addition, the vehicle computer can also collect real-time feature information during the vehicle's driving process to obtain the feature information collected at each moment. Then, the vehicle computer will also determine the environmental feature information that matches the collected feature information from multiple environmental feature information of the parking lot map, and use the posture corresponding to the matching environmental feature information as the feature posture. Finally, the vehicle computer will combine the trajectory posture and the feature posture to determine the vehicle's posture, obtain more accurate vehicle posture information, and then complete parking and other functions based on the vehicle's positioning information. For the solution that only uses feature matching for positioning, there are many repeated scenes in the parking lot, and the positioning based on feature matching is inaccurate. This solution combines the trajectory posture and the feature posture obtained based on feature matching, and can obtain more accurate positioning information.

[0007] In combination with the first aspect, in some implementations, the method further includes: obtaining the vehicle's posture increment within each preset time interval through the vehicle's chassis sensor and / or inertial measurement unit within the first time period; and determining multiple trajectory postures of the vehicle within the first time period based on multiple posture increments of the vehicle within the first time period.

[0008] In the above solution, the vehicle computer can obtain the vehicle's driving trajectory through chassis sensors and / or inertial measurement units, and then obtain the trajectory posture. For scenes with poor signals such as indoor parking lots or underground parking lots, the trajectory posture obtained by chassis sensors and / or inertial measurement units will be more accurate.

[0009] In combination with the first aspect, in some implementations, the method also includes: determining, from multiple environmental feature information included in the environmental feature map, multiple candidate feature information whose similarity with the first environmental feature information corresponding to the first trajectory posture in each trajectory posture is greater than a first threshold; obtaining, from the environmental feature map, neighborhood feature information collected at adjacent time intervals for each candidate feature information; based on the similarity between the first environmental feature information and each neighborhood feature information, selecting, from multiple candidate feature information, target environmental feature information whose corresponding neighborhood feature information has a similarity with the first environmental feature information greater than a second threshold; and based on the environmental feature map, using the posture corresponding to the target environmental feature information as the first feature posture corresponding to the first environmental feature information.

[0010] In the above scheme, the vehicle computer matches the collected first environmental feature information with the feature information in the environmental feature map, and after obtaining multiple candidate feature information with high similarity, in order to further screen out the best match, the neighborhood feature information of the candidate feature information is also matched with the first environmental feature information collected by the vehicle computer. The candidate feature information corresponding to the neighborhood feature information with high similarity to the first environmental feature information is used as the feature information corresponding to the first environmental feature information, and then the posture of the candidate feature information collected can be used as the posture corresponding to the first environmental feature information. In the matching process, not only the single feature information but also the neighborhood information of the feature information is considered, which can improve the accuracy of the feature posture. This method can reduce the positioning error caused by environmental changes or sensor errors.

[0011] In combination with the first aspect, in some implementations, the method also includes: determining a map trajectory corresponding to a second feature posture and a third feature posture among multiple feature postures, the map trajectory being a driving trajectory between environmental feature information corresponding to the second feature posture and environmental feature information corresponding to the third feature posture collected when constructing an environmental feature map; determining a first relative posture change based on the second feature posture and the third feature posture, and determining a second relative posture change based on the map trajectory; corresponding to a ratio of the first relative posture change to the second relative posture change being less than a preset ratio, determining that a second posture pair includes a second trajectory posture corresponding to the second feature posture and a third trajectory posture corresponding to the third feature posture, and determining that a third posture pair includes a third feature posture and a third trajectory posture corresponding to the third feature posture.

[0012] In the above scheme, the pose pairs involved in calculating the target trajectory will also be screened. If the pose change of the adjacent feature poses is significantly different from the path when the corresponding environment feature map is constructed, it can be determined that the corresponding feature pose has a large error. If the ratio of the pose change of the adjacent feature poses to the pose change when the corresponding environment feature map is constructed is small, it can be determined that the corresponding feature pose and trajectory pose can be used to calculate the target trajectory. This method helps to eliminate erroneous pose data, helps to generate a more accurate calculation formula for the target trajectory, and improves the accuracy of positioning.

[0013] In combination with the first aspect, in some implementations, the second threshold is determined by the similarity between two environmental feature information belonging to the first category of environmental feature information in the environmental feature map, and the similarity between two environmental feature information belonging to the second category of environmental feature information, wherein the difference in posture of the two environmental feature information belonging to the first category of environmental feature information is greater than the posture difference, and the difference in posture of the two environmental feature information belonging to the second category of environmental feature information is less than or equal to the posture difference.

[0014] In the above scheme, the similarity threshold can be set according to the actual situation of the constructed environmental feature map, and the environmental feature information with a relatively far posture when constructing the environmental feature map is classified into the first category of environmental feature information, which is used to represent the environmental feature information collected in different scenes. The environmental feature information with a relatively far posture when constructing the environmental feature map is classified into the second category of environmental feature information, which is used to represent the environmental feature information collected in similar scenes. The similarity threshold is determined by the similarity between the environmental feature information in the first category of environmental feature information and the environmental feature information in the second category of environmental feature information, so that the determined similarity threshold conforms to the actual situation of the environmental feature map.

[0015] In combination with the first aspect, in some implementations, the method further includes: updating the environmental feature map according to the posture change trajectory within the first time period and the environmental feature information corresponding to each posture in the posture change trajectory.

[0016] In combination with the first aspect, in some implementations, the method also includes: adding second environmental feature information collected at the second moment in the first time period and the trajectory posture of collecting the second environmental feature information to the environmental feature map; and matching the third environmental feature information collected at the third moment in the first time period with the fourth environmental feature information in the environmental feature map, and replacing the fourth environmental feature information with the third environmental feature information.

[0017] In a second aspect, the present application provides a vehicle-mounted device, comprising a processor and a memory, the memory being used to store instructions executed by one or more processors of the vehicle-mounted device, and the processor, which is one of the processors of the vehicle-mounted device, being used to execute instructions. When the processor executes the instruction, it executes the method described in the first aspect and various implementation methods of the first aspect.

[0018] In a third aspect, the present application provides a vehicle, comprising the vehicle-mounted device as described in the second aspect.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a positioning device, the method described in the first aspect and various implementations of the first aspect is executed.

[0020] In a fifth aspect, the present application provides a computer program product, which includes computer instructions. When the computer program product is executed by a computing device, the computing device executes the method described in the first aspect and various implementations of the first aspect.

[0021] The beneficial effects of the second to fifth aspects mentioned above can refer to the beneficial effects of the first aspect mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for describing the embodiments are briefly introduced below.

[0023] Figure 1 is a schematic diagram of a constructed parking lot map provided in an embodiment of the present application; Figure 2 is a schematic diagram of identifying image features provided by an embodiment of the present application; Figure 3 is a schematic diagram of positioning by feature comparison provided in an embodiment of the present application; Figure 4 is a structural diagram of a positioning system provided in an embodiment of the present application; Figure 5 is a schematic diagram of determining neighborhood feature information provided by an embodiment of the present application; Figure 6 is a schematic diagram of determining a pose pair provided in an embodiment of the present application; Figure 7 It is a schematic diagram of determining positioning by considering odometer constraints and VPR constraints provided in an embodiment of the present application; Figure 8 is a schematic diagram of updating a feature map provided in an embodiment of the present application; Fig. 9 is a structural schematic diagram of a map construction system provided in an embodiment of the present application; Fig.10 is a schematic diagram of determining a similarity threshold provided in an embodiment of the present application; Fig.11 It is a flowchart of a positioning method provided in an embodiment of the present application; Fig.12 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] Illustrative embodiments of the present application include, but are not limited to, positioning methods, apparatus, vehicles, media, and program products.

[0025] As mentioned above, the vehicle equipment included in the vehicle may have an automatic parking function, which can control the vehicle to drive to a designated parking space based on an existing parking lot map and the current vehicle position.

[0026] In some examples, the vehicle computer can be positioned by a global navigation satellite system (GNSS) system and / or in combination with real-time kinematic (RTK) technology.

[0027] However, due to weak GNSS signals in underground parking lots or indoor parking lots, the vehicle computer cannot obtain the accurate positioning of the vehicle, resulting in errors in vehicle positioning. If the vehicle computer locates the vehicle in real time based on GNSS and / or RTK and performs automatic parking, there will be inaccurate positioning due to poor GNSS signals, which will lead to parking failure.

[0028] In other examples, the vehicle computer can also determine the location of the vehicle by collecting feature information of the current environment and matching it with the pre-built parking lot feature map. The feature information is obtained by processing the raw sensor data through the visual place recognition (VPR) deep neural network model. The raw sensor data can be RGB images collected by the vehicle camera or point cloud data collected by the lidar.

[0029] Among them, building a parking lot map can be completed by the vehicle computer of the current vehicle or other equipment. Taking the vehicle computer as an example, the vehicle computer can use the global navigation satellite system (GNSS) system and / or combine the real-time differential positioning (RTK) technology for positioning. In addition, in different postures, the vehicle computer will also collect environmental features. For example, the vehicle computer can collect environmental features through sensors such as cameras and / or lidars, and identify environmental features such as road information (such as lane lines, lane driving direction, etc.), parking space information (such as parking space numbers), and obstacle information (such as pillars, walls, guardrails, etc.). The vehicle computer can combine the collected environmental features with the posture information corresponding to the environmental features to construct the following. Figure 1 Parking map shown.

[0030] In addition, the vehicle computer can also extract feature points from the environmental features collected when building the parking map, and identify edges, corners, etc. in the environment. For example, the vehicle computer can process the image collected by the vehicle camera into a grayscale image, and determine the area where the grayscale value of the pixel point in the grayscale image changes significantly as an edge, and determine the point where two edges intersect as a corner point. Figure 2 The left side shows an environmental feature map collected by the vehicle computer. By extracting the features in the environmental feature map, the corner points included in the environmental feature map can be identified. Figure 2 On the right Figure 2 A magnified schematic diagram of the pixels in the area shown in the white box on the left. Figure 2 In the schematic diagram on the right, the area where the pixel color changes is found to be a corner point. The vehicle computer can identify information such as edges and corner points in the environmental feature map.

[0031] Furthermore, when the vehicle computer performs automatic parking, the vehicle's position can be determined by collecting environmental feature points near the vehicle's current position (hereinafter referred to as positioning feature points). Figure 3 As shown, the vehicle computer can perform feature comparison between the positioning feature points and the environmental feature points in the parking lot map (hereinafter referred to as mapping feature points), determine the mapping feature points that match the positioning feature points from the multiple mapping feature points contained in the parking lot map (for example, the mapping feature points with the highest similarity to the positioning feature points), and determine the posture information corresponding to the matched mapping feature points as the current posture of the vehicle.

[0032] However, the above-mentioned feature comparison between the feature points at the time of positioning and all the feature points at the time of mapping is required, which requires a large amount of calculation and consumes more computing resources. In addition, in the actual environment of the parking lot, due to the high repetition of features such as parking spaces and lane lines, feature comparison errors may also occur during the feature comparison process. Moreover, for underground parking lots or indoor parking lots, the light is weak, and the environmental features collected during mapping and / or positioning may not be accurate enough, and it is easy to collect moving objects in the environment (such as moving vehicles or moving people, etc.). And it is also possible to collect more environmental features such as textureless walls and corridors that lack sufficient texture information, resulting in fewer available features extracted. Furthermore, due to the inaccurate environmental features collected in the parking lot or the small number of available features, the result of the feature comparison is not accurate enough. Since the real-time positioning information of the vehicle cannot be accurately obtained, the vehicle computer cannot accurately perform automatic parking.

[0033] Furthermore, in order to solve the problem that the above-mentioned vehicle is inaccurately positioned in the parking lot, resulting in the inability of the vehicle computer to park accurately, the present application provides a positioning method. When the vehicle computer detects a request to position the vehicle, for example, when a request for automatic parking is detected, the vehicle computer can obtain the vehicle's driving distance and angle over a period of time in real time, and then identify the vehicle's driving trajectory during the time period, and the vehicle's posture corresponding to each moment in the driving trajectory during the time period can also be called a trajectory posture.

[0034] In addition, the vehicle computer can also collect real-time feature information during the vehicle's driving process and obtain the feature information collected at each moment. Then, the vehicle computer will determine the environmental feature information that matches the collected feature information from the multiple environmental feature information of the parking lot map, and use the posture corresponding to the matching environmental feature information as the characteristic posture. Finally, the vehicle computer will combine the trajectory posture and the characteristic posture to determine the vehicle's posture.

[0035] Among them, the vehicle computer can determine the vehicle posture by combining the trajectory posture and the characteristic posture in the following way: the vehicle computer can use the optimization algorithm to combine the mileage data (i.e., multiple trajectory postures) and the visual matching data (i.e., multiple characteristic postures), and use the trajectory posture and characteristic posture collected at the same time as a set of posture pairs, and then use multiple sets of posture pairs to correct and optimize the vehicle posture estimation algorithm, that is, fit the target trajectory function used to calculate the actual posture of the vehicle. The target trajectory function takes into account the uncertainty of the trajectory position and the characteristic position. The target trajectory function can be specifically obtained by calculating the covariance matrix of the errors of the trajectory position and the characteristic position, and then iteratively solving to obtain the minimized posture error, and then obtaining the final target trajectory function. Then, by bringing the trajectory position and characteristic position at the target moment to be solved into the target trajectory function, the actual posture of the vehicle at the target moment can be obtained.

[0036] In some embodiments, the vehicle computer can obtain information such as the vehicle's driving posture, speed, and displacement through vehicle sensors, such as chassis wheel speed sensors and / or inertial measurement units (IMUs), and then the vehicle computer can determine the vehicle's driving trajectory.

[0037] The method provided in the embodiment of the present application can be applied to vehicle equipment of vehicles, etc. Among them, the above-mentioned vehicles may include but are not limited to road vehicles (such as vehicles), water vehicles (such as ships), and air vehicles (such as airplanes). The above-mentioned vehicles can be vehicles in a broad sense, for example, they can be vehicles (such as commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), agricultural equipment (such as mowers, harvesters, etc.), amusement equipment, toy vehicles, etc.

[0038] It should be understood that the above-mentioned method provided in the embodiment of the present application is applied to the parking lot navigation scenario as an example. In actual application, the method provided in the embodiment of the present application can also be applied to other indoor navigation scenarios, such as shopping malls, factories, etc. Corresponding to different application scenarios, the environmental feature map can also be an environmental feature map of a shopping mall or an environmental feature map of a factory, etc.

[0039] For the sake of convenience, the following takes a parking lot navigation scenario as an example to illustrate the positioning method provided in the embodiment of the present application.

[0040] Figure 4A structural schematic diagram of a positioning system is shown, which includes a wheel speed odometer module, a visual place recognition (VPR) deep neural network module, a VPR feature vector module, a candidate feature retrieval module, a neighborhood feature similarity screening module, a multi-frame pose consistency verification module, a combined positioning pose optimization module, and a feature update module.

[0041] It should be understood that the positioning system may also include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. Figure 4 The components shown may be implemented in hardware, software or a combination of software and hardware.

[0042] The wheel speed odometer module is used to calculate the incremental displacement of the odometer, and then obtain the driving trajectory of the vehicle. Specifically, when the wheel speed odometer module detects that the vehicle needs to be positioned, it can obtain multiple trajectory positions of the vehicle within a certain time period (for example, the first time period). Specifically, the wheel speed odometer module can be used to obtain information such as the posture, speed and displacement of the vehicle based on the chassis wheel speed sensor and / or IMU. Then the wheel speed odometer module can calculate the position and posture of the vehicle based on the posture, speed and displacement of the vehicle according to the incremental calculations such as speed integral and angle integral.

[0043] The calculation formula of the velocity integral can refer to the following formula (1), and the calculation formula of the angle integral can refer to the following formula (2).

[0044] (1); (2); in, It represents the incremental displacement of the vehicle along the driving direction within a certain time interval. Indicates the incremental displacement of the left wheel of the vehicle within a certain time interval. Indicates the incremental displacement of the right wheel of the vehicle within a certain time interval.

[0045] Represents the angular velocity measured by the gyroscope, i.e. the rate at which the vehicle is rotating about an axis perpendicular to the ground (usually the longitudinal axis of the vehicle). It represents the time interval, that is, the length of time considered for calculating displacement and posture changes. The time interval can be fixed or dynamically adjusted according to actual conditions. Indicates the time interval The incremental change of the vehicle's posture, that is, the rotation angle of the vehicle.

[0046] In addition, the wheel speed odometer module can also store the vehicle's position and corresponding timestamp every preset time T1 or every preset distance L1 to obtain the vehicle's driving trajectory.

[0047] The VPR feature vector module, the candidate feature retrieval module, the neighborhood feature similarity screening module, and the multi-frame pose consistency verification module are used to obtain multiple feature poses of the vehicle in the first time period when it is detected that the vehicle needs to be positioned.

[0048] The VPR deep neural network module is used to send the sensor data such as the RGB image I collected by the vehicle camera in the first time period or the point cloud data collected by the vehicle laser radar into the trained VPR deep neural network Θ. For the sake of convenience, the following description is taken as an example of inputting the RGB image I into the VPR deep neural network Θ.

[0049] The VPR deep neural network Θ can extract key semantic feature information from the collected image and aggregate the extracted feature information into a fixed-dimensional VPR feature vector F. For example, F can be an n-dimensional vector, and each component is a real number, that is, it belongs to the real number domain The VPR deep neural network module can send the images at each moment into the VPR deep neural network Θ in time sequence, and a set of VPR feature sequences can be obtained. ,in, , to Represents the feature vector corresponding to the image at different times. For details, please refer to the following formulas (3) and (4).

[0050] (3); (4); In some embodiments, the VPR deep neural network Θ may include a feature extraction backbone network and a feature aggregation network using a cross-attention mechanism. The feature extraction backbone network is used to extract key semantic feature information such as edges, textures, shapes, and colors from the input image. The feature aggregation network is used to convert the extracted semantic feature information into special features for visual scene recognition to obtain a VPR feature vector F.

[0051] The candidate feature retrieval module is used to collect the VPR feature vectors based on a certain posture of the vehicle in the first time period. , obtain the K most similar candidate feature information in the parking lot map Among them, the candidate feature retrieval module can be a calculation The similarity with each feature information in the parking lot map is then sorted to obtain the K candidate feature information with the highest similarity. Alternatively, feature information with a similarity greater than a first threshold may be used as candidate feature information.

[0052] The neighborhood feature similarity screening module is used to filter K candidate feature information Neighborhood feature information , for K candidate features Screen and obtain K candidate feature information The optimal matching feature information under this pose .

[0053] For example, based on the acquired feature image, the VPR deep neural network module outputs the corresponding feature vector Afterwards, the candidate feature retrieval module can determine the feature vector Multiple candidate feature information with the highest similarity Then for each candidate feature information , and the candidate feature information will also be determined from the environmental feature map At least one characteristic information (hereinafter referred to as neighborhood characteristic information) collected from adjacent frames ). And calculate the eigenvector Neighborhood feature information If the feature vector Neighborhood feature information The similarity is low, then the candidate feature information is determined With this feature vector Furthermore, the neighborhood feature information can be With the eigenvector Candidate feature information with a high matching degree As the best matching feature information of the feature vector .

[0054] For example, Figure 5 As shown, for The candidate feature information of the i-th frame included in , the candidate feature information can be The preset time T2 before and after is used as the neighborhood interval, and then, in the VPR trajectory map, the candidate features The feature information collected before and after the preset time T2 is the candidate feature Neighborhood feature information, such as the candidate features of the i-th frame The neighborhood feature information may include the j-th frame feature information and the feature information of the first frame .

[0055] For example, feature information Neighborhood feature information The similarity can be the Euler distance, for feature information With multiple neighborhood feature information The candidate feature information whose Euler distance is less than the preset distance , the candidate feature information can be determined The best matching feature information .

[0056] Then, by calculating the characteristic information of each neighborhood and The similarity between the neighborhood feature information and The candidate features whose similarity with is lower than the similarity threshold H (as an example of the second threshold) are determined as false recall. The similarity is sorted and the neighborhood feature information is The candidate feature with the highest similarity is used as the optimal matching feature information The calculation method of the similarity threshold H can also refer to the following Fig. 9 Description of the similarity confidence threshold calculation module in .

[0057] The multi-frame pose consistency verification module is used to integrate multiple feature information obtained by the candidate feature retrieval module and the neighborhood feature similarity screening module The corresponding pose (hereinafter referred to as feature pose) is matched with the parking lot map by sliding window, and a feature pose that can be used to calculate the target trajectory is determined from multiple feature poses. The determined feature pose and the corresponding trajectory pose are then used as a set of pose pairs for the subsequent combined positioning pose optimization module to generate the target trajectory function. For example, Figure 6 The dashed box in represents the sliding window.

[0058] For example, a parking lot map is constructed by the feature information collected by the vehicle computer during the driving process. The parking lot map can also be called a VPR trajectory map, and then there is a driving trajectory when constructing the parking lot map (hereinafter referred to as a map trajectory). The map trajectory includes multiple map poses, and each map pose has the feature information collected at the map pose. Figure 6 As shown, if the environmental feature information in the parking lot map matches the feature information of the feature posture, the vehicle computer can determine that the map posture corresponding to the collection of the environmental feature information matches the feature posture. The vehicle computer will also determine the map posture corresponding to each of the multiple feature postures screened by the neighborhood feature similarity screening module, where the trajectory formed by the multiple feature postures can also be called a positioning trajectory.

[0059] Then, the vehicle computer will calculate the relative pose transformation between adjacent feature poses , and calculate the relative pose transformation between the map poses corresponding to the set of adjacent feature poses And, by calculating and Ratio , if the If the value is less than the ratio threshold, it is determined that the set of characteristic poses and the trajectory poses corresponding to the characteristic poses can be used to calculate the target trajectory, where the trajectory pose corresponding to the characteristic pose refers to the pose obtained by the odometer module at the same time as the characteristic pose. Figure 6 The relationship between adjacent trajectory poses and feature poses in the dashed box and are all smaller than the ratio threshold and can be used to calculate the target trajectory.

[0060] Optionally, the vehicle computer can also set the window size, for example, the window size can be 4 characteristic postures. If the relative posture transformation between adjacent characteristic postures in the continuous 4 characteristic postures is detected , and the relative pose transformation between the corresponding map pose satisfy If the condition is less than the threshold, the four feature poses in the window and the corresponding trajectory poses can be used as pose pairs.

[0061] The combined positioning pose optimization module is used to fit the target trajectory function used to calculate the vehicle pose according to the multiple pose pairs determined by the multi-frame pose consistency verification module. Among them, the target trajectory function takes into account the errors of the trajectory pose and the characteristic pose, such as Figure 7 As shown, p1, p2 and p3 can be used to represent vehicle pose, odo is used to represent trajectory pose constraints (also called keyframe odometer pose constraints), and vpr is used to represent feature pose constraints (also called VPR observation constraints).

[0062] Optionally, the target trajectory function can be obtained by the Gauss-Newton method or the Levenberg-Marquardt algorithm based on multiple groups of pose pairs. For example, the target trajectory function is composed of the covariance matrix of the trajectory pose noise in the multiple groups of poses obtained from the multiple groups of pose pairs, the covariance matrix of the characteristic pose noise, the pose residual between the trajectory pose frames, and the pose residual between the characteristic pose frames. Finally, the target trajectory function obtained by combining the positioning pose optimization module is brought into the trajectory pose and the characteristic pose collected at the target time, and then the actual pose of the vehicle at the target time can be solved.

[0063] The feature update module is used to update the historical parking lot map according to the current positioning results.

[0064] Among them, Figure 8As shown, updating the parking lot map includes, if the actual location of the vehicle can be obtained and multiple VPR features are collected at the location of the location, the feature information of the location in the historical parking lot map (also called the original trajectory of the map) can be replaced with the latest collected VPR features.

[0065] If a map of a historical parking lot that was not previously covered is collected based on the current vehicle's positioning and / or driving trajectory (also referred to as a positioning trajectory), the positioning trajectory and its characteristic information may be added to the parking lot map.

[0066] In some embodiments, since the current navigation application map does not cover most parking lots, especially for newly built residential areas or shopping malls, the car machine can also independently build a parking lot map. The specific process of the car machine building a parking lot map can be referred to as follows Fig. 9 shown.

[0067] Fig. 9 A vehicle mapping system is shown, which may include a wheel speed odometer module, a VPR deep neural network module, a trajectory-VPR feature timestamp association module, a positive and negative sample association module, and a similarity confidence threshold calculation module.

[0068] Among them, the wheel speed odometer module is used to obtain the vehicle wheel speed odometer, and then calculate the incremental displacement of the vehicle odometer. The VPR deep neural network module is used to generate a VPR feature vector for the image captured by the vehicle camera based on the VPR deep neural network, and for the multi-frame image input by the vehicle camera, the VPR deep neural network module can also generate multiple VPR feature vectors accordingly. For the description of the wheel speed odometer module and the VPR deep neural network module, please refer to the aforementioned Figure 4 The relevant description in will not be repeated here.

[0069] The trajectory-VPR feature timestamp association module is used to find the corresponding driving trajectory point for each VPR feature vector based on the timestamp and trajectory posture, and obtain the corresponding multiple feature postures.

[0070] In addition, since the VPR features with close spatial positions have low differentiation, the associated VPR feature vector can be downsampled according to the cumulative trajectory distance. In curved scenes with large curvature changes, the sampling density can be increased to ensure the scene coverage of VPR features under large viewing angle changes.

[0071] The positive and negative sample association module is used to classify the collected multiple VPR feature vectors into positive samples and negative samples for subsequent calculation of the similarity threshold H. Among them, for two trajectory features with small odometer position and posture during map construction, their corresponding feature postures can be regarded as a group of positive samples; for two trajectory features with large position and posture differences during map construction / from different parking lots, their corresponding feature postures can be regarded as a group of negative samples. Optionally, for the case where there is a constructed historical feature map (also called a historical VPR map) for the current scene, the positive and negative sample association module can also obtain the data of the historical VPR map.

[0072] In addition, the positive and negative sample association module will calculate the similarity (such as cosine similarity) of any two VPR feature vectors to obtain the similarity results of each group of samples (including positive samples and negative samples). For example, the similarity result can be expressed as a numerical value in the interval [0,1], where the closer the similarity result is to 1, the higher the similarity between the two feature vectors. Since the neighborhood feature similarity screening module needs to use the similarity threshold H to determine whether the neighborhood feature information is similar to the currently collected features, but due to the influence of various factors such as lighting and weather conditions, dynamic and static target distribution, the feature spatial distribution of VPR features in different parking lots is very different, and a universal confidence threshold cannot be used for judgment. Furthermore, the similarity confidence threshold calculation module can generate the similarity threshold H of the current scene based on the currently collected feature information.

[0073] The similarity confidence threshold calculation module is used to generate a similarity threshold H according to the similarity of each group of positive samples and the similarity of each group of negative samples. The similarity confidence threshold calculation module can first be used to calculate the mean of the similarities of all positive samples. and standard deviation , and the mean similarity of all negative samples , standard deviation .

[0074] Then, the preliminary thresholds for positive and negative samples are generated, where the negative sample-guided threshold is calculated by taking the mean of the negative sample similarities. and standard deviation and negative sample range control coefficient The negative sample-oriented threshold is based on the "confidence upper bound" of negative samples, which can ensure that most negative samples are lower than this value. The positive sample-oriented threshold is calculated by the mean of the similarity of positive samples. and standard deviation and negative sample range control coefficient The positive sample-guided threshold is based on the "confidence lower bound" of the positive sample, which ensures that most positive samples are higher than this value.

[0075] Furthermore, the similarity threshold H can be obtained by the negative sample-oriented threshold, the negative sample coefficient, the positive sample-oriented threshold, and the positive sample coefficient. Fig.10 As shown, the similarity threshold H is equal to the product of the negative sample-oriented threshold and the negative sample coefficient, plus the product of the positive sample-oriented threshold and the positive sample coefficient, and the sum of the negative sample coefficient and the positive sample coefficient is 1. The negative sample coefficient and the positive sample coefficient can also be adjusted by the number of positive and negative samples. For example, the more positive samples there are, the larger the positive sample coefficient is.

[0076] Optionally, the above data can also be exported as a serialized binary file and stored in a local storage medium on the vehicle side.

[0077] The positioning method provided by this application is introduced in detail below. Fig.11 According to an embodiment of the present application, a schematic diagram of an implementation flow of a positioning method is provided. As mentioned above, the method is applied to the above-mentioned vehicle equipment. Fig.11 As shown, the method specifically includes: S1101: A positioning request is detected.

[0078] The vehicle computer can detect a request initiated by a user to obtain the vehicle's position information at a target time, for example, when the user turns on the autonomous navigation function.

[0079] S1102: Acquire multiple trajectory positions of the vehicle in a first time period, and environmental feature information collected by each trajectory position.

[0080] The vehicle computer will obtain information such as the vehicle's driving posture, speed, and displacement based on the chassis wheel speed sensor and / or IMU, and then obtain multiple trajectory postures of the vehicle in the first time period. The vehicle computer will also collect feature information in the first time period based on the on-board camera, etc. The first time period can be before or after the target time, and can also include a period of time before the target time.

[0081] The vehicle computer can also refer to the relevant descriptions of the aforementioned wheel speed odometer module and VPR deep neural network module for obtaining multiple trajectory postures and environmental feature information, which will not be repeated here.

[0082] S1103: Determine a characteristic posture corresponding to the environmental characteristic information of each trajectory posture from the environmental characteristic map.

[0083] The vehicle computer can determine the environmental feature information that matches the collected feature information from the multiple environmental feature information of the parking lot map, and use the posture corresponding to the matched environmental feature information as the feature posture. For details, please refer to the relevant descriptions of the aforementioned VPR feature vector module, candidate feature retrieval module, neighborhood feature similarity screening module and multi-frame posture consistency verification module, which will not be repeated here.

[0084] S1104: According to the multiple trajectory postures and the multiple feature postures, a target trajectory of the vehicle in the first time period is obtained by fitting.

[0085] The vehicle computer can correct and optimize the vehicle's posture estimation algorithm by combining multiple trajectory postures and multiple feature postures through an optimization algorithm, that is, fitting a target trajectory function for calculating the actual posture of the vehicle. For details, please refer to the description of the aforementioned combined positioning posture optimization module, which will not be repeated here.

[0086] S1105: Obtaining the posture information of the vehicle at the target time according to the target trajectory and the trajectory posture and characteristic posture of the vehicle at the target time.

[0087] The vehicle computer uses the target trajectory function obtained, inputs the trajectory posture and feature posture collected at the target time, and then solves the actual posture of the vehicle at the target time.

[0088] In summary, through the above method, the vehicle computer can integrate the trajectory posture and feature posture to obtain more accurate vehicle posture information, and then complete functions such as parking based on vehicle positioning.

[0089] Fig.12 According to an embodiment of the present application, a schematic diagram of the hardware structure of a vehicle machine device is shown.

[0090] like Fig.12 As shown, the vehicle machine 1200 may include a processor 110, a wireless communication module 120, a mobile communication module 130, a power module 140, an audio module 150, an interface module 160, a main camera 170 and a wide-angle camera 171, a memory 180, a sensor module 190, a button 102, a motor 103 and an indicator 104, etc. The sensor module 190 may include a seat sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a touch sensor, an ambient light sensor, etc.

[0091] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the vehicle computer 1200. In other embodiments of the present application, the vehicle computer 1200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0092] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0093] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0094] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0095] In the embodiment of the present application, the processor 110 of the vehicle computer 1200 can implement the above-mentioned Fig.11 For the relevant steps executed by the CRRC locomotive, specific reference may be made to the relevant descriptions in the above embodiments, which will not be elaborated here.

[0096] The interface module 160 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a universal serial bus (USB) interface, etc.

[0097] The power module 140 is used to connect the battery, the charging management module and the processor 110, etc. The power module 140 receives input from the battery and / or the charging management module to power the processor 110, the memory 180, the main camera 170, the wide-angle camera 171 and the wireless communication module 120, etc.

[0098] The wireless communication module 120 can provide wireless communication solutions for application in the vehicle computer 1200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0099] The mobile communication module 130 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the vehicle machine 1200. The mobile communication module 130 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0100] The main camera 170 and the wide-angle camera 171 are used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV and other formats.

[0101] The memory 180 can be used to store computer executable program codes, which include instructions. The internal memory 180 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the vehicle machine 1200 (such as audio data, a phone book, etc.), etc. In addition, the memory 180 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the vehicle machine 1200 by running instructions stored in the memory 180 and / or instructions stored in a memory provided in the processor.

[0102] The vehicle computer 1200 can realize audio functions such as music playing and recording through the audio module 150, speakers, receivers, microphones, and application processors.

[0103] The audio module 150 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 can be arranged in the processor 110, or some functional modules of the audio module 150 can be arranged in the processor 110.

[0104] The speaker, also called a "horn", is used to convert an audio electrical signal into a sound signal. The vehicle computer 1200 can listen to music or listen to hands-free calls through the speaker 170A.

[0105] The receiver, also called a "handset", is used to convert audio electrical signals into sound signals. When the vehicle computer 1200 receives a call or voice message, the voice can be received through the receiver.

[0106] A microphone, also called a "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can put their mouth close to the microphone and speak to input the sound signal into the microphone. The car computer 1200 can be provided with at least one microphone 170C. In other embodiments, the car computer 1200 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the car computer 1200 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0107] The earphone interface 170D is used to connect a wired earphone and can be a USB interface, or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0108] The button 102 includes a power button, a volume button, etc. The button 102 can be a mechanical button or a touch button. The vehicle computer 1200 can receive the button input and generate a key signal input related to the user setting and function control of the vehicle computer 1200.

[0109] Motor 103 can generate vibration prompts. Motor 103 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. Motor 103 can also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen. Different application scenarios (such as: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0110] The indicator 104 may be an indicator light, which may be used to indicate the charging status, power change, messages, missed calls, notifications, etc.

[0111] An embodiment of the present application also provides a vehicle, including the above-mentioned vehicle machine 1200.

[0112] The embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a positioning device, the methods provided in the above embodiments are executed.

[0113] An embodiment of the present application further provides a computer program product, including computer program code. When the computer program code is executed on a computer, the computer executes the positioning method provided in the above embodiments.

[0114] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program module or module code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0115] A computer program module or module code may be applied to input instructions to perform the functions described herein and generate output information. The output information may be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0116] Module code can be implemented with high-level modular language or object-oriented programming language to communicate with the processing system. When necessary, module code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0117] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to a floppy disk, an optical disk, an optical disk, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., a carrier wave, an infrared signal, a digital signal, etc.) using the Internet in an electrical, optical, acoustic, or other form of propagation signal. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

Claims

1. A positioning method, characterized in that: The vehicle machine device is applied to a vehicle, and the method comprises: A positioning request is detected, where the positioning request is used to request obtaining position information of the vehicle at a target time; Acquire multiple trajectory positions of the vehicle in a first time period, and environmental feature information collected at each trajectory position; Determine a characteristic posture corresponding to the environmental characteristic information of each trajectory posture from an environmental characteristic map, wherein the environmental characteristic map includes environmental characteristic information corresponding to different postures; Determine a plurality of pose pairs for calculating the target trajectory according to the plurality of trajectory poses and the plurality of characteristic poses, wherein a first pose pair in the plurality of pose pairs includes a first trajectory pose and a first characteristic pose collected at a first moment; Fitting a target trajectory of the vehicle in the first time period according to the multiple groups of position and posture pairs; The position and posture information of the vehicle at the target moment is obtained according to the target trajectory, the trajectory and posture of the vehicle at the target moment, and the characteristic posture of the vehicle at the target moment.

2. The positioning method according to claim 1, characterized in that: The obtaining of a plurality of trajectory positions of the vehicle in a first time period includes: Acquire, by means of a chassis sensor and / or an inertial measurement unit of the vehicle, a position increment of the vehicle within each preset time interval during the first time period; A plurality of trajectory postures of the vehicle in the first time period are determined according to a plurality of posture increments of the vehicle in the first time period.

3. The positioning method according to claim 1, characterized in that: The various trajectory postures include a first trajectory posture, and the first trajectory posture corresponds to first environmental feature information. The step of determining a characteristic posture corresponding to the environmental characteristic information of each trajectory posture from the environmental characteristic map includes: Determine, from the plurality of pieces of environmental feature information included in the environmental feature map, a plurality of candidate feature information having a similarity with the first environmental feature information greater than a first threshold; Acquire neighborhood feature information collected at adjacent time intervals with each candidate feature information from the environmental feature map; According to the similarity between the first environmental feature information and each piece of neighborhood feature information, select, from the plurality of candidate feature information, target environmental feature information whose corresponding neighborhood feature information has a similarity with the first environmental feature information greater than a second threshold; According to the environmental feature map, the posture corresponding to the target environmental feature information is used as the first characteristic posture corresponding to the first environmental feature information.

4. The positioning method according to claim 3, characterized in that: The step of determining a plurality of groups of pose pairs for calculating the target trajectory according to the plurality of trajectory poses and the plurality of feature poses comprises: Determine a map trajectory corresponding to a second feature posture and a third feature posture among the multiple feature postures, wherein the map trajectory is a driving trajectory between collecting environmental feature information corresponding to the second feature posture and collecting environmental feature information corresponding to the third feature posture when constructing the environmental feature map; Determining a first relative posture change according to the second characteristic posture and the third characteristic posture, and determining a second relative posture change according to the map trajectory; Corresponding to the ratio of the first relative posture change to the second relative posture change being less than a preset ratio, a second posture pair is determined to include the second feature posture and the second trajectory posture corresponding to the second feature posture, and a third posture pair is determined to include the third feature posture and the third trajectory posture corresponding to the third feature posture.

5. The positioning method according to claim 3, characterized in that: The second threshold is determined by the similarity between two pieces of environmental feature information belonging to the first category of environmental feature information in the environmental feature map, and the similarity between two pieces of environmental feature information belonging to the second category of environmental feature information, wherein the difference in posture of the two pieces of environmental feature information collected belonging to the first category of environmental feature information is greater than the posture difference, and the difference in posture of the two pieces of environmental feature information collected belonging to the second category of environmental feature information is less than or equal to the posture difference.

6. The positioning method according to claim 1, characterized in that: The method further comprises: The environmental feature map is updated according to the target trajectory in the first time period and the environmental feature information corresponding to each posture in the target trajectory.

7. The positioning method according to claim 6, characterized in that: The updating of the environmental feature map includes: Adding the second environmental feature information collected at the second moment in the first time period and the trajectory position of collecting the second environmental feature information to the environmental feature map; and The third environmental characteristic information collected at the third moment in the first time period is matched with the fourth environmental characteristic information in the environmental characteristic map, and the fourth environmental characteristic information is replaced by the third environmental characteristic information.

8. A vehicle machine device, characterized in that: include: A memory, used to store instructions executed by one or more processors of the vehicle-mounted device, and a processor, which is one of the processors of the vehicle-mounted device, used to execute the positioning method described in any one of claims 1 to 7.

9. A vehicle, characterized in that: Including the vehicle equipment as described in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a positioning device, enable the positioning device to execute the positioning method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is run on a computer, the computer is enabled to execute the positioning method according to any one of claims 1 to 7.

Images