Vehicle pose graph determination method, apparatus, device, and medium

Abstract

This invention discloses a method, apparatus, device, and medium for determining a vehicle pose map. The method includes: determining a target key image of a target vehicle during a target running period, and a target fusion vector corresponding to the target key image; determining the target pose of the target vehicle based on the target key image, and determining a reference running trajectory based on the target fusion vector; determining a reference running trajectory corresponding to the target pose, and determining an initial vehicle pose map of the target vehicle based on the target pose and the reference running trajectory; determining pose map adjustment data based on the reference running trajectory, the reference running trajectory, and a preset error adjustment function, and updating the initial vehicle pose map based on the pose map adjustment data to obtain the target vehicle pose map. This improves the accuracy of the determined target vehicle pose map.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer vision technology, and in particular to a method, apparatus, device and medium for determining vehicle pose map. Background Technology

[0002] In the field of autonomous driving, vehicle pose maps, by integrating multi-sensor data and optimizing the spatial correlation between vehicle trajectory and environmental features, can significantly improve the system's positioning accuracy, environmental perception capability, and decision robustness. Therefore, improving the accuracy of a given vehicle pose map is crucial. Summary of the Invention

[0003] This invention provides a method, apparatus, device, and medium for determining a vehicle pose map, so as to improve the accuracy of the determined vehicle pose map.

[0004] According to one aspect of the present invention, a method for determining a vehicle pose map is provided, comprising: Determine the target key image of the target vehicle during the target operating period, and the target fusion vector corresponding to the target key image; Based on the target key image, the target pose of the target vehicle is determined, and based on the target fusion vector, the reference running trajectory is determined; Determine the reference trajectory corresponding to the target pose, and determine the initial vehicle pose diagram of the target vehicle based on the target pose and the reference trajectory; Based on the baseline running trajectory, the reference running trajectory, and the preset error adjustment function, pose map adjustment data is determined, and the initial vehicle pose map is updated according to the pose map adjustment data to obtain the target vehicle pose map.

[0005] According to another aspect of the present invention, a vehicle pose map determination apparatus is provided, comprising: The target fusion vector determination module is used to determine the target key image of the target vehicle during the target running period, and the target fusion vector corresponding to the target key image; The baseline trajectory determination module is used to determine the target pose of the target vehicle based on the target key image, and to determine the baseline trajectory based on the target fusion vector. The initial pose determination module is used to determine the reference running trajectory corresponding to the target pose, and to determine the initial vehicle pose of the target vehicle based on the target pose and the reference running trajectory. The target pose map determination module is used to determine pose map adjustment data based on the baseline running trajectory, the reference running trajectory and the preset error adjustment function, and update the initial vehicle pose map based on the pose map adjustment data to obtain the target vehicle pose map.

[0006] According to another aspect of the present invention, an electronic device is provided, comprising: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by one or more processors, the one or more processors are able to execute any of the vehicle pose map determination methods provided in the embodiments of the present invention.

[0007] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute any of the vehicle pose map determination methods provided in the embodiments of the present invention.

[0008] This invention provides a vehicle pose map determination scheme. It involves determining a target key image of the target vehicle within a target runtime period, and the corresponding target fusion vector. Based on the target key image, the target pose of the target vehicle is determined, and a reference running trajectory is determined based on the target fusion vector. A reference running trajectory corresponding to the target pose is determined, and an initial vehicle pose map of the target vehicle is determined based on the target pose and the reference running trajectory. Based on the reference running trajectory, the reference running trajectory, and a preset error adjustment function, pose map adjustment data is determined, and the initial vehicle pose map is updated based on the pose map adjustment data to obtain the target vehicle pose map. This scheme improves the accuracy of the determined target vehicle pose map by optimizing the initial vehicle pose map using the pose map adjustment data determined based on the reference running trajectory, the reference running trajectory, and the preset error adjustment function.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of a vehicle pose diagram determination method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a vehicle pose diagram determination method provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of a vehicle pose determination device provided in Embodiment 4 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device for implementing a vehicle pose map determination method, provided in Embodiment 5 of the present invention. Detailed Implementation

[0012] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0013] Example 1 Figure 1 This is a flowchart of a vehicle pose map determination method provided in Embodiment 1 of the present invention. This embodiment can be applied to the situation of determining the pose map of a vehicle. The method can be executed by a vehicle pose map determination device, which can be implemented in software and / or hardware and can be configured in an electronic device that carries the vehicle pose map determination function.

[0014] See Figure 1 The vehicle pose diagram determination method shown includes: S110. Determine the target key image of the target vehicle during the target running period, and the target fusion vector corresponding to the target key image.

[0015] In this context, the target vehicle refers to the vehicle for which pose mapping needs to be determined. The target runtime period refers to the time period during which images of the target vehicle are acquired. This embodiment of the invention does not impose any limitations on the setting of the target runtime period; it can be set by technical personnel based on experience or needs.

[0016] Here, the target key image refers to the key operational image of the target vehicle within the target operating period. The target fusion vector refers to the fusion vector of the target operating parameters at the time corresponding to the target key image. For example, the target operating parameters at the time corresponding to the target key image are obtained, and the target fusion vector corresponding to the target key image is determined based on the target operating parameters and the Lie algebra. Here, the target operating parameters refer to the vehicle operating parameters of the target vehicle at the time corresponding to the target key image. For example, the target operating parameters may include the target vehicle's speed and operating distance, etc.

[0017] S120. Based on the key image of the target, determine the target pose of the target vehicle, and based on the target fusion vector, determine the reference trajectory.

[0018] Here, the target pose refers to the pose of the target vehicle in the target key image. The reference trajectory refers to the actual trajectory of the target vehicle between any two adjacent target key images at any given time. The reference trajectory can be understood as the actual trajectory of the target vehicle between any two adjacent target poses.

[0019] In an optional embodiment, determining the reference trajectory based on the target fusion vector includes: for any target pose, taking the previous target pose adjacent to the target pose as the reference pose, and taking the target fusion vector corresponding to the reference pose as the reference fusion vector; and determining the reference trajectory corresponding to the target pose based on the target fusion vector and the reference fusion vector.

[0020] Here, the reference pose refers to the preceding target pose adjacent to the target pose. Specifically, for any target pose, the preceding target pose adjacent to it is taken as the reference pose. The reference fusion vector refers to the target fusion vector at the time corresponding to the reference pose.

[0021] It is understandable that by determining the reference trajectory between the target pose and the corresponding reference pose based on the target fusion vector of the target pose and the reference fusion vector of the reference pose corresponding to the target pose, the accuracy of the determined reference trajectory is improved.

[0022] S130. Determine the reference trajectory corresponding to the target pose, and determine the initial vehicle pose diagram of the target vehicle based on the target pose and the reference trajectory.

[0023] Here, the initial vehicle pose map refers to the original pose map of the target vehicle. The reference trajectory refers to the predicted trajectory of the target vehicle between any two adjacent target poses. For example, the reference trajectory can be determined using the front-end visual odometry of the target vehicle.

[0024] Specifically, determine the reference trajectory between any two adjacent key images of the target at any given time; use the target pose as nodes and the reference trajectory as edges; and construct an initial vehicle pose graph based on the nodes and edges.

[0025] S140. Based on the baseline running trajectory, the reference running trajectory, and the preset error adjustment function, determine the pose diagram adjustment data, and update the initial vehicle pose diagram according to the pose diagram adjustment data to obtain the target vehicle pose diagram.

[0026] The preset error adjustment function refers to a pre-set function used to globally optimize the initial trajectory error. The pose graph adjustment data can be used to update the initial vehicle pose graph. For example, the pose graph adjustment data may include data for updating nodes and / or edges in the initial vehicle pose graph. The target vehicle pose graph refers to the pose graph obtained after optimizing the initial vehicle pose graph.

[0027] In one optional embodiment, determining pose graph adjustment data based on a reference trajectory, a reference running trajectory, and a preset error adjustment function includes: determining the initial trajectory error corresponding to the target pose based on the reference and reference running trajectories between the target pose and the corresponding reference pose; updating the initial trajectory error based on the preset error adjustment function to obtain the target trajectory error corresponding to the target pose; and determining pose graph adjustment data based on the target trajectory error.

[0028] The initial trajectory error refers to the least squares error of the trajectory between the target pose and the corresponding reference pose. Specifically, for any target pose, the initial trajectory error between the target pose and the corresponding reference pose is determined based on the reference trajectory and the reference trajectory between the target pose and the reference pose. This initial trajectory error is the initial trajectory error corresponding to the target pose.

[0029] Among them, the target trajectory error refers to the least squares error corresponding to each target pose obtained after global optimization of the initial trajectory error.

[0030] Specifically, based on a preset error adjustment function, all initial trajectory errors are globally optimized to obtain the target trajectory error corresponding to the target pose.

[0031] Understandably, by optimizing the initial trajectory error based on a preset error adjustment function, the target trajectory error corresponding to each target pose is obtained. The pose adjustment data is then determined based on the target trajectory error, which improves the accuracy of the determined pose adjustment data.

[0032] This invention provides a vehicle pose map determination scheme. It involves determining a target key image of the target vehicle within a target runtime period, and the corresponding target fusion vector. Based on the target key image, the target pose of the target vehicle is determined, and a reference running trajectory is determined based on the target fusion vector. A reference running trajectory corresponding to the target pose is determined, and an initial vehicle pose map of the target vehicle is determined based on the target pose and the reference running trajectory. Based on the reference running trajectory, the reference running trajectory, and a preset error adjustment function, pose map adjustment data is determined, and the initial vehicle pose map is updated based on the pose map adjustment data to obtain the target vehicle pose map. This scheme improves the accuracy of the determined target vehicle pose map by optimizing the initial vehicle pose map using the pose map adjustment data determined based on the reference running trajectory, the reference running trajectory, and the preset error adjustment function.

[0033] Example 2 Figure 2 This is a flowchart of a vehicle pose map determination method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further refines the operation of "determining the target key image of the target vehicle within the target running period" into "acquiring candidate running images of the target vehicle within the target running period; determining key image data of the candidate running images based on the candidate running images, the corresponding reference key images, and the reference pose transformation matrix; and determining whether the corresponding candidate running image is a target key image based on the key image data," thereby improving the target key image determination mechanism. It should be noted that for parts not detailed in this embodiment, please refer to the descriptions in other embodiments.

[0034] See Figure 2 The vehicle pose diagram determination method shown includes: S210. Obtain candidate running images of the target vehicle within the target running time segment.

[0035] Candidate running images refer to images of the target vehicle at different times within the target running period. For example, candidate running images can be acquired using a forward-facing camera on the target vehicle.

[0036] Specifically, acquire candidate running images of the target vehicle at different times within the target running period.

[0037] S220. Determine the key image data of the candidate running image based on the candidate running image, the corresponding reference key image, and the reference pose transformation matrix.

[0038] Here, the reference key image refers to the preceding target key image adjacent to the candidate running image. The reference pose transformation matrix refers to the relative pose transformation matrix corresponding to the candidate running image.

[0039] For example, for any candidate running image, the reference pose transformation matrix corresponding to the candidate running image is determined based on the previous target key image adjacent to the candidate running image (i.e., the reference key image corresponding to the candidate running image).

[0040] Among them, key image data refers to data that can be used to determine whether a corresponding candidate running image is a target key image.

[0041] Specifically, for any candidate running image, the key image data of the candidate running image is determined based on the candidate running image, the corresponding reference key image, and the corresponding reference pose transformation matrix.

[0042] In an optional embodiment, determining key image data of a candidate running image based on a candidate running image, a corresponding reference key image, and a reference pose transformation matrix includes: for any candidate running image, determining a reference key image for that candidate running image, and inputting the candidate running image and the reference key image into a trained feature extraction model to obtain a candidate environment vector and a reference environment vector; determining the image similarity of the candidate running image based on the candidate environment vector and the reference environment vector; decomposing the reference pose transformation matrix corresponding to the candidate running image to obtain a reference translation vector and a reference rotation matrix corresponding to the candidate running image, and determining the vehicle translation distance and vehicle rotation angle corresponding to the candidate running image based on the reference translation vector and the reference rotation matrix; determining the number of key feature points based on a preset feature detection algorithm and the candidate running image; and generating key image data of the candidate running image including image similarity, vehicle translation distance, vehicle rotation angle, and the number of key feature points.

[0043] The feature extraction model can be used to extract features from candidate running images and reference key images to obtain candidate environment vectors and reference environment vectors. For example, the candidate running image is input into the trained feature extraction model to obtain the candidate environment vector; the reference key image is input into the trained feature extraction model to obtain the reference environment vector.

[0044] Here, the candidate environment vector refers to the environment vector obtained by feature extraction from the candidate running image. The reference environment vector refers to the environment vector obtained by feature extraction from the reference key image. Image similarity can be used to quantify the degree of similarity between the candidate environment vector and the reference key image.

[0045] The reference translation vector refers to the vehicle translation vector obtained by decomposing the reference pose transformation matrix. The reference rotation matrix refers to the vehicle rotation matrix obtained by decomposing the reference pose transformation matrix.

[0046] The vehicle translation distance refers to the translation distance of the target vehicle in the candidate running image, that is, the translation distance of the target vehicle at the time corresponding to the candidate running image. The vehicle rotation angle refers to the rotation angle of the target vehicle in the candidate running image, that is, the rotation angle of the target vehicle at the time corresponding to the candidate running image.

[0047] The preset feature detection algorithm refers to a pre-set algorithm used to determine key feature points in candidate running images. The number of key feature points refers to the number of valid feature points in the candidate running images. Key feature points are reference points that can characterize the movement of the target vehicle. For example, key feature points may include lane lines, traffic signs, traffic lights, guardrails, buildings, and pedestrians.

[0048] Specifically, for any candidate running image, key feature points are extracted from the candidate running image based on a preset feature detection algorithm; the number of key feature points is determined, which is the number of key feature points corresponding to the candidate running image.

[0049] Understandably, by extracting candidate environment vectors and reference environment vectors based on the feature extraction model, the accuracy of the candidate environment vectors and reference environment vectors is improved; at the same time, the key image data of any candidate running image includes the image similarity, vehicle translation distance, vehicle rotation angle and number of key feature points corresponding to the candidate running image, which improves the comprehensiveness of the determined key image data.

[0050] S230. Based on the key image data, determine whether the corresponding candidate running image is the target key image.

[0051] In an optional embodiment, determining whether a corresponding candidate running image is a target key image based on key image data includes: for any candidate running image, if the image similarity of the candidate running image is less than a preset similarity threshold, the vehicle translation distance of the candidate running image is greater than a preset distance threshold, and the number of key feature points of the candidate running image is greater than a preset feature point number threshold, then the candidate running image is determined to be a target key image.

[0052] In another optional embodiment, for any candidate running image, if the image similarity of the candidate running image is less than a preset similarity threshold, the vehicle rotation angle of the candidate running image is greater than a preset angle threshold, and the number of key feature points of the candidate running image is greater than a preset feature point number threshold, then the candidate running image is determined to be the target key image.

[0053] In this embodiment of the invention, the size of the preset similarity threshold, preset distance threshold, preset feature point number threshold, and preset angle threshold are not limited in any way. They can be set by technicians based on experience or needs, or determined repeatedly through a large number of experiments.

[0054] Understandably, the first condition can be that the image similarity is less than a preset similarity threshold, the third condition can be that the vehicle translation distance is greater than a preset distance threshold or the vehicle rotation angle is greater than a preset angle threshold, and the number of key feature points is greater than a preset feature point number threshold. When the above three conditions are met, the corresponding candidate running image is used as the target key image, thereby improving the accuracy of the determined target key image.

[0055] It should be noted that the first candidate running image obtained within the target runtime period can be directly used as the target key image.

[0056] S240. Determine the target fusion vector corresponding to the key image of the target.

[0057] S250. Based on the key image of the target, determine the target pose of the target vehicle, and based on the target fusion vector, determine the reference trajectory.

[0058] S260. Determine the reference trajectory corresponding to the target pose, and determine the initial vehicle pose diagram of the target vehicle based on the target pose and the reference trajectory.

[0059] S270. Based on the baseline running trajectory, the reference running trajectory, and the preset error adjustment function, determine the pose diagram adjustment data, and update the initial vehicle pose diagram according to the pose diagram adjustment data to obtain the target vehicle pose diagram.

[0060] This invention provides a method for determining a vehicle pose graph. By refining the operation of determining the target key image of a target vehicle within a target runtime period, the method involves acquiring candidate runtime images of the target vehicle within the target runtime period; determining key image data of the candidate runtime images based on the candidate runtime images, their corresponding reference key images, and a reference pose transformation matrix; and determining whether a corresponding candidate runtime image is a target key image based on the key image data, thus improving the target key image determination mechanism. This scheme, by determining the target key image based on the key image data of the candidate runtime images, their corresponding reference key images, and the reference pose transformation matrix, achieves filtering of candidate runtime images, reduces the number of nodes and edges subsequently optimized in the initial vehicle pose graph, lowers the computational load, and improves the accuracy of the determined target key image.

[0061] Example 3 This invention provides an optional example based on the above embodiments. It should be noted that for parts not described in detail in this invention's embodiments, please refer to the descriptions in other embodiments.

[0062] This invention proposes a pose graph optimization algorithm based on deep learning. First, a keyframe discrimination mechanism (i.e., determining target key images) is constructed, integrating high-dimensional features from deep learning with multi-dimensional geometric constraints. This method uses a deep neural network to perform strong representation learning of scene information, combined with quantitative indicators such as camera motion amplitude and image feature richness, to achieve adaptive keyframe determination, improving system efficiency while ensuring localization and mapping accuracy. Then, using keyframes from different camera moments as graph nodes and motion estimates between adjacent frames as graph edges, relative pose relationships are constructed using Lie algebras and transformation matrices, and a least-squares error function is established. A comprehensive optimization objective is constructed by considering observation errors at all moments, and finally, a graph optimization algorithm is used to minimize the weighted error, thereby eliminating the cumulative drift caused by the front-end odometry and obtaining a globally consistent and more accurate localization trajectory.

[0063] For example, to address the problems of poor generalization and difficulty in keyframe extraction in current pose graph-based backend optimization algorithms, this invention proposes a deep learning-based pose graph optimization algorithm. First, a keyframe discrimination mechanism integrating high-dimensional features from deep learning and multi-dimensional geometric constraints is constructed. Then, a pose graph optimization method—an optimization method that only targets camera motion trajectories and ignores complex landmark information—is adopted, a method common in similar algorithms.

[0064] For example, to achieve efficient and robust keyframe selection in a visual SLAM (Simultaneous Localization and Mapping) system and avoid unnecessary computational burden on subsequent pose graph optimization caused by redundant frames, this invention constructs a keyframe discrimination mechanism that integrates high-dimensional features from deep learning with multi-dimensional geometric constraints. This method uses a deep neural network to perform strong representation learning of scene information, combined with quantitative indicators such as camera motion amplitude and image feature richness, to achieve adaptive keyframe determination, thereby improving system efficiency while ensuring localization and mapping accuracy.

[0065] For example, a deep convolutional neural network (i.e., a feature extraction model) is used to perform global feature extraction on the current input image Ik (i.e., any non-first candidate running image Ik) and the previous keyframe Ik-1 (i.e., the reference keyframe Ik-1 adjacent to the candidate running image), respectively, to obtain high-dimensional feature vectors Fk (i.e., the candidate environment vector Fk corresponding to the candidate running image) and Fk-1 (i.e., the reference environment vector Fk-1 of the reference keyframe corresponding to the candidate running image) with strong representational capabilities. Compared with traditional hand-designed features, deep learning features can more comprehensively describe the texture, structure, and semantic information of images, thereby more accurately measuring inter-frame visual differences.

[0066] For example, to quantify the similarity between two images, cosine similarity is used for calculation: ; in, This represents the image similarity between the candidate running image Ik and the reference key image Ik-1, with a value ranging from [0,1]. The closer the image similarity is to 1, the closer the scene content of the two images is, and the higher the information redundancy; the smaller the image similarity is, the more obvious the scene changes, and the corresponding candidate running image has the potential to become a key frame (i.e., the target key image).

[0067] For example, based on this, camera pose change is introduced as a geometric constraint. The relative pose transformation matrix ΔT between the candidate running image Ik and the reference key image Ik-1 is obtained by the front-end visual odometry (i.e., the reference pose transformation matrix corresponding to the candidate running image Ik), and the translation vector t (i.e., the reference translation vector corresponding to the candidate running image Ik) and the rotation matrix R (i.e., the reference rotation matrix corresponding to the candidate running image Ik) are decomposed from it.

[0068] For example, the camera translation distance (i.e., the vehicle translation distance) and rotation angle (i.e., the vehicle rotation angle) can be determined by the following formulas respectively: ; ; Where d represents the vehicle translation distance; This represents the translation vector in the x-direction; This represents the translation vector in the y-direction; This represents the translation vector in the z-direction; tr(R) represents the vehicle rotation angle; tr(R) represents the trace of the rotation matrix.

[0069] For example, a new frame can only provide valid new information and thus have the value of a keyframe if the camera's motion (i.e., the vehicle's motion) is large enough.

[0070] For example, to ensure the stability of pose estimation and optimization, the target key image must contain a sufficient number of effective feature points (i.e., key feature points). Key feature points are extracted from the candidate running image Ik using a preset feature detection algorithm, and the number of key feature points corresponding to the candidate running image Ik is counted to satisfy the minimum feature point number constraint Nmin, thus avoiding instability in pose optimization due to insufficient features.

[0071] For example, combining the above deep learning feature similarity and geometric constraints, the following multi-condition fusion keyframe discrimination rule is constructed: ; in, This indicates a preset similarity threshold; Indicates the preset distance threshold; Indicates the preset angle threshold; This represents the number of key feature points corresponding to the candidate running image Ik; This represents the preset threshold for the number of feature points. A candidate running image Ik is determined to be a keyframe only if all conditions are met simultaneously; otherwise, it is considered a redundant frame and discarded directly to reduce the number of optimization nodes and constraint edges, thereby reducing the computational load on the backend.

[0072] For example, through the above-mentioned fusion discrimination mechanism, the system can ensure both global trajectory accuracy and ground accuracy. Figure 1 While maintaining consistency, redundant data is significantly reduced, making pose graph optimization more efficient and stable.

[0073] For example, a sliding window can be added and participate in subsequent pose graph optimization. If no loops appear in the localization path, the input sliding window for continuous images will be continuously pushed forward over time. The nodes in the initial vehicle pose graph represent the vehicle's pose at the corresponding time, and the edges connecting the nodes represent the motion estimation (i.e., reference trajectory) of the pose points at the two time points corresponding to the two adjacent target key images by the front-end visual odometry.

[0074] For example, in the absence of external influences, the correspondence between the baseline trajectory and the corresponding reference trajectory can be determined using the following formula: ; in, This represents the reference trajectory between the i-th node and the j-th node in the initial vehicle pose graph; This represents the baseline trajectory between the i-th node and the j-th node in the initial vehicle pose graph; This represents the target fusion vector corresponding to the i-th node; This represents the target fusion vector corresponding to the j-th node; This represents vector multiplication; the j-th node refers to the node preceding the i-th node, and the target fusion vector corresponding to the j-th node is the base fusion vector corresponding to the i-th node.

[0075] For example, to facilitate calculation, the correspondence between the above-mentioned baseline trajectory and the corresponding reference trajectory can be replaced by a transformation matrix: ; in, Indicates the converted ; Indicates the converted ; Indicates the converted .

[0076] For example, the pose represented by a node in the initial vehicle pose image can be determined by key feature points in the corresponding target key image. The target fusion vector corresponding to any target pose is determined based on Lie algebra and according to the target's operating parameters at the corresponding time point of that target pose.

[0077] For example, in practice, due to external factors, there may be an error between the reference trajectory and the corresponding baseline trajectory. The initial trajectory error can be determined based on the following formula: ; in, This represents the initial trajectory error between the i-th node and the j-th node in the initial vehicle pose graph, i.e. Let v represent the initial trajectory error corresponding to the i-th node; v represents the adjoint matrix. This represents the baseline trajectory between the i-th node and the j-th node in the initial vehicle pose graph; This represents the reference trajectory between the i-th node and the j-th node in the initial vehicle pose graph.

[0078] For example, a complete pose graph consists of all pose nodes and edges representing the relative poses between nodes. It should be noted that the connecting edges between nodes include the relative motion relationships between keyframes. The algorithm comprehensively considers the errors at all time points, establishing a least-squares problem for the overall task. The overall optimization objective function (i.e., the preset error adjustment function) can be expressed as: ; Where eij represents the information matrix obtained by multiplying the i-th edge and the j-th edge in the initial vehicle pose graph; the i-th edge and the j-th edge belong to the set of all edges in the initial vehicle pose graph.

[0079] For example, in this embodiment of the invention, the backend optimization module uses a graph optimization algorithm to minimize the above objective function, and finally obtains the optimized positioning result.

[0080] This invention proposes a pose graph optimization algorithm based on deep learning. First, a keyframe discrimination mechanism integrating high-dimensional features from deep learning and multi-dimensional geometric constraints is constructed. Then, a pose graph optimization method—an optimization method that focuses only on camera motion trajectory and ignores complex landmark information—is adopted, which is unique among similar algorithms. It uses the camera's pose at different times as graph nodes and the motion estimates between adjacent frames as graph edges. Relative pose relationships are constructed using Lie algebras and transformation matrices, and a least-squares error function is established. The overall optimization objective is constructed by comprehensively considering the observation errors at all times. Finally, the graph optimization algorithm minimizes the weighted error, thereby eliminating the cumulative drift caused by the front-end odometry and obtaining a globally consistent and more accurate positioning trajectory.

[0081] Example 4 Figure 3 This is a schematic diagram of a vehicle pose map determination device provided in Embodiment 4 of the present invention. This embodiment is applicable to the determination of a vehicle pose map. The method can be executed by a vehicle pose map determination device, which can be implemented in software and / or hardware and can be configured in an electronic device that carries the vehicle pose map determination function.

[0082] like Figure 3 As shown, the device includes: a target fusion vector determination module 310, a reference trajectory determination module 320, an initial pose map determination module 330, and a target pose map determination module 340. Among them, The target fusion vector determination module 310 is used to determine the target key image of the target vehicle during the target running period, and the target fusion vector corresponding to the target key image; The reference trajectory determination module 320 is used to determine the target pose of the target vehicle based on the target key image, and to determine the reference trajectory based on the target fusion vector. The initial pose determination module 330 is used to determine the reference running trajectory corresponding to the target pose, and to determine the initial vehicle pose map of the target vehicle based on the target pose and the reference running trajectory. The target pose map determination module 340 is used to determine pose map adjustment data based on the baseline running trajectory, the reference running trajectory and the preset error adjustment function, and update the initial vehicle pose map based on the pose map adjustment data to obtain the target vehicle pose map.

[0083] This invention provides a vehicle pose map determination scheme. It involves determining a target key image of the target vehicle within a target runtime period, and the corresponding target fusion vector. Based on the target key image, the target pose of the target vehicle is determined, and a reference running trajectory is determined based on the target fusion vector. A reference running trajectory corresponding to the target pose is determined, and an initial vehicle pose map of the target vehicle is determined based on the target pose and the reference running trajectory. Based on the reference running trajectory, the reference running trajectory, and a preset error adjustment function, pose map adjustment data is determined, and the initial vehicle pose map is updated based on the pose map adjustment data to obtain the target vehicle pose map. This scheme improves the accuracy of the determined target vehicle pose map by optimizing the initial vehicle pose map using the pose map adjustment data determined based on the reference running trajectory, the reference running trajectory, and the preset error adjustment function.

[0084] Optionally, the target fusion vector determination module 310 includes: A candidate running image acquisition unit is used to acquire candidate running images of the target vehicle during the target running period; The key image data determination unit is used to determine the key image data of the candidate running image based on the candidate running image, the reference key image corresponding to the candidate running image, and the reference pose transformation matrix. The target key image determination unit is used to determine whether a corresponding candidate running image is a target key image based on the key image data.

[0085] Optionally, the key image data determination unit includes: The environment vector determination subunit is used to determine the reference key image of any candidate running image, and input the candidate running image and the reference key image into the trained feature extraction model to obtain the candidate environment vector and the reference environment vector, respectively. The image similarity determination subunit is used to determine the image similarity of the candidate running image based on the candidate environment vector and the reference environment vector. The vehicle rotation angle determination subunit is used to decompose the reference pose transformation matrix corresponding to the candidate running image to obtain the reference translation vector and reference rotation matrix corresponding to the candidate running image, and determine the vehicle translation distance and vehicle rotation angle corresponding to the candidate running image based on the reference translation vector and the reference rotation matrix. The key feature point quantity determination subunit is used to determine the number of key feature points based on the preset feature detection algorithm and the candidate running image; The key image data generation subunit is used to generate key image data for the candidate running image, including the image similarity, the vehicle translation distance, the vehicle rotation angle, and the number of key feature points.

[0086] Optionally, the target key image determination unit is specifically used for: For any candidate running image, if the image similarity of the candidate running image is less than a preset similarity threshold, the vehicle translation distance of the candidate running image is greater than a preset distance threshold, and the number of key feature points of the candidate running image is greater than a preset feature point number threshold, then the candidate running image is determined to be the target key image.

[0087] Optionally, the target key image determination unit is specifically used for: For any candidate running image, if the image similarity of the candidate running image is less than a preset similarity threshold, the vehicle rotation angle of the candidate running image is greater than a preset angle threshold, and the number of key feature points of the candidate running image is greater than a preset feature point number threshold, then the candidate running image is determined to be the target key image.

[0088] Optionally, the reference trajectory determination module 320 includes: The reference fusion vector determination unit is used to determine the previous target pose adjacent to the target pose as the reference pose for any target pose, and to determine the target fusion vector corresponding to the reference pose as the reference fusion vector. The reference trajectory determination unit is used to determine the reference trajectory corresponding to the target pose based on the target fusion vector corresponding to the target pose and the reference fusion vector.

[0089] Optionally, the target pose graph determination module 340 includes: The initial trajectory error determination unit is used to determine the initial trajectory error corresponding to the target pose based on the reference running trajectory and the reference running trajectory between the target pose and the corresponding reference pose. The target trajectory error determination unit is used to update the initial trajectory error based on the preset error adjustment function to obtain the target trajectory error corresponding to the target pose. The pose map adjustment data determination unit is used to determine pose map adjustment data based on the target trajectory error.

[0090] The vehicle pose map determination device provided in the embodiments of the present invention can execute the vehicle pose map determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing each vehicle pose map determination method.

[0091] The collection, storage, use, processing, transmission, provision, and disclosure of candidate running images and other related processes in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0092] Example 5 Figure 4 This is a schematic diagram of an electronic device for implementing a vehicle pose map determination method, provided in Embodiment 5 of the present invention. The electronic device 410 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0093] like Figure 4 As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0094] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0095] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as vehicle pose map determination methods.

[0096] In some embodiments, the vehicle pose determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded into and / or mounted on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the vehicle pose determination method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured to perform the vehicle pose determination method by any other suitable means (e.g., by means of firmware).

[0097] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0101] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0102] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0103] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0104] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for determining a vehicle pose map, characterized in that, include: Determine the target key image of the target vehicle during the target operating period, and the target fusion vector corresponding to the target key image; Based on the target key image, the target pose of the target vehicle is determined, and based on the target fusion vector, the reference running trajectory is determined; Determine the reference trajectory corresponding to the target pose, and determine the initial vehicle pose diagram of the target vehicle based on the target pose and the reference trajectory; Based on the baseline running trajectory, the reference running trajectory, and the preset error adjustment function, pose map adjustment data is determined, and the initial vehicle pose map is updated according to the pose map adjustment data to obtain the target vehicle pose map.

2. The method according to claim 1, characterized in that, The determination of the target key image of the target vehicle within the target operating period includes: Acquire candidate running images of the target vehicle during the target running period; Based on the candidate running image, the corresponding reference key image, and the reference pose transformation matrix, the key image data of the candidate running image is determined. Based on the key image data, determine whether the corresponding candidate running image is the target key image.

3. The method according to claim 2, characterized in that, The step of determining the key image data of the candidate running image based on the candidate running image, the corresponding reference key image, and the reference pose transformation matrix includes: For any candidate running image, a reference key image for that candidate running image is determined, and the candidate running image and the reference key image are respectively input into the trained feature extraction model to obtain the candidate environment vector and the reference environment vector; The image similarity of the candidate running image is determined based on the candidate environment vector and the reference environment vector. The reference pose transformation matrix corresponding to the candidate running image is decomposed to obtain the reference translation vector and reference rotation matrix corresponding to the candidate running image. Based on the reference translation vector and the reference rotation matrix, the vehicle translation distance and vehicle rotation angle corresponding to the candidate running image are determined. The number of key feature points is determined based on the preset feature detection algorithm and the candidate running image; Generate key image data for the candidate running image, including the image similarity, the vehicle translation distance, the vehicle rotation angle, and the number of key feature points.

4. The method according to claim 3, characterized in that, The step of determining whether a corresponding candidate running image is a target key image based on the key image data includes: For any candidate running image, if the image similarity of the candidate running image is less than a preset similarity threshold, the vehicle translation distance of the candidate running image is greater than a preset distance threshold, and the number of key feature points of the candidate running image is greater than a preset feature point number threshold, then the candidate running image is determined to be the target key image.

5. The method according to claim 3, characterized in that, The step of determining whether a corresponding candidate running image is a target key image based on the key image data includes: For any candidate running image, if the image similarity of the candidate running image is less than a preset similarity threshold, the vehicle rotation angle of the candidate running image is greater than a preset angle threshold, and the number of key feature points of the candidate running image is greater than a preset feature point number threshold, then the candidate running image is determined to be the target key image.

6. The method according to claim 1, characterized in that, The step of determining the baseline trajectory based on the target fusion vector includes: For any target pose, the previous target pose adjacent to the target pose is taken as the reference pose, and the target fusion vector corresponding to the reference pose is taken as the reference fusion vector. Based on the target fusion vector corresponding to the target pose and the reference fusion vector, the reference trajectory corresponding to the target pose is determined.

7. The method according to claim 1, characterized in that, The step of determining pose map adjustment data based on the baseline running trajectory, the reference running trajectory, and the preset error adjustment function includes: Based on the reference trajectory and reference trajectory between the target pose and the corresponding reference pose, determine the initial trajectory error corresponding to the target pose; Based on the preset error adjustment function, the initial trajectory error is updated to obtain the target trajectory error corresponding to the target pose; Based on the target trajectory error, determine the pose map adjustment data.

8. A vehicle pose map determination device, characterized in that, include: The target fusion vector determination module is used to determine the target key image of the target vehicle during the target running period, and the target fusion vector corresponding to the target key image; The baseline trajectory determination module is used to determine the target pose of the target vehicle based on the target key image, and to determine the baseline trajectory based on the target fusion vector. The initial pose determination module is used to determine the reference running trajectory corresponding to the target pose, and to determine the initial vehicle pose of the target vehicle based on the target pose and the reference running trajectory. The target pose map determination module is used to determine pose map adjustment data based on the baseline running trajectory, the reference running trajectory and the preset error adjustment function, and update the initial vehicle pose map based on the pose map adjustment data to obtain the target vehicle pose map.

9. An electronic device, comprising: include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a vehicle pose map determination method as described in any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When executed by the processor, the program implements a vehicle pose diagram determination method as described in any one of claims 1-7.

Images