Visual repositioning method, device, computer equipment and storage medium

Abstract

The present invention relates to the field of global positioning technology, and specifically to a visual repositioning method, apparatus, computer equipment, and storage medium. The visual repositioning method comprises: obtaining a current local frame corresponding to a current trajectory point in an initial trajectory, aligning it with a priori point cloud map to obtain an alignment position, back-projecting the priori point cloud map onto multiple acquired images of the current local frame according to the alignment position to obtain evaluation parameters corresponding to each alignment position, and then determining a reliability value of the current local frame according to the evaluation parameters. When the reliability value of the current local frame is greater than a preset first threshold, the current local frame is used as an alignment frame; and repositioning the initial trajectory according to the multiple alignment frames to obtain an anchor trajectory. Thus, the co-visibility of the alignment position in multiple acquired images can be evaluated by the evaluation parameters, and further, whether the current local frame is reliable can be determined based on the co-visibility, thereby eliminating some mismatched local frames and reducing the mismatch rate.

Description

Technical Field

[0001] The present invention relates to the field of global positioning technology, and in particular to a visual repositioning method, device, computer equipment and storage medium. Background Art

[0002] Visual relocalization analyzes image content, matches 2D or 3D features, and associates query images with pre-built scenes to determine the precise camera pose within the known scene. With the advancement of multi-sensor data fusion technology, relocalization systems leveraging multi-sensor data have become the preferred choice for high-precision positioning architectures. However, in practice, the long time span and wide variations in viewpoints associated with relocalization tasks can lead to high mismatch rates. Summary of the Invention

[0003] In view of this, the present invention provides a visual relocalization method, apparatus, computer device and storage medium to solve the problem of high mismatching rate in current visual relocalization methods.

[0004] In a first aspect, the present invention provides a visual relocalization method, comprising: obtaining an initial trajectory and a reference image library; obtaining a current local frame corresponding to a current trajectory point in the initial trajectory, and aligning the current local frame with a priori point cloud map to obtain an alignment position; wherein the current local frame includes captured images of multiple cameras; back-projecting the priori point cloud map into multiple captured images of the current local frame according to the alignment position, and obtaining evaluation parameters corresponding to each alignment position; determining a reliability value of the current local frame according to the evaluation parameters corresponding to each alignment position; when the reliability value of the current local frame is greater than a preset first threshold, using the current local frame as an alignment frame; and relocating the initial trajectory according to the multiple alignment frames to obtain an anchor trajectory.

[0005] The visual relocalization method provided by the present invention obtains a current local frame corresponding to the current trajectory point in the initial trajectory, aligns the current local frame with the prior point cloud map to obtain an alignment position, and back-projects the prior point cloud map onto multiple captured images of the current local frame based on the alignment position to obtain evaluation parameters corresponding to each alignment position; thereby, the reliability value of the current local frame can be calculated based on the evaluation parameters, and when the reliability value of the current local frame is greater than a preset first threshold, the current local frame is used as the alignment frame; the initial trajectory is relocated based on the multiple alignment frames to obtain an anchor trajectory. This is because the cameras of the same vehicle have a certain common view range. After back-projecting the prior point cloud map onto multiple captured images of the current local frame based on the alignment position, the common view of the alignment position in the multiple captured images can be evaluated using the evaluation parameters. Further, the reliability of the current local frame can be determined based on the common view, thereby eliminating some mismatched local frames and reducing the mismatch rate.

[0006] In an optional embodiment, the evaluation parameters corresponding to each alignment position include at least one of the following: the number of acquired images in all acquired images of the current local frame that meet the reprojection error constraint, the number of points in all acquired images of the current local frame that meet the reprojection error constraint, and the average reprojection error of all acquired images of the current local frame.

[0007] This is because the cameras of the same vehicle have a certain common view range. After the prior point cloud map is back-projected into multiple acquired images of the current local frame according to the alignment position, the evaluation parameters are used to evaluate the common view of the alignment position in multiple acquired images.

[0008] In an optional embodiment, determining the reliability value of the current local frame based on the evaluation parameters corresponding to each alignment position includes: calculating the ratio of the number of captured images corresponding to each alignment position that meet the reprojection error constraint to the total number of captured images in the current local frame to obtain a first ratio; calculating the ratio of the number of points corresponding to each alignment position that meet the reprojection error constraint to the total number of matching points of all captured images in the current local frame to obtain a second ratio; calculating the ratio of the average reprojection error corresponding to each alignment position to a preset reprojection error limit to obtain a third ratio; calculating the reliability value corresponding to each of the alignment positions based on the first ratio, the second ratio and the third ratio; and determining the reliability value of the current local frame based on the reliability value corresponding to each alignment position.

[0009] In this way, the reliability value of the current local frame can be accurately determined.

[0010] In an optional embodiment, relocating the initial trajectory according to the multiple aligned frames to obtain the anchor trajectory includes: aligning each aligned frame with the reference image to obtain multiple relocated trajectories; determining a weight value for each relocated trajectory; and fusing the multiple relocated trajectories according to the weight values ​​of the multiple relocated trajectories to obtain the anchor trajectory.

[0011] The present invention divides the initial trajectory into multiple line segments, which can make the anchor point frames evenly distributed, and in each interval there is a result with high anchoring quality to constrain the trajectory.

[0012] In an optional embodiment, determining the weight value of each relocalization trajectory includes: dividing the initial trajectory into multiple line segments; obtaining multiple alignment frames belonging to the current line segment, selecting an alignment frame with the highest reliability value from the multiple alignment frames belonging to the current line segment, and using the selected alignment frame as the anchor frame of the current line segment, and setting the weight value of the relocalization trajectory corresponding to the anchor frame to 1; and determining the weights of the relocalization trajectories corresponding to the other alignment frames based on the reliability values ​​of the other alignment frames except the anchor frame.

[0013] That is to say, according to the length of the initial trajectory, the initial trajectory is segmented, and the alignment frame with the highest reliability value in each segment is set as the anchor frame. During the optimization process, the weight of the anchor frame is 1, and the posture of the anchor frame will be fixed. The postures of the remaining alignment frames except the anchor frame are determined by the weights of the reliability values. Come to.

[0014] In an optional embodiment, aligning the current local frame with the prior point cloud map to obtain the alignment position includes: determining the 2D-2D matching relationship between the captured image of the current local frame and the reference image; constructing a 2D-3D correspondence between the captured image of the current local frame and the prior point cloud map based on the 2D-2D matching relationship and the prior point cloud map; and calculating the alignment position of the current local frame in the prior point cloud map based on the 2D-3D correspondence between the captured image of the current local frame and the prior point cloud map.

[0015] In this way, the current local frame can be accurately aligned with the prior point cloud map.

[0016] In an optional embodiment, before aligning the current local frame with the prior point cloud map to obtain the alignment position, it also includes: extracting global descriptors for all acquired images in the current local frame to obtain a first result; extracting global descriptors for all reference images in the reference image library to obtain a second result; based on the first result and the second result, judging whether the number of reference images corresponding to the current acquired image in the current local frame in the reference image library is less than a preset second threshold; when the number of reference images corresponding to the current acquired image in the reference image library is less than the second threshold, removing the current acquired image from the current local frame.

[0017] In this way, the collected images without the common viewing condition in the current local frame can be eliminated, thereby speeding up subsequent calculations.

[0018] In an optional embodiment, the visual relocalization method further includes the following steps: after obtaining the anchor trajectory, obtaining inter-frame constraint parameters and vehicle velocity, wherein the inter-frame constraint parameters include at least one of the following: vehicle angular velocity, vehicle acceleration; calculating a pre-integration term based on the inter-frame constraint parameters; and optimizing multiple anchor trajectories corresponding to the initial trajectory using a nonlinear least squares method based on the pre-integration term and the vehicle velocity to obtain a relocalization trajectory.

[0019] Therefore, a multi-sensor data fusion factor graph can be constructed based on multi-sensor data to further optimize the accuracy of the anchor trajectory.

[0020] In an optional embodiment, the visual relocalization method further includes the following steps: after obtaining the anchor trajectory, obtaining a vector map in a reference image library; and optimizing multiple anchor trajectories corresponding to the initial trajectory using the Levenberg-Marquardt method according to the vector map to obtain a relocalization trajectory.

[0021] The accuracy of the anchor trajectory is thus further optimized based on the vector map.

[0022] In an optional embodiment, the visual relocalization method further includes the following steps: after obtaining the anchor trajectory, obtaining a jitter degree parameter of the anchor trajectory; determining an evaluation result of the anchor trajectory based on the jitter degree parameter; and / or; after obtaining the anchor trajectory, back-projecting the vector map onto the captured image in the current local frame to obtain a degree of overlap between the vector map and the captured image; and determining an evaluation result of the anchor trajectory based on the degree of overlap.

[0023] Therefore, starting from the perspective of trajectory geometry and / or the consistency between the high-precision map and the actual scene, the jitter degree of the trajectory and the coincidence index of the high-precision map back-projected semantic image can be calculated to perform a fully automatic quantitative evaluation of the obtained relocalization results.

[0024] In the second aspect, the present invention also provides a visual repositioning device, which includes a first acquisition module, a preprocessing module, an alignment module, a second acquisition module, a reliability value calculation module, an alignment frame determination module and an anchor trajectory determination module; the first acquisition module is used to acquire an initial trajectory and a reference image library; the preprocessing module is used to determine a priori point cloud map based on the reference image; the alignment module is used to acquire a current local frame corresponding to the current trajectory point in the initial trajectory, and align the current local frame with the priori point cloud map to obtain an alignment position; wherein the current local frame includes captured images of multiple cameras; the second acquisition module is used to back-project the priori point cloud map into multiple captured images of the current local frame according to the alignment position, and obtain evaluation parameters corresponding to each alignment position; the reliability value calculation module is used to determine the reliability value of the current local frame according to the evaluation parameters corresponding to each alignment position; the alignment frame determination module is used to use the current local frame as the alignment frame when the reliability value of the current local frame is greater than a preset first threshold; the anchor trajectory determination module is used to reposition the initial trajectory according to multiple alignment frames to obtain an anchor trajectory.

[0025] In a third aspect, the present invention further provides a computer device comprising a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the visual repositioning method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0026] In a fourth aspect, the present invention further provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the visual repositioning method of the first aspect or any corresponding embodiment thereof.

[0027] In a fifth aspect, the present invention further provides a computer program product, comprising computer instructions for causing a computer to execute the visual repositioning method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 is a flowchart of a visual relocalization method according to an embodiment of the present invention;

[0030] Figure 2 is a flow chart of another visual relocalization method according to an embodiment of the present invention;

[0031] Figure 3 is a flowchart of another visual relocalization method according to an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of constructing a multi-sensor fusion factor graph according to an embodiment of the present invention;

[0033] Figure 5 is a schematic diagram of track jitter inspection according to an embodiment of the present invention;

[0034] Figure 6 2. It is a schematic diagram of a vector map back-projection semantic image according to an embodiment of the present invention;

[0035] Figure 7 is a structural block diagram of a visual relocalization device according to an embodiment of the present invention;

[0036] Figure 8 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0038] According to an embodiment of the present invention, an embodiment of a visual repositioning method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] In this embodiment, a visual relocalization method is provided, which can be used in a computer device. Figure 1 is a flow chart of a visual relocalization method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0040] Step S101: Obtain an initial trajectory and a reference image library.

[0041] In the embodiment, a plurality of cameras are provided on the vehicle, and the initial trajectory is obtained from images collected by the plurality of cameras (also referred to as captured images); the reference image library includes reference images and vector maps.

[0042] Step S102: Determine a priori point cloud map based on the reference image.

[0043] Step S103: obtaining a current local frame corresponding to the current trajectory point in the initial trajectory, and aligning the current local frame with the prior point cloud map to obtain an alignment position; wherein the current local frame includes images collected by multiple cameras.

[0044] That is to say, multiple images taken by different cameras at the same time are called local frames.

[0045] Step S104: back-projecting the prior point cloud map onto multiple captured images of the current local frame according to the alignment positions, and obtaining evaluation parameters corresponding to each alignment position.

[0046] The cameras of the same vehicle have a certain common view range. After the prior point cloud map is back-projected into multiple acquired images of the current local frame according to the alignment position, the evaluation parameters are used to evaluate the common view of the alignment position in multiple acquired images.

[0047] Step S105: determining a reliability value of the current local frame according to the evaluation parameters corresponding to each alignment position.

[0048] That is, in this embodiment, whether the current local frame is reliable is determined based on the co-viewing situation of the alignment position in multiple captured images, so that some mismatched local frames can be eliminated, thereby improving the accuracy of repositioning.

[0049] Step S106: When the reliability value of the current partial frame is greater than a preset first threshold, the current partial frame is used as the alignment frame.

[0050] Step S107: relocating the initial trajectory according to the multiple aligned frames to obtain an anchor trajectory.

[0051] The visual relocalization method provided in this embodiment determines a priori point cloud map based on a reference image library, obtains a current local frame corresponding to the current trajectory point in the initial trajectory, aligns the current local frame with the priori point cloud map to obtain an alignment position, and back-projects the priori point cloud map onto multiple captured images of the current local frame based on the alignment position to obtain evaluation parameters corresponding to each of the alignment positions; thereby, a reliability value of the current local frame can be calculated based on the evaluation parameters, and when the reliability value of the current local frame is greater than a preset first threshold, the current local frame is used as the alignment frame; the initial trajectory is relocated based on the multiple aligned frames to obtain an anchored trajectory. This is because the cameras of the same vehicle have a certain common view range. After back-projecting the priori point cloud map onto multiple captured images of the current local frame based on the alignment position, the common view of the alignment position in the multiple captured images can be evaluated using the evaluation parameters. Further, the reliability of the current local frame can be determined based on the common view, thereby eliminating some mismatched local frames and reducing the mismatch rate.

[0052] In this embodiment, a visual relocalization method is provided, which can be used in a computer device. Figure 2 is a flow chart of another visual relocalization method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0053] Step S201: Acquire an initial trajectory and a reference image library.

[0054] Step S202: Determine a priori point cloud map based on the reference image.

[0055] Step S203: obtaining a current local frame corresponding to the current trajectory point in the initial trajectory, and aligning the current local frame with the prior point cloud map to obtain an alignment position; wherein the current local frame includes images collected by multiple cameras.

[0056] In an optional embodiment, aligning the current local frame with the prior point cloud map to obtain the alignment position includes: determining the 2D-2D matching relationship between the captured image of the current local frame and the reference image; constructing a 2D-3D correspondence between the captured image of the current local frame and the prior point cloud map based on the 2D-2D matching relationship and the prior point cloud map; and calculating the alignment position of the current local frame in the prior point cloud map based on the 2D-3D correspondence between the captured image of the current local frame and the prior point cloud map.

[0057] It should be noted that, in this embodiment, the current partial frame includes multiple images, for example, 7 images. After the current partial frame is aligned with the reference image, 7 alignment positions are obtained.

[0058] Specifically, before aligning the current local frame with the prior point cloud map to obtain the alignment position, the current local frame is also subjected to semantic segmentation and feature extraction. Semantic segmentation uses deep learning to segment dynamic objects in the image, such as vehicles and pedestrians, as feature extraction masks. It also extracts information such as lane lines and road signs from the scene for subsequent pixel-level matching optimization in the high-precision map. During feature extraction, dynamic objects are removed using this mask, and feature points and their descriptors are extracted from the static scene as much as possible.

[0059] Furthermore, before aligning the current local frame with the prior point cloud map to obtain the alignment position, the following steps are also included: extracting global descriptors for all acquired images in the current local frame to obtain a first result; extracting global descriptors for all reference images in the reference image library to obtain a second result; based on the first result and the second result, judging whether the number of reference images corresponding to the current acquired image in the current local frame in the reference image library is less than a preset second threshold; when the number of reference images corresponding to the current acquired image in the reference image library is less than the second threshold, removing the current acquired image from the current local frame.

[0060] In this way, the collected images without the common viewing condition in the current local frame can be eliminated, thereby speeding up subsequent calculations.

[0061] Step S204: back-projecting the prior point cloud map onto multiple captured images of the current local frame according to the alignment positions, and obtaining evaluation parameters corresponding to each alignment position.

[0062] In an optional embodiment, back-projecting the prior point cloud map into multiple acquired images of the current local frame based on the alignment position, and obtaining evaluation parameters corresponding to each alignment position includes: back-projecting the prior point cloud map into multiple acquired images of the current local frame based on the alignment position and the 2D-3D relationship, and obtaining evaluation parameters corresponding to each alignment position.

[0063] Specifically, the evaluation parameters corresponding to each alignment position include at least one of the following: the number of acquired images that meet the reprojection error constraint in all acquired images of the current local frame, the number of points that meet the reprojection error constraint in all acquired images of the current local frame, and the average reprojection error of all acquired images of the current local frame.

[0064] Step S205: determining a reliability value of the current local frame according to the evaluation parameters corresponding to each alignment position.

[0065] Specifically, determining the reliability value of the current local frame according to the evaluation parameters corresponding to each alignment position includes the following steps:

[0066] Step S2051: Calculate the ratio of the number of collected images corresponding to each alignment position that meet the reprojection error constraint to the total number of collected images in the current local frame to obtain a first ratio.

[0067] Step S2052: Calculate the ratio of the number of points corresponding to each alignment position that meet the reprojection error constraint to the total number of matching points of all captured images in the current local frame to obtain a second ratio.

[0068] Step S2053: Calculate the ratio of the average reprojection error corresponding to each alignment position to the preset reprojection error tolerance to obtain a third ratio.

[0069] Step S2054: calculating a reliability value corresponding to each alignment position according to the first ratio, the second ratio, and the third ratio; and determining a reliability value of the current local frame according to the reliability value corresponding to each alignment position.

[0070] For example, the reliability value corresponding to each alignment position can be calculated by the following formula:

[0071]

[0072] in, A reliable value representing the current alignment position, represents the first weight, Indicates the number of collected images in the current local frame that meet the reprojection error constraint; Indicates the total number of images collected in the current local frame; represents the second weight; Represents the number of points in all acquired images of the current local frame that meet the reprojection error constraint; Indicates the total number of matching points of all captured images in the current local frame; represents the third weight; Represents the average reprojection error of all collected images in the current local frame; Represents the reprojection error tolerance. 、 and It is a fixed value in actual implementation.

[0073] For example, after aligning the current partial frame with the reference image, 7 alignment positions are obtained, and thus 7 reliable values ​​are obtained. The value with the highest score among the 7 reliable values ​​is selected as the reliable value of the current partial frame.

[0074] Step S206: When the reliability value of the current partial frame is greater than a preset first threshold, the current partial frame is used as an alignment frame.

[0075] In this embodiment, the cameras of the same vehicle have a certain common view range. The multiple images taken by the cameras at the same time are called local frames. The 2D-2D matching relationship between the local frame and the reference image and the prior point cloud map corresponding to the reference image features are used to construct the 2D-3D correspondence between the local frame and the prior point cloud map. The position of the local frame in the prior point cloud map is calculated by the PnP algorithm, so that the prior point cloud map can be back-projected into the various camera images in the local frame, and the common view of the prior point cloud map in each camera image is counted, so as to determine whether the relocation result is valid.

[0076] Step S207: relocating the initial trajectory according to the multiple aligned frames to obtain an anchor trajectory.

[0077] In an optional embodiment, relocating the initial trajectory according to the multiple aligned frames to obtain the anchor trajectory includes the following steps:

[0078] Step S2071: Align each alignment frame with the reference image to obtain multiple relocalization trajectories.

[0079] Specifically, each aligned frame can be aligned with the prior point cloud map to obtain multiple relocalization trajectories.

[0080] Step S2072: Determine the weight value of each relocalization trajectory.

[0081] Specifically, determining the weight value of each relocalization trajectory includes: dividing the initial trajectory into multiple line segments; obtaining multiple alignment frames belonging to the current line segment, selecting the alignment frame with the highest reliability value from the multiple alignment frames belonging to the current line segment, and using the selected alignment frame as the anchor frame of the current line segment, and setting the weight value of the relocalization trajectory corresponding to the anchor frame to 1; and determining the weights of the relocalization trajectories corresponding to the other alignment frames based on the reliability values ​​of the other alignment frames except the anchor frame.

[0082] That is to say, according to the length of the initial trajectory, the initial trajectory is segmented, and the alignment frame with the highest reliability value in each segment is set as the anchor frame. During the optimization process, the weight of the anchor frame is 1, and the posture of the anchor frame will be fixed. The postures of the remaining alignment frames except the anchor frame are determined by the weights of the reliability values. Come to.

[0083] In this embodiment, the initial trajectory is divided into multiple line segments, so that the anchor frames are evenly distributed, and within each segment, there is a result with high anchoring quality to constrain the trajectory.

[0084] Step S2073: According to the weight values ​​of the multiple relocation trajectories, the multiple relocation trajectories are fused to obtain the anchor trajectory.

[0085] The visual relocalization method provided in this embodiment can not only construct a stable alignment frame by using multi-view common view constraints; but also segment the initial trajectory according to its length, and set the alignment frame with the highest reliability value in each segment as the anchor frame, so that the anchor frames are evenly distributed, and there is a result with high anchoring quality in each interval to constrain the trajectory; in the optimization process, the weight of the anchor frame is 1, and the posture of the anchor frame will be fixed. The postures of the remaining alignment frames except the anchor frame are determined by the weight of the reliability value. It is concluded that the factors affecting the failure of matching from local frames to reference images can be better reduced, and the stability of the relocalization system in the stage of acquiring anchor trajectories can be improved.

[0086] In this embodiment, a visual relocalization method is provided, which can be used in a computer device. Figure 3 FIG. 1 is a flow chart of another visual relocalization method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0087] Step S301: Obtain an initial trajectory and a reference image library.

[0088] Step S302: Determine a priori point cloud map based on the reference image.

[0089] Step S303: obtaining a current local frame corresponding to the current trajectory point in the initial trajectory, and aligning the current local frame with the prior point cloud map to obtain an alignment position; wherein the current local frame includes images collected by multiple cameras.

[0090] Step S304: back-projecting the prior point cloud map onto multiple captured images of the current local frame according to the alignment positions, and obtaining evaluation parameters corresponding to each alignment position.

[0091] Step S305: determining a reliability value of the current partial frame according to the evaluation parameters corresponding to each alignment position.

[0092] Step S306: When the reliability value of the current partial frame is greater than a preset first threshold, the current partial frame is used as the alignment frame.

[0093] Step S307: relocate the initial trajectory according to the multiple aligned frames to obtain an anchor trajectory.

[0094] Step S308: Obtain inter-frame constraint parameters, wherein the inter-frame constraint parameters include at least one of the following: gyroscope bias, accelerometer bias, gyroscope random walk noise, accelerometer random walk noise, and vehicle body speed.

[0095] In the visual relocalization method, after obtaining the anchor track corresponding to each current local frame, it is necessary to optimize the multiple anchor tracks corresponding to the initial track. In this embodiment, the vehicle body is imaged by multiple cameras in each frame. Therefore, during the matching stage, in addition to matching the shared field of view between different cameras within a local frame, the shared field of view between local frames is also matched. The inter-frame matching method adopts a hybrid matching method of feature and optical flow: 1) For all images, feature points and feature descriptors are used for matching. 2) In the inter-frame matching, the optical flow method is added between all consecutive images to further add feature points with good photometric consistency to provide more pose constraints.

[0096] In addition, in the local inter-frame constraints, the inter-frame motion state is estimated by loosely coupling the IMU and the wheel speed meter. Therefore, it is necessary to obtain the inter-frame constraint parameters and the vehicle velocity. The inter-frame constraint parameters include at least one of the following: vehicle angular velocity and vehicle acceleration.

[0097] Step S309: Calculate the pre-integration term according to the inter-frame constraint parameters.

[0098] Among them, the pre-integration term is the integral quantity related to the inter-frame motion state, and this integral quantity can participate in the subsequent posture optimization.

[0099] Specifically, the pre-integration term is calculated according to the inter-frame constraint parameters using the following formula:

[0100]

[0101]

[0102]

[0103] in, 、 、 are all pre-integrated terms, and are the biases of the gyroscope and accelerometer, and are the random walk noise of the gyroscope and accelerometer, represents the angular velocity measurement value of the gyroscope, represents the integration interval, represents the acceleration measurement value of the accelerometer, represents the rotation matrix from time k to time i. Specifically, the vehicle speed data obtained by the wheel speed meter can be converted to the vehicle speed in the local coordinate system through coordinate system transformation, and this speed is used as the speed prior constraint.

[0104] Then, the anchor trajectory is used to triangulate the feature points matched between local frames to generate a local point cloud map. Both the prior point cloud map and the local point cloud map are back-projected into the image in the local frame, with the reprojection error as the constraint. The reprojection error is calculated as follows:

[0105]

[0106] in is the image coordinate of the image feature point, K is the camera intrinsic parameter matrix, T is the camera extrinsic parameter matrix, p is the three-dimensional map point, represents the reprojection error; n represents the number of image feature points to be reprojected. When using multiple cameras (for example, seven cameras), the intrinsic and extrinsic parameter matrices of each camera differ due to their different installation locations, models, and usage status. In the subsequent factor graph optimization process, the intrinsic and extrinsic parameters of all cameras can be jointly optimized as optimization variables. This process is also known as self-calibration. By combining feature point triangulation, reprojection error modeling, and joint optimization of multi-camera parameters, the system achieves more accurate positioning and mapping.

[0107] Step S310: Optimize multiple anchor trajectories corresponding to the initial trajectory using a nonlinear least squares method according to the pre-integral term and the vehicle body velocity to obtain a repositioning trajectory.

[0108] For example, Figure 4 The multi-sensor data fusion factor graph constructed by all constraint factors is presented. Based on this factor graph, all trajectories are tightly coupled optimized with multi-sensor data fusion to obtain high-precision relocalization trajectories. Figure 4The multi-sensor data fusion factor graph in [1] includes anchor trajectory constraints, landmark point reprojection error constraints (also called reprojection error), wheel speed prior constraints, and IMU pre-integration constraints. Each circle vertex represents the alignment position of the local frame to be optimized, and each triangle vertex corresponds to the prior point cloud map and the local point cloud map. After obtaining a high-precision relocalization trajectory, the high-precision map near the high-precision relocalization trajectory is projected onto the images captured by all onboard cameras. It can be seen that the high-precision map and the actual lane features in the image are well aligned.

[0109] Step S311: Obtain a vector map from a reference image library.

[0110] Step S312: Optimize multiple anchor trajectories corresponding to the initial trajectory using the Levenberg-Marquardt method according to the vector map to obtain a relocalization trajectory.

[0111] In other words, to obtain pixel-level alignment accuracy, the vector map in the prior map is back-projected into the camera image, and the trajectory is relocated with high precision. Based on the results of semantic segmentation of lane lines, road signs, etc., the vector map is matched with the semantic pixels of the image, and pixel-level optimization is performed to ultimately achieve pixel-level relocation accuracy.

[0112] Step S313: Obtain the jitter degree parameter of the anchoring trajectory, and determine the evaluation result of the anchoring trajectory according to the jitter degree parameter.

[0113] This is because the real image acquisition trajectory should be geometrically smooth. If the trajectory has a lot of sharp jitters, it means that the positioning effect of the trajectory is unstable. On the contrary, it means that the continuity of the trajectory positioning is high. Figure 5 As shown in the figure, the convex hull of the trajectory can be detected, and a set of baselines can be obtained by smoothing the trajectory. If a trajectory point deviates from the baseline by a certain distance, it will be considered as an abnormally jittery trajectory point, as shown in red; and the trajectory point within the distance range is considered to be a normal trajectory point, as shown in blue.

[0114] The above step S312 may also be replaced by the following steps: back-projecting the vector map onto the captured image in the current local frame to obtain the degree of overlap between the vector map and the captured image; and determining the evaluation result of the anchoring trajectory according to the degree of overlap.

[0115] like Figure 6 As shown, the degree of overlap between the vector map and the actual scene in the pixel plane coordinate system is calculated. If the overlap between the vector map and the actual scene image is low, it indicates low trajectory positioning accuracy; otherwise, it indicates high positioning accuracy and is consistent with the actual real world. The high-precision map back-projection semantic image score evaluates the anchor trajectory from a semantic perspective. Figure 6The various rectangular boxes in the figure represent the back projection of the high-precision map elements on the current image, while the solid arrows, road lines, etc. in the image are the results of semantic segmentation of the image. The evaluation method is to calculate the overlap between the two for evaluation.

[0116] The visual relocalization method provided in this embodiment not only utilizes multi-view common view constraints to construct a stable alignment frame; it also constructs a multi-sensor data fusion factor graph based on multi-sensor data to further optimize the accuracy of the anchor trajectory; it back-projects the vector map onto the camera image, and based on the results of image semantic segmentation, it further aligns the camera pose to the vector map projection to obtain pixel-level alignment accuracy; further, from the perspective of trajectory geometry and the consistency between the high-precision map and the actual scene, it calculates the degree of trajectory jitter and the overlap index of the semantic image back-projected from the high-precision map to perform a fully automatic quantitative evaluation of the obtained relocalization results.

[0117] In summary, the visual relocalization method provided in this embodiment has the following beneficial effects:

[0118] (1) The strategy of constructing anchor frames with multi-view common-view constraints can be added as an independent module to the current mainstream multi-camera or multi-view relocalization system.

[0119] (2) Compared with the relocalization system that does not construct anchor frames through multi-view common view constraints, it can better reduce the impact of failure in matching local frames to reference images and improve the stability of the relocalization system in the stage of acquiring anchor trajectories.

[0120] (3) Multi-sensor data and high-precision map back-projection provide more positioning constraints for the relocalization system. Compared with the relocalization system that does not use multi-sensor data and high-precision map back-projection constraints for joint optimization, this system can achieve higher positioning accuracy.

[0121] (4) The smoothness of the trajectory and the degree of conformity to the semantic image of the high-precision map back projection are used to score, and a fully automatic evaluation quantitative system is constructed to automatically evaluate the relocation trajectory.

[0122] In this embodiment, a visual repositioning device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. The details that have been described will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation by hardware, or a combination of software and hardware, is also possible and conceivable.

[0123] This embodiment provides a visual repositioning device, such as Figure 7 Shown, including:

[0124] The first acquisition module 701 is used to acquire an initial trajectory and a reference image library.

[0125] The pre-processing module 702 is used to determine a priori point cloud map according to the reference image.

[0126] The alignment module 703 is used to obtain a current local frame corresponding to the current trajectory point in the initial trajectory, and align the current local frame with the prior point cloud map to obtain an alignment position; wherein the current local frame includes images collected by multiple cameras.

[0127] The second acquisition module 704 is used to back-project the prior point cloud map onto multiple acquired images of the current local frame according to the alignment positions, and obtain evaluation parameters corresponding to each alignment position.

[0128] The reliability value calculation module 705 is used to determine the reliability value of the current local frame according to the evaluation parameters corresponding to each alignment position.

[0129] The alignment frame determination module 706 is configured to use the current partial frame as the alignment frame when the reliability value of the current partial frame is greater than a preset first threshold.

[0130] The anchor trajectory determination module 707 is configured to reposition the initial trajectory according to the multiple aligned frames to obtain the anchor trajectory.

[0131] In some optional embodiments, the evaluation parameters corresponding to each alignment position include at least one of the following: the number of acquired images in all acquired images of the current local frame that meet the reprojection error constraint, the number of points in all acquired images of the current local frame that meet the reprojection error constraint, and the average reprojection error of all acquired images of the current local frame.

[0132] In some optional embodiments, the reliability value calculation module 705 is specifically used to: calculate the ratio of the number of captured images corresponding to each alignment position that meet the reprojection error constraint to the total number of captured images in the current local frame to obtain a first ratio; calculate the ratio of the number of points corresponding to each alignment position that meet the reprojection error constraint to the total number of matching points of all captured images in the current local frame to obtain a second ratio; calculate the ratio of the average reprojection error corresponding to each alignment position to a preset reprojection error limit to obtain a third ratio; calculate the reliability value corresponding to each alignment position based on the first ratio, the second ratio and the third ratio; and determine the reliability value of the current local frame based on the reliability value corresponding to each alignment position.

[0133] In some optional embodiments, the anchor trajectory determination module 707 includes a relocalization trajectory determination unit, a weight determination unit, and a fusion unit. The relocalization trajectory determination unit is configured to align each alignment frame with the reference image to obtain multiple relocalization trajectories; the weight determination unit is configured to determine a weight value for each relocalization trajectory; and the fusion unit is configured to fuse the multiple relocalization trajectories according to their weight values ​​to obtain the anchor trajectory.

[0134] In some optional embodiments, the weight determination unit is specifically used to: divide the initial trajectory into multiple line segments; obtain multiple alignment frames belonging to the current line segment, select an alignment frame with the highest reliability value from the multiple alignment frames belonging to the current line segment, and use the selected alignment frame as the anchor frame of the current line segment, and set the weight value of the relocation trajectory corresponding to the anchor frame to 1; determine the weight of the relocation trajectory corresponding to the other alignment frames according to the reliability values ​​of the other alignment frames except the anchor frame.

[0135] In some optional embodiments, the alignment module 703 is specifically used to: determine the 2D-2D matching relationship between the captured image of the current local frame and the reference image; construct a 2D-3D correspondence between the captured image of the current local frame and the prior point cloud map based on the 2D-2D matching relationship and the prior point cloud map; and calculate the alignment position of the current local frame in the prior point cloud map based on the 2D-3D correspondence between the captured image of the current local frame and the prior point cloud map.

[0136] In some optional embodiments, the visual relocalization device further includes a local frame preprocessing module. Before aligning the current local frame with the prior point cloud map to obtain the alignment position, the local frame preprocessing module is used to extract global descriptors from all acquired images in the current local frame to obtain a first result; extract global descriptors from all reference images in the reference image library to obtain a second result; based on the first result and the second result, determine whether the number of reference images corresponding to the current acquired image in the current local frame in the reference image library is less than a preset second threshold; when the number of reference images corresponding to the current acquired image in the reference image library is less than the second threshold, remove the current acquired image from the current local frame.

[0137] In some optional embodiments, the visual relocalization apparatus further includes a first optimization module. After obtaining the anchor trajectory, the first optimization module is specifically configured to: obtain inter-frame constraint parameters and vehicle velocity, wherein the inter-frame constraint parameters include at least one of the following: vehicle angular velocity and vehicle acceleration; calculate a pre-integral term based on the inter-frame constraint parameters; and optimize multiple anchor trajectories corresponding to the initial trajectory using a nonlinear least squares method based on the pre-integral term and the vehicle velocity to obtain a relocalization trajectory.

[0138] In some optional embodiments, the visual relocalization apparatus further includes a second optimization module. After obtaining the anchor trajectory, the second optimization module is specifically configured to: obtain a vector map from a reference image library after obtaining the anchor trajectory; and optimize multiple anchor trajectories corresponding to the initial trajectory using the Levenberg-Marquardt method based on the vector map to obtain a relocalization trajectory.

[0139] In some optional embodiments, the visual relocalization apparatus further includes an evaluation module. After obtaining the anchor track, the evaluation module is specifically configured to: obtain a jitter parameter of the anchor track; determine an evaluation result of the anchor track based on the jitter parameter; and / or back-project the vector map onto the captured image in the current local frame to obtain a degree of overlap between the vector map and the captured image; and determine an evaluation result of the anchor track based on the degree of overlap.

[0140] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0141] The visual repositioning device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0142] The embodiment of the present invention also provides a computer device having the above Figure 7 The visual repositioning device shown.

[0143] See also Figure 8 , Figure 8 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0144] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0145] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0146] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0148] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 8 The bus connection is taken as an example.

[0149] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device. Examples include a touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, etc. The output device 40 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). Such display devices include, but are not limited to, liquid crystal displays, light emitting diodes, monitors, and plasma displays. In some optional embodiments, the display device may be a touch screen.

[0150] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0151] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0152] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A visual relocalization method, characterized in that: include: Acquire an initial trajectory and a reference image library, wherein the reference image library stores reference images; Determine a priori point cloud map based on the reference image; Acquire a current local frame corresponding to a current trajectory point in the initial trajectory, and align the current local frame with the prior point cloud map to obtain an alignment position; wherein the current local frame includes images captured by multiple cameras; Back-projecting the prior point cloud map onto the multiple acquired images of the current local frame according to the alignment positions, and obtaining an evaluation parameter corresponding to each alignment position; determining a reliability value of the current partial frame according to an evaluation parameter corresponding to each of the alignment positions; When the reliability value of the current partial frame is greater than a preset first threshold, taking the current partial frame as an alignment frame; relocating the initial trajectory according to the plurality of aligned frames to obtain an anchor trajectory; The evaluation parameter corresponding to each alignment position includes at least one of the following: the number of acquired images in all acquired images of the current partial frame that meet the reprojection error constraint, the number of points in all acquired images of the current partial frame that meet the reprojection error constraint, and the average reprojection error of all acquired images of the current partial frame; The back-projecting the prior point cloud map onto the multiple acquired images of the current partial frame according to the alignment position to obtain the evaluation parameter corresponding to each alignment position includes: back-projecting the prior point cloud map onto the multiple acquired images of the current partial frame according to the alignment position and the 2D-3D relationship to obtain the evaluation parameter corresponding to each alignment position; Relocating the initial trajectory according to the plurality of aligned frames to obtain an anchor trajectory includes: aligning each of the alignment frames with the reference image to obtain a plurality of relocalization trajectories; determining a weight value for each of the relocation trajectories; fusing the plurality of relocation trajectories according to their weight values ​​to obtain the anchoring trajectory; Determining the weight value of each relocation trajectory includes: Dividing the initial trajectory into a plurality of line segments; Acquire multiple alignment frames belonging to the current line segment, select the alignment frame with the highest reliability value from the multiple alignment frames belonging to the current line segment, use the selected alignment frame as the anchor frame of the current line segment, and set the weight value of the relocalization trajectory corresponding to the anchor frame to 1; According to the reliability values ​​of other aligned frames except the anchor frame, weights of relocalization tracks corresponding to the other aligned frames are determined.

2. The method according to claim 1, characterized in that Determining the reliability value of the current partial frame according to the evaluation parameter corresponding to each alignment position includes: Calculating a ratio of the number of collected images satisfying the reprojection error constraint corresponding to each alignment position to the total number of collected images in the current partial frame to obtain a first ratio; Calculating a ratio of the number of points satisfying the reprojection error constraint corresponding to each alignment position to the total number of matching points of all captured images in the current partial frame to obtain a second ratio; Calculating a ratio of the average reprojection error corresponding to each alignment position to a preset reprojection error tolerance to obtain a third ratio; calculating a reliability value corresponding to each of the alignment positions according to the first ratio, the second ratio, and the third ratio; The reliability value of the current partial frame is determined according to the reliability value corresponding to each of the aligned positions.

3. The method according to claim 1, characterized in that The aligning the current local frame with the prior point cloud map to obtain an aligned position includes: Determining a 2D-2D matching relationship between the captured image of the current partial frame and the reference image; Constructing a 2D-3D correspondence between the captured image of the current local frame and the prior point cloud map according to the 2D-2D matching relationship and the prior point cloud map; The alignment position of the current partial frame in the prior point cloud map is calculated according to the 2D-3D correspondence between the acquired image of the current partial frame and the prior point cloud map.

4. The method according to claim 1, wherein Before aligning the current local frame with the prior point cloud map to obtain an aligned position, the method further includes: Extracting global descriptors from all collected images in the current local frame to obtain a first result; Extracting global descriptors from all reference images in the reference image library to obtain a second result; Determining, based on the first result and the second result, whether the number of reference images corresponding to the current captured image in the current partial frame in the reference image library is less than a preset second threshold; When the number of reference images corresponding to the current collected image in the reference image library is less than the second threshold, the current collected image is eliminated from the current partial frame.

5. The method according to claim 1, wherein Also includes: After obtaining the anchor trajectory, obtaining an inter-frame constraint parameter and a vehicle velocity, wherein the inter-frame constraint parameter includes at least one of the following: a vehicle angular velocity and a vehicle acceleration; Calculating a pre-integration term according to the inter-frame constraint parameter; A plurality of anchor trajectories corresponding to the initial trajectory are optimized using a nonlinear least squares method according to the pre-integral term and the vehicle body velocity to obtain a repositioning trajectory.

6. The method according to claim 1, characterized in that Also includes: After obtaining the anchor trajectory, obtaining a vector map in a reference image library; A plurality of anchor trajectories corresponding to the initial trajectory are optimized according to the vector map using the Levenberg-Marquardt method to obtain a relocalization trajectory.

7. The method according to claim 1, characterized in that Also includes: After obtaining the anchor track, obtaining a jitter degree parameter of the anchor track; Determining an evaluation result of the anchoring trajectory according to the jitter degree parameter; and / or; After obtaining the anchoring trajectory, the vector map is back-projected onto the acquired image in the current local frame to obtain a degree of overlap between the vector map and the acquired image; and an evaluation result of the anchoring trajectory is determined according to the degree of overlap.

8. A visual repositioning device, characterized in that: The device comprises: A first acquisition module is used to obtain an initial trajectory and a reference image library; A preprocessing module, configured to determine a priori point cloud map based on the reference image; an alignment module, configured to obtain a current local frame corresponding to a current trajectory point in the initial trajectory, and align the current local frame with the prior point cloud map to obtain an alignment position; wherein the current local frame includes images captured by multiple cameras; A second acquisition module is configured to back-project the prior point cloud map into the multiple acquired images of the current local frame according to the alignment position, and acquire an evaluation parameter corresponding to each of the alignment positions; the evaluation parameter corresponding to each of the alignment positions includes at least one of the following: the number of acquired images that satisfy the reprojection error constraint in all acquired images of the current local frame, the number of points that satisfy the reprojection error constraint in all acquired images of the current local frame, and the average reprojection error of all acquired images of the current local frame; back-projecting the prior point cloud map into the multiple acquired images of the current local frame according to the alignment position, and acquiring the evaluation parameter corresponding to each of the alignment positions includes: back-projecting the prior point cloud map into the multiple acquired images of the current local frame according to the alignment position and the 2D-3D relationship, and acquiring the evaluation parameter corresponding to each of the alignment positions; A reliability value calculation module, configured to calculate the reliability value of the current local frame according to the evaluation parameters; an alignment frame determining module, configured to use the current partial frame as an alignment frame when the reliability value of the current partial frame is greater than a preset first threshold; An anchor trajectory determination module is configured to relocate the initial trajectory according to the plurality of aligned frames to obtain an anchor trajectory; relocating the initial trajectory according to the plurality of aligned frames to obtain the anchor trajectory comprises: aligning each of the aligned frames with the reference image to obtain a plurality of relocated trajectories; determining a weight value for each of the relocated trajectories; fusing the plurality of relocated trajectories according to the weight values ​​of the plurality of relocated trajectories to obtain the anchor trajectory; determining the weight value for each of the relocated trajectories comprises: dividing the initial trajectory into a plurality of line segments; obtaining a plurality of aligned frames belonging to the current line segment, selecting an aligned frame with the highest reliability value from the plurality of aligned frames belonging to the current line segment, using the selected aligned frame as the anchor frame of the current line segment, and setting the weight value of the relocated trajectory corresponding to the anchor frame to 1; and determining the weight of the relocated trajectory corresponding to the other aligned frames according to the reliability values ​​of the other aligned frames except the anchor frame.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the visual repositioning method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the visual relocalization method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the visual relocalization method according to any one of claims 1 to 7.

Images