Visual positioning evaluation method and device

Abstract

The application discloses a visual positioning evaluation method and device, and the method comprises the following steps: constructing a first model of a first scene based on an image of the first scene and a visual angle corresponding to the image, wherein the first model outputs a rendering image corresponding to any visual angle in the first scene when the first model inputs the visual angle; inputting a first pose in the first model and obtaining a first rendering image corresponding to the first pose; processing the first rendering image based on a to-be-evaluated algorithm to obtain a second pose, wherein the first pose input in the first model and the second pose obtained by the to-be-evaluated algorithm are realized based on the same coordinate system; and determining an evaluation result of the to-be-evaluated algorithm based on the first pose corresponding to the first rendering image and the second pose.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and more specifically, to a visual positioning evaluation method and apparatus. Background Technology

[0002] Visual positioning technology is widely used in fields such as AR and robotics. The accuracy of visual positioning methods is also inextricably linked to user experience. Current evaluations of visual positioning methods require multiple simultaneous acquisitions of scene data from various sensors, such as radar, visual cameras, depth cameras, and Bluetooth. Subsequent data fusion and pose registration techniques from these sensors are used to obtain a globally unified pose as the ground truth pose of the RGB image, which is then used to evaluate various visual positioning technologies. However, the aforementioned evaluation schemes for visual positioning methods involve significant hardware and computational costs and contain unavoidable errors. Summary of the Invention

[0003] In view of the above, this application provides the following technical solution:

[0004] A visual positioning evaluation method, comprising:

[0005] A first model of the first scene is constructed based on the image of the first scene and the viewpoint corresponding to the image. The first model will output the rendered image corresponding to any viewpoint in the first scene when any viewpoint is input.

[0006] Input the first pose into the first model and obtain the first rendered image corresponding to the first pose;

[0007] The first rendered image is processed based on the algorithm to be evaluated to obtain the second pose. The first pose input by the first model and the second pose obtained by the algorithm to be evaluated are implemented based on the same coordinate system.

[0008] The evaluation result of the algorithm to be evaluated is determined based on the first pose and the second pose corresponding to the first rendered image.

[0009] Optionally, the first pose may include multiple poses, and the step of inputting the first pose into the first model and obtaining the first rendered image corresponding to the first pose includes:

[0010] The three-dimensional space corresponding to the first model is divided into multiple grid regions;

[0011] The control inputs at least one random first pose from a random viewpoint to each grid area.

[0012] Optionally, after inputting the first pose into the first model and obtaining the first rendered image corresponding to the first pose, the method further includes:

[0013] The input parameters of the first model are adjusted. When the first model is given the first pose, the output rendering image is different from the first rendering image.

[0014] A random first pose is input into the first model after the input parameters are adjusted to obtain a second rendered image;

[0015] The process of processing the first rendered image based on the algorithm to be evaluated to obtain the second pose includes:

[0016] The first and second rendered images are processed based on the algorithm to be evaluated to obtain two corresponding second poses.

[0017] Optionally, the first pose may include multiple poses, resulting in multiple first poses, including:

[0018] Interpolation is performed between the two first poses input in the first model to obtain a first pose sequence that conforms to the first trajectory. The first pose sequence corresponds to a continuous camera movement.

[0019] Optionally, the step of processing the first rendered image based on the algorithm to be evaluated to obtain the second pose includes:

[0020] Construct a coordinate map corresponding to the coordinate system of the first model;

[0021] The first rendered image is processed based on the algorithm to be evaluated under the coordinate map to obtain the second pose.

[0022] Optionally, constructing a coordinate map corresponding to the first model coordinate system includes:

[0023] The coordinate data of the first model is obtained by analyzing the first part of the data composed of multiple first poses of the first model and the corresponding first rendered image.

[0024] Based on the coordinate data, a coordinate map corresponding to the coordinate system of the first model is constructed.

[0025] Optionally, the first model further performs the operation of generating point cloud data based on the first pose, wherein constructing a coordinate map corresponding to the coordinate system of the first model includes:

[0026] Based on the point cloud data, a coordinate map corresponding to the coordinate system of the first model is constructed.

[0027] Optionally, the first model further performs the operation of obtaining a depth rendering image corresponding to the first pose, wherein constructing a coordinate map corresponding to the coordinate system of the first model includes:

[0028] A coordinate map corresponding to the first model coordinate system is constructed based on the first rendered image and the depth rendered image.

[0029] Optionally, inputting the first pose into the first model and obtaining the first rendered image corresponding to the first pose includes:

[0030] Input the first pose in the first model;

[0031] Based on the obtained binocular camera parameters of the algorithm to be evaluated and the first pose, the corresponding binocular pose is obtained, and the binocular pose includes two poses.

[0032] Two corresponding rendered images are obtained based on the binocular pose and used as the first rendered image.

[0033] Furthermore, this application also discloses a visual positioning evaluation device, including: a model building module, used to build a first model of the first scene based on an image of the first scene and the viewpoint corresponding to the image, wherein the first model outputs a rendered image corresponding to any viewpoint in the first scene when any viewpoint is input;

[0034] The rendering image acquisition module is used to input the first pose into the first model and obtain the first rendering image corresponding to the first pose;

[0035] The pose estimation module is used to process the first rendered image based on the algorithm to be evaluated to obtain a second pose, wherein the first pose input by the first model and the second pose obtained by the algorithm to be evaluated are implemented based on the same coordinate system; the algorithm evaluation module is used to determine the evaluation result of the algorithm to be evaluated based on the first pose and the second pose corresponding to the first rendered image. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0037] Figure 1 This is a flowchart of a visual positioning evaluation method disclosed in an embodiment of this application;

[0038] Figure 2 A flowchart of another visual positioning evaluation algorithm disclosed in an embodiment of this application;

[0039] Figure 3 This is a flowchart of the process for obtaining the second pose disclosed in the embodiments of this application;

[0040] Figure 4 This is a flowchart of obtaining the first rendered image disclosed in an embodiment of this application;

[0041] Figure 5 This is a schematic diagram illustrating the implementation process of a visual positioning evaluation method disclosed in an embodiment of this application;

[0042] Figure 6 This is a schematic diagram of the structure of a visual positioning evaluation device disclosed in an embodiment of this application;

[0043] Figure 7 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Figure 1 This is a flowchart illustrating a visual positioning evaluation method disclosed in an embodiment of this application. See also... Figure 1 As shown, visual positioning evaluation methods may include:

[0046] Step 101: Construct a first model of the first scene based on the image of the first scene and the viewpoint corresponding to the image. The first model will output the rendered image corresponding to any viewpoint in the first scene when any viewpoint is input.

[0047] The first scenario is not specific and can be any scenario in the actual environment. Of course, when combined with the application of visual positioning technology, a scenario that is similar to the application scenario of visual positioning technology can be selected. For example, if the scenario of visual positioning should be a CS game scenario, then some factory buildings can be selected as the first scenario.

[0048] The first scene comprises multiple images, ensuring higher accuracy of the constructed first model. When acquiring each image of the first scene, the corresponding pose—that is, the position and angle of the captured image—is known. Based on these images with known poses, a first model of the first scene can be constructed. The first model can be either an implicit or explicit model; this application does not impose any fixed limitations on it.

[0049] Step 102: Input the first pose into the first model and obtain the first rendered image corresponding to the first pose.

[0050] After the first model is constructed, its input is a pose, and its output is a rendered image corresponding to the input pose. The first pose can be any randomly determined pose. After inputting the first pose into the first model, the first model will obtain the corresponding first rendered image based on the first pose. The first rendered image will be used as the test image for the algorithm to be evaluated in this application.

[0051] Step 103: Process the first rendered image based on the algorithm to be evaluated to obtain the second pose. The first pose input by the first model and the second pose obtained by the algorithm to be evaluated are implemented based on the same coordinate system.

[0052] After obtaining the first rendered image, the algorithm to be evaluated will be controlled to perform pose estimation based on the first rendered image, which is the process of inferring the pose based on the rendered image. The inferred pose is referred to as the second pose in this application.

[0053] The first pose input to the first model and the second pose obtained by the algorithm to be evaluated are implemented based on the same coordinate system, that is, they are determined on the same coordinate map. It is understandable that for the same point, its coordinate values ​​will be different in different coordinate systems. If this application wants to compare the first pose of the first model and the second pose obtained by the algorithm to be evaluated, it also needs to ensure that they have the same coordinate map, that is, they are implemented based on the same coordinate system.

[0054] Step 104: Determine the evaluation result of the algorithm to be evaluated based on the first pose and the second pose corresponding to the first rendered image.

[0055] Since the first pose is a specified pose input into the model and is not obtained based on any data processing, its accuracy can be considered 100%. The second pose, estimated by the algorithm under evaluation based on the first rendered image, is an estimation. Therefore, the second pose estimated by the algorithm under evaluation can be compared with the first pose, which serves as a standard value, to determine the accuracy of the second pose. Understandably, the closer the second pose is to the first pose, the higher the accuracy of the algorithm under evaluation; conversely, the greater the deviation between the second and first poses, the lower the accuracy of the algorithm under evaluation.

[0056] The visual positioning evaluation scheme described in this embodiment only requires an image acquisition device on the physical device, and it constructs a first model based on a first scene. The relevant data of the standard pose is not obtained through acquisition, but can be arbitrarily specified in the first model. Therefore, the accuracy of the standard pose can be guaranteed, and efficient and accurate visual positioning evaluation can be achieved.

[0057] In the above embodiments, the first pose may include multiple poses, and the step of inputting the first pose into the first model and obtaining the first rendered image corresponding to the first pose may include: dividing the three-dimensional space corresponding to the first model into multiple grid areas; and controlling the input of at least one random viewpoint first pose to each grid area.

[0058] This implementation involves uniformly inputting multiple first poses within the virtual space of the first model corresponding to the first scene. Specifically, the 3D space corresponding to the first model is first divided into multiple grid regions on an even scale. Then, at least one first pose is input into each grid region. This ensures that a first pose exists in every region of the 3D space corresponding to the first model, achieving full coverage of the first pose within the first model. Each grid region can contain one or multiple first poses, depending on the requirements of the application scenario.

[0059] Of course, in practice, if it is necessary to input multiple first poses into the first model, it is not necessary to divide the three-dimensional space of the first model into meshes and then input the first poses into each mesh. It can also be achieved in other ways, such as randomly inputting multiple first poses at any position in the first model.

[0060] Figure 2 This is a flowchart of another visual positioning evaluation algorithm disclosed in an embodiment of this application. (In conjunction with...) Figure 2 As shown, the visual positioning evaluation algorithm may include:

[0061] Step 201: Construct a first model of the first scene based on the image of the first scene and the viewpoint corresponding to the image.

[0062] The first model outputs a rendered image corresponding to any viewpoint in the first scene.

[0063] Step 202: Input the first pose into the first model and obtain the first rendered image corresponding to the first pose.

[0064] Step 203: Adjust the input parameters of the first model. After adjusting the input parameters, the first model outputs a different rendering image than the first rendering image when the first pose is input.

[0065] The input parameters are used to change the environmental parameters of the first model, such as illumination, and can also change the presence or absence of a certain object in the first model. For example, if the first scene is at noon and the ambient light intensity is high, and if the first scene is at dusk and the ambient light intensity is low, the brightness of the first rendered image at noon will be greater than that of the first rendered image at dusk for the same first pose.

[0066] Step 204: Input a random first pose into the first model after adjusting the input parameters to obtain the second rendered image.

[0067] After adjusting the first model, a first pose can be randomly input into the first model, and the first model renders a second rendered image based on this input first pose. That is, the first rendered image is obtained before the input parameters of the first model are adjusted, and the second rendered image is obtained after the input parameters of the first model are adjusted. The first poses corresponding to the first rendered image and the second rendered image can be the same or different.

[0068] Step 205: Process the first and second rendered images based on the algorithm to be evaluated to obtain two corresponding second poses.

[0069] Step 206: Determine the evaluation result of the algorithm to be evaluated based on the first pose corresponding to the first rendered image and the two second poses.

[0070] After obtaining the first and second rendered images, the algorithm to be evaluated can estimate the two corresponding second poses based on the two rendered images, compare the two second poses with the corresponding first poses, and comprehensively determine the evaluation result of the algorithm to be evaluated.

[0071] In this embodiment, the first model can not only render the corresponding rendered image based on the first pose, but also adjust the input parameters to simulate different environmental conditions of the first scene in the first model, making the rendered image closer to the real environment, which helps to evaluate the evaluation algorithm more accurately.

[0072] In another optional implementation, multiple first poses can also be input into the first model. Obtaining multiple first poses may include: performing interpolation between two first poses input into the first model to obtain a sequence of first poses that conforms to a first trajectory, wherein the first pose sequence corresponds to a continuous camera movement.

[0073] Understandably, interpolation between different first poses results in a smooth and continuous sequence of first poses. This sequence conforms to a first trajectory that corresponds to a scenario where the user is using a handheld image acquisition device to capture images while the device is moving. Therefore, evaluating the algorithm based on this first pose sequence more closely resembles a real-world mobile shooting scenario, facilitating a better assessment of the algorithm's performance in visual positioning processing for such scenarios (continuous movement scenes).

[0074] As previously mentioned, the first pose input to the first model and the second pose obtained by the algorithm to be evaluated are implemented based on the same coordinate system, that is, the first pose and the second pose are determined based on the same coordinate map. Since different scenes may correspond to different coordinate maps, in the implementation of this application, when the algorithm to be evaluated estimates the pose based on the rendered image, it is necessary to first clarify what coordinate map the rendering of the first rendered image from the first pose implemented by the first model is based on. Therefore, the process by which the algorithm to be evaluated processes the first rendered image to obtain the second pose can be found in [reference needed]. Figure 3 Combining Figure 3 The second pose is obtained by processing the first rendered image based on the algorithm to be evaluated, which may include:

[0075] Step 301: Construct a coordinate map corresponding to the coordinate system of the first model.

[0076] In an optional implementation, constructing a coordinate map corresponding to the first model coordinate system includes: analyzing a first portion of data composed of multiple first poses of the first model and corresponding first rendered images to obtain coordinate data of the first model; and constructing a coordinate map corresponding to the first model coordinate system based on the coordinate data.

[0077] In this implementation, it is necessary to deduce the coordinate system of the first pose based on the first pose of the first model and the corresponding first rendered image, that is, the coordinate system data corresponding to the first model. Then, based on the obtained coordinate system data, a coordinate map of the coordinate system of the first model is constructed. Subsequently, the pose of the first rendered image needs to be estimated in this coordinate map.

[0078] In another optional implementation, the first model further performs the operation of generating point cloud data based on the first pose, wherein constructing a coordinate map corresponding to the coordinate system of the first model includes: constructing a coordinate map corresponding to the coordinate system of the first model based on the point cloud data.

[0079] Since the pose of the first scene image is known, the corresponding 2D image 3D point cloud data can be obtained based on the first scene image and its pose. This 3D point cloud data itself contains coordinate information, so the subsequent evaluation algorithm can directly determine the coordinate map of the first model coordinate system based on this point cloud data.

[0080] In another optional implementation, the first model further performs the operation of obtaining the depth rendering image corresponding to the first pose, and the construction of the coordinate map corresponding to the first model coordinate system includes: constructing the coordinate map corresponding to the first model coordinate system based on the first rendering image and the depth rendering image.

[0081] In this implementation, after inputting the first pose to the first model, the first model will not only render a two-dimensional first rendered image, but also process and obtain a depth rendered image corresponding to the first pose. The depth rendered image includes depth information, and the subsequent evaluation algorithm can determine the coordinate map of the coordinate system corresponding to the first model based on the first rendered image and the depth rendered image.

[0082] The above content introduces several implementations for constructing a coordinate map corresponding to the first model coordinate system. However, this does not constitute a fixed limitation on the construction of the coordinate map. Those skilled in the art can construct a coordinate map of the first model coordinate system based on the alternatives and variations of the above implementations.

[0083] After constructing the coordinate map corresponding to the first model coordinate system, proceed to step 302.

[0084] Step 302: Process the first rendered image on the coordinate map based on the algorithm to be evaluated to obtain the second pose.

[0085] The evaluation algorithm estimates the second pose and the first pose based on the first rendered image under the coordinate map, and these are determined using the same coordinate map, thus making them comparable. The first pose and the second pose can be directly compared. If the first pose and the second pose are very close, it can be determined that the visual localization processing accuracy of the algorithm under evaluation is relatively high; otherwise, it indicates that its visual localization processing accuracy is poor.

[0086] Figure 4 This is a flowchart illustrating the process of obtaining a first rendered image as disclosed in an embodiment of this application. See also... Figure 4 As shown in the foregoing embodiment, the step of inputting the first pose into the first model and obtaining the first rendered image corresponding to the first pose includes:

[0087] Step 401: Input the first pose into the first model. The first pose can be a pose at any position and any angle.

[0088] Step 402: Based on the obtained binocular camera parameters of the algorithm to be evaluated and the first pose, obtain the corresponding binocular pose, which includes two poses.

[0089] In this implementation, the algorithm to be evaluated is a binocular vision localization processing algorithm. The first model can pre-obtain the binocular camera parameters of the algorithm to be evaluated, such as the distance between the binocular cameras. Then, after obtaining the first pose, based on the binocular camera parameters, the first pose is processed into a binocular pose. The binocular pose includes two poses, one of which corresponds to one camera. Here, the binocular camera can be compared to a user's two eyes. The distance between the two eyes is fixed, and the poses of the left and right eyes are different. The combination of the left and right eye poses corresponds to the binocular pose.

[0090] Step 403: Obtain two corresponding rendered images based on the binocular pose as the first rendered image.

[0091] Since binocular pose involves two poses, the first model will also render corresponding images for each pose, and the two rendered images together constitute the first rendered image. The algorithm to be evaluated is a binocular vision localization processing algorithm, so it can estimate the corresponding estimated binocular pose based on the two rendered images. Subsequently, the estimated binocular pose can be compared with the binocular pose obtained by the first model to determine the processing accuracy of the algorithm to be evaluated.

[0092] Figure 5 This is a schematic diagram illustrating the implementation process of a visual positioning evaluation method disclosed in an embodiment of this application.

[0093] Combination Figure 5 As shown, in a specific implementation, the solution may include:

[0094] 1. Select a first scene from a variety of typical scenes, and collect image data of the first scene, wherein the image data includes the image and the corresponding pose;

[0095] 2. Based on the acquired data, an implicit model of the first scene is established using the Nerf (Neural Radiance Fields) reconstruction method;

[0096] 3. Divide the Nerf space (the three-dimensional space of the first model). The Nerf space can be divided according to a given resolution to generate multiple sampleable grid points.

[0097] 4. Sample the available grid points, and randomly generate an arbitrary viewpoint for each sampling point, that is, randomly input an arbitrary viewpoint into each sampling point;

[0098] 5. Traverse the sampled grid points in a certain order, and interpolate the pose of each sampled point according to the traversal order to obtain the virtual camera pose sequence in Nerf space, which is also the first pose sequence.

[0099] 6. Then, the lighting in the nerf space can be adjusted for each virtual camera (corresponding to the first pose) to simulate the lighting changes in the real scene;

[0100] 7. For each virtual camera, add, delete, and modify objects in the nerf space to simulate changes in appearance in the real scene, such as adding an object in the nerf space.

[0101] 8. Based on the first pose, obtain the RGB image (corresponding to the first rendered image) using Nerf volume rendering technology;

[0102] 9. The evaluation dataset consists of multiple first poses and their corresponding RGB images;

[0103] 10. Evaluate the visual localization algorithm to be evaluated based on the evaluation dataset.

[0104] The visual positioning evaluation scheme described in this embodiment only requires the acquisition of RGB images in real-world scenarios, without the need for other expensive hardware devices (such as LiDAR), resulting in very low overall hardware costs. Moreover, since the true pose is directly determined by the input of the first model, it has absolute accuracy. In implementation, a virtual camera can be set from the Nerf space as needed to acquire poses, thereby ensuring pose diversity and meeting evaluation requirements. Furthermore, this scheme can quickly simulate changes in lighting and appearance in the first model, meeting multi-view requirements and reducing the overall evaluation time cost.

[0105] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0106] The methods described in the above-disclosed embodiments of this application are detailed in terms of the methods. The methods of this application can be implemented by various forms of apparatus. Therefore, this application also discloses an apparatus. Specific embodiments are given below for detailed description.

[0107] Figure 6 This is a schematic diagram of the structure of a visual positioning evaluation device disclosed in an embodiment of this application. See also... Figure 6 As shown, the visual positioning evaluation device 60 may include:

[0108] The model building module 601 is used to build a first model of the first scene based on the image of the first scene and the viewpoint corresponding to the image. The first model will output the rendered image corresponding to any viewpoint in the first scene when any viewpoint is input.

[0109] The rendering image acquisition module 602 is used to input the first pose into the first model and obtain the first rendering image corresponding to the first pose.

[0110] The pose estimation module 603 is used to process the first rendered image based on the algorithm to be evaluated to obtain a second pose. The first pose generated by the first model and the second pose obtained by the algorithm to be evaluated are implemented based on the same coordinate system.

[0111] The algorithm evaluation module 604 is used to determine the evaluation result of the algorithm to be evaluated based on the first pose and the second pose corresponding to the first rendered image.

[0112] The visual positioning evaluation scheme described in this embodiment only requires an image acquisition device on the physical device, and it constructs a first model based on a first scene. The relevant data of the standard pose is not obtained through acquisition, but can be arbitrarily specified in the first model. Therefore, the accuracy of the standard pose can be guaranteed, and efficient and accurate visual positioning evaluation can be achieved.

[0113] In one implementation, the first pose includes multiple poses, and the rendering module can be specifically used to: divide the three-dimensional space corresponding to the first model into multiple grid regions; and control the input of at least one random first pose from a random viewpoint to each grid region.

[0114] In one implementation, the apparatus may further include: a model processing module, configured to adjust the input parameters of the first model, wherein the first model after adjusting the input parameters outputs a rendered image that is different from the first rendered image when a first pose is input; and to input a random first pose into the first model after adjusting the input parameters to obtain a second rendered image.

[0115] In one implementation, the first pose includes multiple poses, and the rendering image acquisition module can be specifically used to: perform interpolation processing between two first poses input in the first model to obtain a first pose sequence that conforms to a first trajectory, wherein the first pose sequence corresponds to a continuous camera movement.

[0116] In one implementation, the pose estimation module includes: a map construction module for constructing a coordinate map corresponding to the first model coordinate system; and a pose estimation submodule for processing the first rendered image based on the algorithm to be evaluated under the coordinate map to obtain a second pose.

[0117] In one implementation, the map building module can be used to: analyze a first part of the data composed of multiple first poses of the first model and the corresponding first rendered image to obtain the coordinate data of the first model; and build a coordinate map corresponding to the coordinate system of the first model based on the coordinate data.

[0118] In one implementation, the first model also performs the operation of generating point cloud data based on the first pose, and the map building module can be specifically used to: build a coordinate map corresponding to the coordinate system of the first model based on the point cloud data.

[0119] In one implementation, the first model also performs the operation of obtaining the depth rendering image corresponding to the first pose. Specifically, the map building module can be used to: build a coordinate map corresponding to the coordinate system of the first model based on the first rendering image and the depth rendering image.

[0120] In one implementation, the rendering image acquisition module can be used to input a first pose into the first model; obtain a corresponding binocular pose based on the obtained binocular camera parameters of the algorithm to be evaluated and the first pose, wherein the binocular pose includes two poses; and obtain two corresponding rendering images based on the binocular pose as the first rendering image.

[0121] Any of the visual positioning evaluation devices described in the above embodiments includes a processor and a memory. The model building module, rendering map acquisition module, pose estimation module, algorithm evaluation module, etc. in the above embodiments are all stored as program modules in the memory, and the processor executes the above program modules stored in the memory to realize the corresponding functions.

[0122] The processor contains a kernel, which retrieves the corresponding program modules from memory. One or more kernels can be configured, and the processing of backtracking data can be achieved by adjusting kernel parameters.

[0123] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0124] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer, and contains software code. After being loaded and executed by the computer, the computer program can implement the steps shown in any embodiment of the above-described visual positioning evaluation method.

[0125] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by the computer, the computer program can implement the steps shown in any embodiment of the visual positioning evaluation method described above.

[0126] Furthermore, embodiments of this application provide an electronic device. Figure 7 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. See also... Figure 7As shown, the electronic device 70 includes at least one processor 701, at least one memory 702 connected to the processor, and a bus 703; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the above-mentioned visual positioning evaluation method.

[0127] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0128] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0129] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0130] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A visual positioning evaluation method, comprising: A first model of the first scene is constructed based on the image of the first scene and the viewpoint corresponding to the image. The first model will output the rendered image corresponding to any viewpoint in the first scene when any viewpoint is input. Input the first pose into the first model and obtain the first rendered image corresponding to the first pose; The first rendered image is processed based on the algorithm to be evaluated to obtain the second pose. The first pose input by the first model and the second pose obtained by the algorithm to be evaluated are implemented based on the same coordinate system. The evaluation result of the algorithm to be evaluated is determined based on the first pose and the second pose corresponding to the first rendered image.

2. The visual positioning evaluation method according to claim 1, wherein the first pose includes multiple poses, and the step of inputting the first pose into the first model and obtaining the first rendered image corresponding to the first pose includes: The three-dimensional space corresponding to the first model is divided into multiple grid regions; The control inputs at least one random first pose from a random viewpoint to each grid area.

3. The visual positioning evaluation method according to claim 1, after inputting the first pose into the first model and obtaining the first rendered image corresponding to the first pose, further includes: The input parameters of the first model are adjusted. When the first model is given the first pose, the output rendering image is different from the first rendering image. A random first pose is input into the first model after the input parameters are adjusted to obtain a second rendered image; The process of processing the first rendered image based on the algorithm to be evaluated to obtain the second pose includes: The first and second rendered images are processed based on the algorithm to be evaluated to obtain two corresponding second poses.

4. The visual positioning evaluation method according to claim 1, wherein the first pose includes multiple poses, and obtaining multiple first poses includes: Interpolation is performed between the two first poses input in the first model to obtain a first pose sequence that conforms to the first trajectory. The first pose sequence corresponds to a continuous camera movement.

5. The visual positioning evaluation method according to claim 1, wherein processing the first rendered image based on the algorithm to be evaluated to obtain the second pose includes: Construct a coordinate map corresponding to the coordinate system of the first model; The first rendered image is processed based on the algorithm to be evaluated under the coordinate map to obtain the second pose.

6. The visual positioning evaluation method according to claim 5, comprising constructing a coordinate map corresponding to the first model coordinate system, including: The coordinate data of the first model is obtained by analyzing the first part of the data composed of multiple first poses of the first model and the corresponding first rendered image. Based on the coordinate data, a coordinate map corresponding to the coordinate system of the first model is constructed.

7. The visual positioning evaluation method according to claim 5, wherein the first model further performs the operation of generating point cloud data based on the first pose, and the construction of a coordinate map corresponding to the coordinate system of the first model includes: Based on the point cloud data, a coordinate map corresponding to the coordinate system of the first model is constructed.

8. The visual positioning evaluation method according to claim 5, wherein the first model further performs the operation of obtaining a depth rendering image corresponding to the first pose, and the construction of a coordinate map corresponding to the coordinate system of the first model includes: A coordinate map corresponding to the first model coordinate system is constructed based on the first rendered image and the depth rendered image.

9. The visual positioning evaluation method according to claim 1, wherein a first pose is input into the first model and a first rendered image corresponding to the first pose is obtained, comprising: Input the first pose in the first model; Based on the obtained binocular camera parameters of the algorithm to be evaluated and the first pose, the corresponding binocular pose is obtained, and the binocular pose includes two poses. Two corresponding rendered images are obtained based on the binocular pose and used as the first rendered image.

10. A visual positioning evaluation device, comprising: The model building module is used to build a first model of the first scene based on the image of the first scene and the viewpoint corresponding to the image. The first model will output the rendered image corresponding to any viewpoint in the first scene when any viewpoint is input. The rendering image acquisition module is used to input the first pose into the first model and obtain the first rendering image corresponding to the first pose; The pose estimation module is used to process the first rendered image based on the algorithm to be evaluated to obtain the second pose. The first pose generated by the first model and the second pose obtained by the algorithm to be evaluated are implemented based on the same coordinate system. The algorithm evaluation module is used to determine the evaluation result of the algorithm to be evaluated based on the first pose and the second pose corresponding to the first rendered image.

Images