Self-localization device, self-localization method, and self-localization program

Abstract

To provide a self position estimation device, a self position estimation method, and a self position estimation program, capable of improving accuracy of self position estimation.SOLUTION: A self position estimation device includes a map information acquisition unit (112), a simulation semantic image generation unit (113), an actual semantic image generation unit (114), an optimization processing unit (115), and a self position calculation unit (116). The optimization processing unit (115) uses an evaluation function to calculate a coincidence degree between a simulation semantic image and an actual semantic image, executes optimization by positioning the simulation semantic image so as to increase the coincidence degree, and derives a relative attitude matrix when the simulation semantic image has been optimized. The evaluation function calculates the coincidence degree of when performing optimization processing, using weighting variables that are set in pixels included in the simulation semantic image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This disclosure relates to a self-localization device, a self-localization method, and a self-localization program. [Background technology]

[0002] For example, Non-Patent Document 1 discloses a method for self-localization using semantic segmentation as an image analysis technique. In this document, first, a simulated semantic image including depth information is generated by rendering on a 3D model to which semantic labels have been assigned, using an inaccurate initial position obtained from GPS or the like. On the other hand, a real semantic image is generated through semantic segmentation using an image obtained from a camera mounted on a moving object. Then, the self-localization is determined by matching the simulated semantic image and the real semantic image using a single evaluation function. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Markus Herb, Matthias Lemberger, Marcel M. Schmitt, Alexander Kurz, Tobias Weiherer, Nassir Navab and Federico Tombari(2021),Semantic Image Alignment for Vehicle Localization [Overview of the project] [Problems that the invention aims to solve]

[0004] However, the configuration described in Non-Patent Document 1 above has a problem in that, in monotonous scenes with few features, such as highways and main roads, if the simulated semantic image and the actual semantic image match for pixels that make up the majority of the road, the system will judge that the simulated semantic image and the actual semantic image match even if the actual position is different, making it impossible to correctly estimate the position. Therefore, a self-localization device capable of improving the accuracy of self-localization is desired. [Means for solving the problem]

[0005] This disclosure can be implemented in the following forms:

[0006] According to one embodiment of the present disclosure, a self-position estimation device is provided. This self-position estimation device is mounted on a mobile body (10) having a camera (122) that captures images of the surroundings, and estimates the position of the mobile body, and includes: an initial position acquisition unit (111) that acquires initial position information representing the position of the mobile body; a map information acquisition unit (112) that acquires 3D map information representing a 3D map to which semantic labels are assigned; a simulated semantic image generation unit (113) that generates a simulated semantic image, which is an image visible through the camera from the position indicated by the initial position information on the 3D map information and includes depth information; and a real semantic image that is obtained by performing semantic segmentation on the surrounding image captured by the camera. The system includes a real semantic image generation unit (114), an optimization processing unit (115) that calculates the degree of agreement between the simulated semantic image and the real semantic image using an evaluation function, aligns the simulated semantic image to maximize the degree of agreement and performs optimization, and derives a relative pose matrix when the simulated semantic image is optimized, and a self-position calculation unit (116) that calculates the self-position of the moving object using the relative pose matrix derived by the optimization processing unit and the initial position information, wherein the optimization processing unit calculates the degree of agreement during optimization using weight variables set for pixels included in the simulated semantic image in the evaluation function.

[0007] According to the self-position estimation device of this form, the optimization processing unit aligns the simulated semantic image so that the degree of coincidence between the simulated semantic image and the actual semantic image becomes high, and the relative pose matrix when the simulated semantic image is optimized is derived. Then, using the relative pose matrix obtained by the optimization process and the initial position information obtained by the initial position acquisition unit, the position and orientation of the moving body are estimated. Further, when collating the degree of optimization coincidence between the simulated semantic image and the actual semantic image, it is performed by an evaluation function using a weight variable set for the pixels included in the simulated semantic image. Therefore, for example, regarding pixels that are landmarks that are easy to uniquely determine the position, adjustments such as increasing the weight in the optimization are possible. Thereby, the accuracy when collating the coincidence between the simulated semantic image and the actual semantic image can be improved compared to the case of using an evaluation function that does not have a weight variable. As a result, the accuracy of self-position estimation can be improved.

Brief Description of the Drawings

[0008] [Figure 1] It is a block diagram showing a schematic functional configuration of the self-position estimation device in the first embodiment of the present disclosure. [Figure 2] It is a flowchart showing the procedure of the self-position estimation process executed by the self-position estimation device. [Figure 3] It is a graph plotting the vertical position error after optimization by the method of the first embodiment. [Figure 4] It is a graph plotting the vertical position error after optimization by the method of the comparative form.

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described based on the drawings. A. First Embodiment: A1. Configuration of the self-position estimation device 110: The configuration of the self-position estimation device 110 of the first embodiment will be described with reference to Figure 1. The self-position estimation device 110 of the first embodiment is mounted on a vehicle 10 as a moving object and estimates the position of the vehicle 10 while it is in motion. The self-position estimation device 110 estimates the position and orientation of the vehicle by aligning a 3D map on which semantic labels are assigned to each surface with semantic images obtained from camera images actually captured by a camera 122 on the vehicle 10. The vehicle 10 can move by autonomous driving based on its own position estimated by the self-position estimation device 110. Note that "self-position" includes the position and orientation of the moving object and means position and orientation.

[0010] As shown in Figure 1, the vehicle 10 includes a self-position estimation device 110, a camera 122, an object sensor 124, a vehicle position sensor 126, a map information storage unit 130, and a communication unit 140. The vehicle 10 also includes a vehicle control unit (not shown), and the self-position estimation device 110 and the vehicle control unit are electrically connected.

[0011] Camera 122 captures images of the area around the vehicle 10 and acquires images. Object sensor 124 acquires location information of geographical features. Examples of object sensors 124 include laser radar, millimeter-wave radar, ultrasonic sensors, and other object sensors that utilize reflected waves. In this embodiment, object sensor 124 has an infrared sensor 125 that irradiates infrared light with a laser and acquires location information of geographical features based on the reflected infrared light. In this embodiment, "geographical features" is a concept that includes only the leaves of street trees and snow accumulated on the road.

[0012] The vehicle position sensor 126 detects the current position of the vehicle 10. Examples of the vehicle position sensor 126 include a Global Navigation Satellite System (GNSS) such as GPS (Global Positioning System). In this embodiment, the information representing the position of the vehicle 10 detected by the vehicle position sensor 126 is called the initial position information.

[0013] The map information storage unit 130 stores map information. "Map information" is information indicating the location of various objects. In this embodiment, the map information is 3D map information to which semantic labels are assigned.

[0014] The self-position estimation device 110 comprises an initial position acquisition unit 111, a map information acquisition unit 112, a simulated semantic image generation unit 113, a real semantic image generation unit 114, an optimization processing unit 115, and a self-position calculation unit 116. The self-position estimation device 110 consists of a central processing unit (CPU), a microcomputer composed of RAM and ROM, and the functions of each of these units are realized by the microcomputer executing a pre-installed program. However, some or all of the functions of each of these units may be realized by hardware circuits.

[0015] The initial position acquisition unit 111 acquires initial position information, which is information about the current position of the vehicle 10, from the vehicle position sensor 126. The initial position information contains an error compared to the accurate position information of the vehicle 10.

[0016] The map information acquisition unit 112 acquires 3D map information from the map information storage unit 130, which is information about the locations of features included in the range corresponding to the location represented by the initial location information.

[0017] The simulated semantic image generation unit 113 uses the initial position information to generate a simulated semantic image that is an image visible from the initial position information through the camera 122 on the acquired 3D map information, and which includes depth information.

[0018] The actual semantic image generation unit 114 generates an actual semantic image, which is an image obtained by performing semantic segmentation on the actual surrounding image captured by the camera 122.

[0019] The optimization processing unit 115 uses an evaluation function to calculate the degree of agreement between the simulated semantic image and the real semantic image, and performs optimization by aligning the simulated semantic image to increase the degree of agreement. It then derives the relative pose matrix when the simulated semantic image is optimized. The specific evaluation function and the details of the self-localization process, including the generation of the simulated and real semantic images, will be explained in detail in the "Self-Localization Method" section below.

[0020] The self-position calculation unit 116 calculates the vehicle 10's own position using the relative attitude matrix derived by the optimization processing unit 115 and the initial position information.

[0021] A2. Self-position estimation method: Next, we will explain the method for estimating the self-position of the vehicle 10 using the self-position estimation device 110 described above. We will explain the basic flow of self-position estimation. First, the initial position and attitude P is obtained in advance from a vehicle position sensor 126 such as GPS. R The pose P is obtained and displayed on a 3D map with semantic labels. R The system simulates the semantic image seen through camera 122. In other words, it generates a simulated semantic image. Because the generated simulated semantic image is created based on 3D map information, each pixel contains depth information.

[0022] Next, the in-vehicle camera images actually captured by camera 122 are subjected to semantic segmentation to generate a real semantic image. Then, the simulated camera semantic image is rotated and translated to approximate the real semantic image. Rotation and translation are performed using the relative attitude matrix " F P R This can be expressed as ''. This relative attitude matrix F P R By adjusting this, alignment is performed to increase the degree of agreement with the actual semantic image. The relative pose matrix obtained through this optimization process. F P R And the initial position and attitude PR Using and, the position and orientation P of the vehicle 10 F can be obtained as in the following equation (1).

Equation

[0023] Next, the specific processing procedure of the self-position estimation method will be described with reference to FIG. 2. This process is a process repeatedly executed by the self-position estimation device 110 during the running of the vehicle 10, and is, for example, a process repeatedly executed every 100 ms.

[0024] When the self-position estimation process is started, in step 101 (hereinafter, "step" is abbreviated as "S"), the initial position acquisition unit 111 acquires initial position information from the vehicle position sensor 126.

[0025] Next, in S102, the simulated semantic image generation unit 113 generates a simulated semantic image including depth information, which is an image visible through the camera 122 from the position indicated by the initial position information. At this time, the 3D map information, which is the information on the positions of ground objects included in the range corresponding to the position represented by the initial position information, is acquired from the map information storage unit 130 by the map information acquisition unit 112. Then, the acquired 3D map information is used to generate a simulated semantic image. Here, since the simulated semantic image is cut out from the 3D map, each pixel has depth information.

[0026] Next, in S103, the actual semantic image generation unit 114 generates an actual semantic image by passing the actual surrounding image captured by the camera 122 through semantic segmentation.

[0027] Next, in S104, the optimization processing unit 115 performs the relative attitude matrix F P R The simulated semantic image is rotated and translated. In the initial relative pose, the simulated semantic image is not rotated or translated. Next, in S105, the degree of agreement between the simulated semantic image and the actual semantic image is calculated.

[0028] Then, in S106, the relative attitude matrix is ​​generated based on the degree of agreement. F P R Optimization is performed. In S107, it is determined whether the process from S104 onwards has been repeated N times, which is a predetermined number. If the N repetitions have not been completed (S107: NO), the process returns to S104, and the process from S104 to S106 is repeated. The process from S104 to S107 rotates and translates the simulated semantic image by appropriately changing each movement parameter and aligning it to obtain the optimal relative pose matrix. F P R This is a process that runs an optimization loop to find the value.

[0029] Specifically, edges are extracted at the semantic boundaries in a simulated semantic image to create a 3D edge point image. Then, the created point cloud is rotated and translated to approximate a real semantic image. In other words, the row number of rotations and translations of the edge points is optimized to increase the degree of agreement between the edge point image of the simulated semantic image and the real semantic image.

[0030] On the other hand, if the N iterations are completed (S107: YES), the process proceeds to S108, where the self-position calculation unit 116 calculates the optimized relative attitude matrix obtained in S106. F P R and initial position and attitude PR By substituting and into equation (1) above, the self-position and attitude P of the vehicle 10 can be obtained. F The result is calculated. This completes the processing routine.

[0031] The optimization process described in S104 to S106 by the optimization processing unit 115 will be explained in more detail below. Assume there are two different images I1 and I2 obtained from the same camera, and that the position and orientation of the images taken are P1 and P2, respectively, and the depths of the images are D1 and D2, respectively. Here, we assume that image I is given by the following equation (2). We assume that depth D is given by the following equation (3).

number

number

[0032] The projection function that transforms the pixel position x in image I1 to the pixel position in image I2 is expressed by the following equation (4).

number

number

number

[0033] In this embodiment, semantic agreement is optimized instead of luminance agreement. The degree of agreement between the simulated semantic image and the actual semantic image is calculated using the following equation (7).

number

number

[0034] Furthermore, c is the index of the correct class of the simulated semantic image corresponding to pixel P. Based on this, the evaluation function to be optimized can be defined as equation (9) below.

number

[0035] In equation (9) above, "k" is a weight variable set according to the pixel. The weight variable k differs depending on the frequency of occurrence of the class to which the pixel belongs. The weight variable k is set higher for pixels belonging to classes that appear less frequently in the image than for pixels belonging to classes that appear more frequently. "Frequency of occurrence" can be understood by the number of points.

[0036] For example, pixels that appear infrequently, such as utility poles or support posts for signs, have a large weight variable k, while pixels that appear frequently, such as roads, have a small weight variable k. Generally, classes with low occurrence frequency are easier to uniquely locate, while classes with high occurrence frequency have fewer features and are more difficult to uniquely locate. Therefore, in optimization, pixels that serve as landmarks and are easy to uniquely locate are given a larger weight, while pixels of objects with few features, such as highways and main roads, are given a smaller weight. This improves the accuracy of matching simulated semantic images with real semantic images compared to using an evaluation function without a weight variable k.

[0037] In image matching methods, it is rare for the correct orientation to be reached with a single optimization; therefore, optimization is continued over multiple cycles to gradually approach the true value. Specifically, the position and orientation estimated by optimization in an arbitrary frame (1st cycle) is added to the displacement amount obtained from an odometry sensor (inertial measurement unit, internal sensor, etc.) to determine the initial position for the next frame (2nd cycle), and optimization is then performed.

[0038] [effect] In the self-localization device 110 and self-localization method of the first embodiment described above, in the optimization process to bring the simulated semantic image closer to the real semantic image, the simulated semantic image is moved in a direction that increases the number of points that match the real semantic image. At that time, the weights used to determine which points match are adjusted by using an evaluation function (equation (9) above) that uses a weight variable k.

[0039] In the comparative method described in Non-Patent Document 1 above (hereinafter also simply referred to as the "comparative method"), the following equation (10), which does not have a weight variable k, is used as the evaluation function.

number

[0040] In contrast, in the first embodiment described above, the degree of agreement is compared using an evaluation function (equation (9) above) in which a weight variable k is set for each pixel. Therefore, in monotonous scenes with few features, such as highways and main roads, it is possible to suppress the inaccurate judgment that the simulated semantic image and the real semantic image match when the majority of pixels, such as those on the road, match. In other words, even if many pixels appear to match between the simulated semantic image and the real semantic image, the degree of agreement can be judged as not high because the weights are applied to adjust for this. If the position of a pixel is difficult to uniquely determine, the degree of agreement can be judged as not high. Through the above processing, the optimal relative pose matrix is ​​obtained. F P R This allows us to derive the relative attitude matrix. F P R This method can improve the accuracy of self-position calculation.

[0041] Figure 3 is a graph plotting the vertical position error after optimization using the method of the first embodiment, in the case where vehicle 10 is traveling straight on a main road which is a straight path. Figure 4 is a graph plotting the vertical position error after optimization using the method of the comparison form, in the case where vehicle 10 is traveling straight on a main road which is a straight path, similar to Figure 3. In Figures 3 and 4, the horizontal axis represents the number of optimization attempts (1 division = 200 ms, 150 divisions = 30 s), and the vertical axis represents the vertical position error (m). Eight estimation trials were performed for both the comparison form and the first embodiment, and the data is shown. "Vertical position error" refers to the error in the position of vehicle 10 in the direction of travel. Note that the "number of optimization attempts" here does not correspond to "N times" in S107 above, but rather counts a series of processes of repeating the processing from S104 to S107 N times as "1 time".

[0042] As shown in Figure 4, in the comparative method, even after optimization, the overall vertical position error did not converge to within 30 cm. In Figures 3 and 4, the line where the vertical position error = 30 cm is shown as a horizontal dashed line. Furthermore, the average vertical position error μ after optimization was μ = 1.379 m after 30 seconds. On the other hand, as shown in Figure 3, in the method of the first embodiment described above, when the pole weight was set to k = 100, convergence was achieved in 6 out of 8 cases, and the average vertical position error in the cases where convergence occurred was μ = 0.179 m, confirming the effect of improvement. In other words, according to the first embodiment described above, the accuracy of self-position estimation can be improved.

[0043] B. Second Embodiment: Next, a self-position estimation device 110 and a self-position estimation method of a second embodiment of this disclosure will be described. Components similar to those in the first embodiment are denoted by the same reference numerals and their descriptions are omitted. Note that the following embodiments, including the second embodiment, differ from the first embodiment only in the way the weight variable k is set.

[0044] In the second embodiment, the weight variable k varies depending on the distance of the pixel from the camera 122, with smaller values ​​set for pixels that are closer than pixels that are farther away. Points closer to the camera 122 occupy a larger proportion of the image, but small positional errors can significantly affect pixel misalignment, which is thought to cause errors in pose estimation. Therefore, the weight of pixels that are close in depth is reduced, and the weight of pixels in the mid-to-far range is increased. More specifically, the weight may be changed in proportion to the distance of the target point.

[0045] According to the second embodiment, the accuracy of self-localization can be improved, similar to the first embodiment.

[0046] C. Third Embodiment: Next, a self-position estimation device 110 and a self-position estimation method of a third embodiment of this disclosure will be described. In the third embodiment, the weight variable k is set differently depending on the properties of the object to which the pixel belongs. Specifically, the weight of easily changing objects such as leaves and branches of trees is reduced. For easily changing objects, there may be differences between the pre-stored 3D map information and the actual camera image, which is thought to cause errors in pose estimation. Therefore, by reducing the weight of pixels of such objects and minimizing their impact on the degree of agreement, it is possible to prevent the device from being aligned to an incorrect pose.

[0047] According to the third embodiment, the accuracy of self-localization can be improved, similar to the first embodiment.

[0048] D. Fourth Embodiment: Next, a self-position estimation device 110 and a self-position estimation method according to a fourth embodiment of this disclosure will be described. In the fourth embodiment, the weight variable k is a parameter adjusted by machine learning. The weight variable k is adjusted and set by machine learning as a value that minimizes the evaluation function so that the position error is small.

[0049] According to the fourth embodiment, since the optimal weight variable k is set appropriately by machine learning, the accuracy of self-localization can be improved, similar to the first embodiment.

[0050] E. Other embodiments: (E1) In each of the above embodiments, the self-position estimation device 110 is mounted on a vehicle 10, but it may also be mounted on other objects such as ships, drones, autonomous mobile robots, or moving objects such as people.

[0051] (E2) In the self-position estimation device 110 and self-position estimation method of each embodiment described above, the evaluation function shown in equation (9) above was adopted, but other equations may also be used. These equations have a weight variable k set according to the pixels and a relative pose matrix. F P R Any function that can perform optimization will suffice.

[0052] (E3) In setting the weight variable k of the self-localization device 110 and self-localization method of each embodiment described above, the setting methods in each embodiment may be combined as appropriate. For example, the weight variable k may be set by combining the class occurrence frequency and the distance to the target pixel. In addition, the weight variable k may be set by combining other elements such as the properties of the object and adjustments by machine learning as appropriate.

[0053] (E4) The self-localization devices 110 and methods described herein may be implemented by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. Alternatively, the self-localization devices 110 and methods described herein may be implemented by a dedicated computer provided by configuring a processor by one or more dedicated hardware logic circuits. Alternatively, the self-localization devices 110 and methods described herein may be implemented by one or more dedicated computers configured by a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. Furthermore, the computer program (self-localization program) may be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium.

[0054] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, the technical features in each embodiment corresponding to the technical features in the embodiments described in the summary of the invention can be replaced or combined as appropriate in order to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate. [Explanation of symbols]

[0055] 110...Self-position estimation device, 111...Initial position acquisition unit, 112...Map information acquisition unit, 113...Simulated semantic image generation unit, 114...Actual semantic image generation unit, 115...Optimization processing unit, 116...Self-position calculation unit, 122...Camera, 124...Object sensor, 125...Infrared sensor, 126...Vehicle position sensor, 130...Map information storage unit, 140...Communication unit

Claims

1. A self-position estimation device mounted on a mobile body (10) having a camera (122) for imaging the surrounding area, which estimates the position of the mobile body, An initial position acquisition unit (111) acquires initial position information representing the position of the moving body, A map information acquisition unit (112) acquires 3D map information representing a 3D map to which semantic labels have been assigned, A simulated semantic image generation unit (113) generates a simulated semantic image that includes depth information and is an image visible through the camera from the position indicated by the initial position information on the 3D map information, A real semantic image generation unit (114) generates a real semantic image, which is an image obtained by performing semantic segmentation on the surrounding image captured by the camera, An optimization processing unit (115) calculates the degree of agreement between the simulated semantic image and the actual semantic image using an evaluation function, aligns the simulated semantic image to maximize the degree of agreement, performs optimization, and derives the relative pose matrix when the simulated semantic image is optimized. A self-position calculation unit (116) calculates the self-position of the moving body using the relative attitude matrix derived by the optimization processing unit and the initial position information, Equipped with, The optimization processing unit calculates the degree of agreement during the optimization using weight variables set for pixels included in the simulated semantic image in the evaluation function, in a self-localization device.

2. The self-localization device according to claim 1, wherein the weight variable is set according to the frequency of occurrence of the class to which the pixel belongs, and the weight variable is set to be larger for the pixel belonging to the class with a low frequency of occurrence than for the pixel belonging to the class with a high frequency of occurrence.

3. The self-localization device according to claim 1 or 2, wherein the weight variable is set according to the distance of the pixel from the camera, and the weight variable is set to be smaller for pixels that are closer than the camera than for pixels that are farther away.

4. The self-localization device according to claim 1 or claim 2, wherein the weight variables are set according to the properties of the object to which the pixels belong.

5. The self-position estimation device according to claim 1 or claim 2, wherein the weight variable is set by machine learning as a value that minimizes the evaluation function so that the position error is small.

6. A self-position estimation method for estimating the position of a moving body (10) using a self-position estimation device mounted on a moving body (10) having a camera (122) that captures images of the surrounding area, The self-position estimation device comprises an initial position acquisition unit (111), a map information acquisition unit (112), a simulated semantic image generation unit (113), a real semantic image generation unit (114), an optimization processing unit (115), and a self-position calculation unit (116). The process of acquiring initial position information representing the position of the moving body using the initial position acquisition unit, The process of acquiring 3D map information representing a 3D map to which semantic labels have been assigned by the map information acquisition unit, The process of generating a simulated semantic image using the simulated semantic image generation unit, which is an image visible through the camera from the position indicated by the initial position information on the 3D map information and includes depth information, The process of generating a real semantic image, which is an image obtained by performing semantic segmentation on the surrounding image captured by the camera using the real semantic image generation unit, The optimization processing unit calculates the degree of agreement between the simulated semantic image and the actual semantic image using an evaluation function, aligns the simulated semantic image to maximize the degree of agreement, performs optimization, and derives the relative pose matrix when the simulated semantic image is optimized. The self-position calculation unit calculates the self-position of the moving object using the relative attitude matrix derived by the optimization processing unit and the initial position information. Equipped with, The optimization processing unit calculates the degree of agreement during optimization using weight variables set for pixels included in the simulated semantic image in the evaluation function, as a self-localization method.

7. A self-position estimation program is mounted on a mobile body (10) having a camera (122) that takes images of the surrounding area, and estimates the position of the mobile body, A function to acquire initial position information representing the position of the aforementioned moving object, A function to acquire 3D map information representing a 3D map to which semantic labels have been assigned, A function to generate a simulated semantic image, which includes depth information, that is an image visible through the camera from the position indicated by the initial position information on the 3D map information, The camera has a function to generate a real semantic image, which is an image obtained by performing semantic segmentation on the surrounding image captured by the camera. A function that calculates the degree of agreement between the simulated semantic image and the actual semantic image using an evaluation function that includes weight variables set for pixels in the simulated semantic image, optimizes the simulated semantic image by aligning it to increase the degree of agreement, and derives the relative pose matrix when the simulated semantic image is optimized. A function to calculate the self-position of the moving object using the derived relative attitude matrix and the initial position information, A self-localization program that enables computers to perform this task.

Images