Map generation device, map generation method, and program

Abstract

This map generation device 10 comprises: a data acquisition unit 11 that acquires image data in units of frames, the image data being outputted from a depth sensor for which the view angle is fixed; a feature point extraction unit 12 that extracts feature points from the image data for each frame; a filtering unit 13 that assesses, for each frame, whether a section is present in the given frame that does not overlap another frame other than the given frame, and, when it is found as a result of the assessment that such a section is present, eliminates any feature points extracted from the non-overlapping section; and an environment map generation unit 14 that identifies a set of feature points that correspond between the frames, furthermore calculates three-dimensional coordinates of the feature points identified as the set, and generates, using the result of calculation, an environment map configured from aggregations of feature points identified as the set.

Description

Technical Field

[0001] The present disclosure relates to a map generation device and a map generation method for generating an environmental map constructed from point cloud data, and further relates to a program for realizing these.

Background Art

[0002] In recent years, with the improvement of sensing technology, SLAM (Simultaneous Localization and Mapping) has attracted attention. SLAM is a technology that simultaneously estimates the self-position and creates an environmental map by a moving body equipped with sensors (see, for example, Patent Document 1). According to SLAM, a moving body such as a robot that autonomously travels does not need to move randomly and can move along an autonomous driving map obtained from the environmental map, so its moving efficiency will be improved.

[0003] Specifically, the moving body acquires image data output from a camera for each frame, searches for feature points corresponding to the feature points extracted from past frames in the latest frame, and extracts a set of feature points composed of the corresponding feature points. Then, the moving body calculates the camera matrix in the latest frame using the set of feature points, and calculates the three-dimensional coordinates of the feature points for each set of feature points using the camera matrix and the two-dimensional coordinates of the feature points in the frame.

[0004] Also, when the moving body calculates the camera matrix, it calculates the position of the camera from the camera matrix and performs self-position estimation. Further, the moving body generates or updates an environmental map (hereinafter referred to as "3D environmental map") composed of a three-dimensional point cloud using the feature points whose three-dimensional coordinates have been calculated. Then, the moving body generates an autonomous driving map by converting the 3D environmental map into two dimensions and performs autonomous driving using the autonomous driving map.

[0005] In addition to cameras, depth sensors that can measure the distance (depth) to an object for each pixel of image data are used as sensors. By using a depth sensor, the density of the point cloud that constitutes the environmental map can be increased compared to the case of using a camera, enabling highly accurate autonomous driving. As a specific example of a depth sensor, a LiDAR (light detection and ranging) sensor can be mentioned (see, for example, Patent Document 2).

[0006] Patent Document 2 discloses a 3D LiDAR sensor. The 3D LiDAR sensor disclosed in Patent Document 2 includes a light source, a light receiver, a mirror, and a control device. The laser light emitted from the light source is reflected by the mirror and then irradiated onto an object. Then, the laser light reflected by the object enters the light receiver. The control device calculates the depth to the object from the time from when the laser light is emitted from the light source until it enters the light receiver.

[0007] In addition, the 3D LiDAR sensor disclosed in Patent Document 2 is a rotational 3D LiDAR sensor and also includes a rotation mechanism for rotating the mirror. Therefore, by using the 3D LiDAR sensor disclosed in Patent Document 2, scanning (search for feature points) is possible in a 360-degree range or a range close thereto.

Prior Art Documents

Patent Documents

[0008]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0009] By the way, the above-described rotational 3D LiDAR sensor enables wide-range scanning, but on the other hand, since a rotating mechanism is required, there is a problem that the manufacturing cost is high. For this reason, in SLAM, a 3D LiDAR without a rotating mechanism and with a fixed field of view (hereinafter referred to as "3D LiDAR with a fixed field of view") has also been proposed.

[0010] However, in the 3D LiDAR with a fixed field of view, since the field of view is fixed, there may be a situation where a part of the latest frame does not overlap with the past frame in SLAM. When such a situation occurs, a wrong set of feature points is extracted, and a low-precision environmental map is generated. As a result, the accuracy of autonomous driving is greatly reduced.

[0011] An example of the object of the present disclosure is to improve the accuracy of an environmental map when using a depth sensor with a fixed field of view.

Means for Solving the Problems

[0012] To achieve the above object, a map generation device according to an aspect of the present disclosure includes: a data acquisition unit that acquires image data output from a depth sensor with a fixed field of view in units of frames; a feature point extraction unit that extracts feature points from the image data for each frame; for each frame, determines whether there is a portion that does not overlap with another frame other than the frame in the frame, and if the determination result is that there is, excludes the feature points extracted from the non-overlapping portion, a filtering unit; identifies a set of corresponding feature points between frames, further calculates the 3D coordinates of the feature points identified as a set, and generates an environmental map composed of the set of feature points identified as a set using the calculation result, an environmental map generation unit; is provided. It is characterized by that.

[0013] In order to achieve the above object, a map generation method according to one aspect of the present disclosure includes: a data acquisition step of acquiring, in frame units, image data output from a depth sensor with a fixed viewing angle; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of determining, for each frame, whether there is a portion of the frame that does not overlap with another frame other than the frame, and if the determination result is positive, excluding the feature points extracted from the non-overlapping portion; a surrounding map generation step of identifying a set of corresponding feature points between frames, further calculating three-dimensional coordinates of the feature points identified as a set, and generating a surrounding map composed of the set of the feature points identified as a set using the calculation result; and is characterized in that.

[0014] Furthermore, in order to achieve the above object, a program according to one aspect of the present disclosure causes a computer to perform a data acquisition step of acquiring, in frame units, image data output from a depth sensor with a fixed viewing angle; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of determining, for each frame, whether there is a portion of the frame that does not overlap with another frame other than the frame, and if the determination result is positive, excluding the feature points extracted from the non-overlapping portion; a surrounding map generation step of identifying a set of corresponding feature points between frames, further calculating three-dimensional coordinates of the feature points identified as a set, and generating a surrounding map composed of the set of the feature points identified as a set using the calculation result; and is characterized in that.

Advantages of the Invention

[0015] According to the present disclosure as described above, when using a depth sensor with a fixed field of view, the accuracy of the environmental map can be improved.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0017] (Embodiment) Hereinafter, a map generation device, a map generation method, and a program in the embodiment will be described with reference to FIGS. 1 to 7.

[0018] [Device Configuration] First, the schematic configuration of the map generation device will be described with reference to FIG. 1. FIG. 1 is a configuration diagram showing the schematic configuration of the map generation device.

[0019] The map generation device 10 shown in FIG. 1 is a device for generating an environmental map constructed from point cloud data. As shown in FIG. 1, the map generation device 10 includes a data acquisition unit 11, a feature point extraction unit 12, a filtering unit 13, and an environmental map generation unit 14.

[0020] The data acquisition unit 11 acquires, in frame units, the image data output from a depth sensor with a fixed viewing angle. The feature point extraction unit 12 extracts feature points from the image data acquired by the data acquisition unit 11 for each frame.

[0021] The filtering unit 13 determines, for each frame, whether there is a portion in that frame that does not overlap with another frame other than that frame. Then, if the determination result is that there is such a portion, the filtering unit 13 excludes the feature points extracted from the non-overlapping portion.

[0022] The environmental map generation unit 14 identifies pairs of corresponding feature points between frames, and further calculates the three-dimensional coordinates of the feature points identified as a pair. Then, the environmental map generation unit 14 generates an environmental map composed of a set of feature points identified as a pair using the calculation result.

[0023] In this way, in the embodiment, the map generation device 10 excludes the extracted feature points in the portion of each frame that does not overlap with another frame. For this reason, the extraction of incorrect pairs of feature points that occurs when the viewing angle of the depth sensor is fixed is avoided. That is, according to the embodiment, when using a depth sensor with a fixed viewing angle, the accuracy of the environmental map can be improved.

[0024] Subsequently, with reference to FIGS. 2 and 3, the configuration and functions of the map generation device in the embodiment will be specifically described. FIG. 2 is a configuration diagram specifically showing the configuration of the map generation device. FIG. 3 is an explanatory diagram for explaining the viewing angle of the depth sensor.

[0025] As shown in FIG. 2, in the embodiment, the map generation device 10 is mounted on a mobile body 100 capable of autonomous driving such as a robot. In addition to the map generation device 10, the mobile body 100 includes a depth sensor 20, a control device 30, a steering device 40, and a power train 50.

[0026] In the embodiment, the depth sensor 20 is attached to the moving body 100 such that the direction of travel thereof is imaged. Further, the depth sensor 20 is a sensor capable of capturing a depth image and outputs depth image data at a set frame rate. In the example of FIG. 2, a fixed field-of-view 3D LiDAR is used as the depth sensor 20.

[0027] As shown in FIG. 3, the field of view of the depth sensor 20 is fixed, and its imaging region 21 changes depending on the orientation of the moving body 100. In the example of FIG. 3, the change in the imaging region 21 from time (t-2) to time (t) is shown.

[0028] The control device 30 is constructed by a computer mounted on the moving body 100. The control device 30 controls the traveling direction and moving speed of the moving body 100 using the environmental map generated by the map generation device 10. For example, the control device 30 sets a route from the current location to the destination using the environmental map, determines the traveling direction and moving speed of the moving body 100 so that the moving body 100 moves on the set route, and controls the power train 50 and the steering device 40.

[0029] The power train 50 is composed of an electric motor for traveling, a power transmission mechanism, etc. Tires, caterpillars, etc. are connected to the power train 50. The power train 50 rotates tires, caterpillars, etc. in response to an instruction from the control device 30.

[0030] The steering device 40 includes a mechanism for controlling the direction of the steering wheel of the moving body 100. The steering device 40 determines the direction of the steering wheel in response to an instruction from the control device 30. Further, the steering device 40 may include a mechanism for controlling the moving direction by controlling the torque of the left and right drive wheels.

[0031] In the embodiment, the map generation device 10 is constructed by a program in the embodiment described later on a computer mounted on the moving body 100. Further, the map generation device 10 may be constructed by a device (for example, an electronic circuit or the like) different from the computer mounted on the moving body 100.

[0032] In the embodiment, the data acquisition unit 11 acquires depth image data from the depth sensor 20 in units of frames. The data acquisition unit 11 sequentially inputs the acquired image data (frames) to the feature point extraction unit 12.

[0033] In the embodiment, when a frame is input, the feature point extraction unit 12 extracts feature points of an object for each frame using, for example, a general FAST algorithm. Then, the feature point extraction unit 12 inputs information for specifying the feature points extracted for each frame to the filtering unit 13.

[0034] In the embodiment, when feature points for each frame are input, the filtering unit 13 first obtains the traveling direction and position of the moving body 100 based on the information of the moving body 100 for each of the input frames. Specifically, for each frame, the filtering unit 13 obtains the position of the moving body 100 based on the information of the moving body 100 when the frame is output, and further obtains the absolute pose as the traveling direction. The absolute pose is represented by the translational vector T(x, y, z) of the moving body 100 and the rotation component R(roll, pitch, yaw).

[0035] Examples of the information of the moving body 100 include position information of the moving body 100 and information indicating the traveling direction. The position information may be obtained from a GPS (Global Positioning System) receiver mounted on the moving body 100, or may be obtained by self-position estimation described later. Further, the information indicating the traveling direction can be obtained, for example, from an angular velocity sensor mounted on the moving body 100.

[0036] Next, the filtering unit 13 determines, for each frame, whether there is a portion in the frame that does not overlap with another frame, using the traveling direction and position of the moving body 100. In the embodiment, the filtering unit 13 determines, for each frame, whether there is another frame around the frame, using the traveling direction and position of the moving body 100. Then, when there is no other frame around the frame, the filtering unit 13 determines that there is a portion in the frame that does not overlap with another frame.

[0037] Specifically, the filtering unit 13 identifies a frame located at an edge on the right side, left side, upper side, or lower side of the moving body. In the example of FIG. 3, the frame output at time t is the frame located at the edge. There may be a portion in the frame located at the edge that does not overlap with another frame.

[0038] Next, the filtering unit 13 calculates the amount of change in the posture of the moving body 100 between the frame located at the edge and the frame output before it. Specifically, let the time when the frame located at the edge is output be t, and the time when the previous frame is output be t - 1, and the rotation components at each time be R t 、R t-1 In this case, the amount of change in the posture γ of the moving body is calculated by Equation (1) below.

[0039] (Equation (1)) γ = R t - R t-1

[0040] Next, the filtering unit 13 determines, based on the calculated amount of change in the posture γ, whether there is a portion in the frame located at the edge that does not overlap with another frame. As a result of the determination, if it exists, the feature points extracted from the non-overlapping portion are excluded. Here, with reference to FIG. 4, the determination process for whether the frames overlap will be described. FIG. 4 is a diagram showing the determination process in the filtering unit of the map generation device.

[0041] In FIG. 4, 21 indicates the imaging area when the frame located at the end is imaged (time t). Also, in FIG. 4, the depth direction is the z-axis direction, the horizontal direction of the image is the x-axis direction, and the vertical direction of the image is the y-direction. Further, let the angle of view of the depth sensor 20 be H, the extracted feature point be P(x, y, z), and the angle in the zx plane between the normal line passing through the center of the imaging surface of the depth sensor 20 and the line extending from the feature point P passing through the center be θ.

[0042] In this case, when the angle θ of the feature point P is larger than the angle ((H / 2) - abs(γ)) obtained by subtracting the posture change amount γ from half of the angle of view H, the feature point P exists in the area where the frames do not overlap.

[0043] Therefore, the filtering unit 13 determines whether each of the feature points extracted from the frame at time t satisfies the following equation (2). And when there exists a feature point that satisfies the following equation (2), the filtering unit 13 determines that there exists a non-overlapping portion with another frame in the frame located at the end.

[0044] (Equation 2) (H / 2) - abs(γ) > θ

[0045] After that, the filtering unit 13 excludes the feature points existing in the corresponding portion of the frame determined not to overlap with another frame. Specifically, the filtering unit 13 excludes the feature points (the feature points that satisfy the above equation (2)) existing in the non-overlapping portion of the frame located at the end with another frame.

[0046] The environmental map generation unit 14 first associates, for each frame, the feature points extracted therefrom with the feature points extracted from past frames using the feature points of each frame that were not excluded by the filtering unit 13, and specifies a set of feature points.

[0047] Next, for each frame, the environmental map generation unit 14 calculates the camera matrix in that frame using the set of feature points specified in that frame. Then, the environmental map generation unit 14 calculates the three-dimensional coordinates of the feature points using the calculated camera matrix and the two-dimensional coordinates of the feature points in the frame.

[0048] Subsequently, when the environmental map generation unit 14 calculates the camera matrix, it calculates the position of the camera from the camera matrix and performs self-position estimation. Further, the mobile object generates or updates an environmental map (hereinafter referred to as a "three-dimensional environmental map") composed of a three-dimensional point cloud using the feature points whose three-dimensional coordinates have been calculated. FIG. 5 is a diagram showing an example of a three-dimensional environmental map.

[0049] In addition, the environmental map generation unit 14 inputs the generated three-dimensional environmental map to the control device 30. Further, the environmental map generation unit 14 can also generate a two-dimensional environmental map by converting the three-dimensional environmental map into two dimensions. In this case, the environmental map generation unit 14 passes the generated two-dimensional environmental map to the control device 30. Thereafter, the control device 30 controls the traveling direction and traveling speed of the mobile object using the obtained environmental map.

[0050] Note that in the above-described example, as the depth sensor 20, a three-dimensional LiDAR capable of measuring the depth in a three-dimensional space is used. However, in the embodiment, the depth sensor 20 is not limited to this. In the embodiment, examples of the depth sensor 20 also include a fixed-angle two-dimensional LiDAR that measures the depth on a specific plane and a TOF (Time Of Flight) camera.

[0051] [Device Operation] Next, the operation of the map generation device 10 will be described with reference to FIG. 6. FIG. 6 is a flowchart showing the operation of the map generation device. In the following description, FIGS. 1 to 5 will be referred to as appropriate. Also, in the embodiment, the map generation method is implemented by operating the map generation device 10. Therefore, the description of the map generation method in the embodiment will be replaced with the following description of the operation of the map generation device 10.

[0052] As shown in FIG. 6, first, when depth image data is output from the depth sensor 20 in frame units, the data acquisition unit 11 acquires the output image data (step A1). Further, when the data acquisition unit 11 acquires image data for the set number of frames, the acquired image data is input to the feature point extraction unit 12.

[0053] Next, when the image data acquired in step A1 is input, the feature point extraction unit 12 extracts feature points for each frame (step A2). Further, the feature point extraction unit 12 outputs the extracted feature points for each frame to the filtering unit 13.

[0054] Next, when the feature points for each frame are input by step A2, the filtering unit 13 obtains the traveling direction and position of the moving body 100 based on the information of the moving body 100 for each of the input frames (step A3).

[0055] Next, the filtering unit 13 determines, for each frame, whether there is a portion in that frame that does not overlap with another frame using the traveling direction and position of the moving body 100 (step A4).

[0056] Specifically, in step A4, the filtering unit 13 identifies the frames located at the ends on the right side, left side, upper side, or lower side of the moving body. Further, the filtering unit 13 calculates the amount of change in the posture of the moving body 100 between the frame located at the end and the frame output before it using the above number 1.

[0057] Then, the filtering unit 13 determines, for each feature point extracted from the frame located at the end, whether it satisfies the above number 2 using the calculated amount of change in posture γ. When there are feature points that satisfy the above number 2, the filtering unit 13 determines that there is a portion in the frame located at the end that does not overlap with another frame.

[0058] Next, in a frame where there is a portion that does not overlap with another frame, the filtering unit 13 excludes the feature points in that corresponding portion (step A5). Specifically, in A5, the filtering unit 13 excludes the feature points that satisfy the above-mentioned number 2 in the frame located at the edge.

[0059] Next, the environmental map generation unit 14 associates, for each frame, the feature points extracted therefrom with the feature points extracted from past frames using the feature points not excluded in step A5, and specifies a set of feature points (step A6).

[0060] Next, the environmental map generation unit 14 calculates the three-dimensional coordinates of the feature points specified as a set in step A6, and generates an environmental map composed of the set of feature points specified as a set using the calculation results (step A7).

[0061] Thereafter, the environmental map generation unit 14 inputs the environmental map generated in step A7 to the control device 30. Thereby, the control device 30 controls the traveling direction and traveling speed of the moving body using the obtained environmental map.

[0062] As described above, in the embodiment, even if the viewing angle of the depth sensor 20 is fixed, since the feature points are excluded in the portion of the frame located at the edge that does not overlap with another frame, extraction of incorrect sets of feature points is avoided. Therefore, according to the embodiment, when using a depth sensor with a fixed viewing angle, the accuracy of the environmental map can be improved.

[0063] [Program] The program in the embodiment may be a program that causes a computer to execute steps A1 to A7 shown in FIG. 6. By installing and executing this program on a computer, the map generation device 10 and the map generation method in the embodiment can be realized. In this case, the processor of the computer functions as the data acquisition unit 11, the feature point extraction unit 12, the filtering unit 13, and the environmental map generation unit 14, and performs processing.

[0064] In addition, examples of the computer include the computer mounted on the moving body 100. Other examples of the computer include general-purpose PCs, smartphones, and tablet terminal devices.

[0065] In addition, the program in the embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of the data acquisition unit 11, the feature point extraction unit 12, the filtering unit 13, and the environmental map generation unit 14.

[0066] [Physical Configuration] Here, the computer that realizes the map generation device 10 by executing the program in the embodiment will be described with reference to FIG. 7. FIG. 7 is a block diagram showing an example of a computer that realizes the map generation device.

[0067] As shown in FIG. 7, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0068] In addition, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this mode, the GPU or FPGA can execute the program in the embodiment.

[0069] The CPU 111 expands the program in the embodiment composed of a code group stored in the storage device 113 into the main memory 112, and executes each code in a predetermined order to perform various operations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0070] Also, the program in the embodiment is provided in a state stored in a computer-readable recording medium 120. Note that the program in the present embodiment may also circulate on the Internet connected via the communication interface 117.

[0071] As a specific example of the storage device 113, in addition to a hard disk drive, a semiconductor storage device such as a flash memory can be mentioned. The input interface 114 mediates data transmission between the CPU 111 and an input device 118 such as a keyboard and a mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

[0072] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, and executes reading of the program from the recording medium 120 and writing of the processing result in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0073] As specific examples of the recording medium 120, general-purpose semiconductor memory devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as a flexible disk, or optical recording media such as a CD-ROM (Compact Disk Read Only Memory) can be mentioned.

[0074] Note that the map generation device 10 in the embodiment can also be realized by using hardware corresponding to each part, for example, an electronic circuit, instead of a computer installed with a program. Furthermore, part of the map generation device 10 may be realized by a program and the remaining part may be realized by hardware. In the embodiment, the computer is not limited to the computer shown in FIG. 7.

[0075] Some or all of the above-described embodiments can be expressed by (Appendix 1) to (Appendix 9) described below, but are not limited to the following description.

[0076] (Appendix 1) A data acquisition unit that acquires image data output from a depth sensor with a fixed viewing angle in frame units; A feature point extraction unit that extracts feature points from the image data for each frame; For each frame, it is determined whether there is a portion in the frame that does not overlap with another frame other than the frame. As a result of the determination, if it exists, the feature points extracted from the non-overlapping portion are excluded. A filtering unit; Identify pairs of corresponding feature points between frames, further calculate the three-dimensional coordinates of the feature points identified as a pair, and use the calculation result to generate an environmental map composed of the set of feature points identified as a pair. An environmental map generation unit; It is characterized by comprising: A map generation device.

[0077] (Appendix 2) The depth sensor is mounted on a moving body, The filtering unit obtains the traveling direction and position of the moving body for each frame based on the information of the moving body, and uses the traveling direction and position of the moving body for each obtained frame to determine whether there is a portion in the frame that does not overlap with another frame other than the frame. The map generation device according to Appendix 1.

[0078] (Appendix 3) For each frame, the filtering unit determines whether there is another frame around the frame using the determined direction and position of the moving object for each frame, and when there is no other frame around the frame, determines that there is a non-overlapping portion in the frame that does not overlap with another frame other than the frame. The map generation device according to Appendix 2.

[0079] (Appendix 4) A data acquisition step of acquiring image data output from a depth sensor with a fixed viewing angle in units of frames, A feature point extraction step of extracting feature points from the image data for each frame, For each frame, it is determined whether there is a non-overlapping portion in the frame that does not overlap with another frame other than the frame. If it is determined that there is, the feature points extracted from the non-overlapping portion are excluded. A filtering step, A step of identifying pairs of corresponding feature points between frames, further calculating the three-dimensional coordinates of the feature points identified as a pair, and using the calculation results to generate an environmental map composed of the set of the feature points identified as a pair. An environmental map generation step, having A map generation method characterized by the above.

[0080] (Appendix 5) The depth sensor is mounted on a moving object, In the filtering step, based on the information of the moving object, for each frame, the traveling direction and position of the moving object are obtained, and using the obtained traveling direction and position of the moving object for each frame, it is determined whether there is a non-overlapping portion in the frame that does not overlap with another frame other than the frame. The map generation method according to Appendix 4.

[0081] (Appendix 6) In the filtering step, for each frame, using the orientation and position of the moving object for each obtained frame, determine whether there is another frame around the frame, and when there is no other frame around the frame, determine that there is a portion of the frame that does not overlap with another frame other than the frame. The map generation method according to Supplementary Note 5.

[0082] (Supplementary Note 7) Cause a computer to A data acquisition step of acquiring image data output from a depth sensor with a fixed field angle in units of frames, A feature point extraction step of extracting feature points from the image data for each frame, For each frame, determine whether there is a portion of the frame that does not overlap with another frame other than the frame, and if it exists as a result of the determination, exclude the feature points extracted from the non-overlapping portion, a filtering step; Identify a set of corresponding feature points between frames, further calculate the three-dimensional coordinates of the feature points identified as a set, and generate an environmental map composed of the set of feature points identified as a set using the calculation results, an environmental map generation step; A program for causing the execution.

[0083] (Supplementary Note 8) The depth sensor is mounted on a moving object, In the filtering step, based on the information of the moving object, for each frame, obtain the traveling direction and position of the moving object, and use the obtained traveling direction and position of the moving object for each frame to determine whether there is a portion of the frame that does not overlap with another frame other than the frame. The program according to Supplementary Note 7.

[0084] (Supplementary Note 9) In the filtering step, for each frame, using the obtained direction and position of the moving object for each frame, it is determined whether there is another frame around the current frame. If there is no other frame around the current frame, it is determined that there is a non-overlapping part in the current frame with respect to another frame other than the current frame. The program according to Supplementary Note 8.

[0085] As described above, the present invention has been described with reference to the embodiments, but the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0086] This application claims priority based on Japanese Patent Application No. 2022-91128 filed on June 3, 2022, and incorporates all of its disclosures herein.

Industrial Applicability

[0087] As described above, according to the present disclosure, when using a depth sensor with a fixed viewing angle, the accuracy of the environmental map can be improved. The present disclosure is useful in the fields where SLAM is used.

Explanation of Signs

[0088] 10 Map generation device 11 Data acquisition unit 12 Feature point extraction unit 13 Filtering unit 14 Environmental map generation unit 20 Depth sensor 30 Control device 40 Steering device 50 Power train 100 Moving object 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. A data acquisition unit that acquires image data output from a depth sensor with a fixed viewing angle in units of frames; A feature point extraction unit that extracts feature points from the image data for each frame; For each frame, it is determined whether there is a portion in the frame that does not overlap with another frame other than the frame. As a result of the determination, if it exists, the feature points extracted from the non-overlapping portion are excluded, a filtering unit; A loop map generation unit that identifies pairs of corresponding feature points between frames, further calculates the three-dimensional coordinates of the feature points identified as a pair, and generates a loop map composed of the set of the feature points identified as a pair using the calculation result; It is provided with; A map generation device characterized by this.

2. The depth sensor is mounted on a moving body, The filtering unit obtains the traveling direction and position of the moving body for each frame based on the information of the moving body, and uses the traveling direction and position of the moving body for each obtained frame to determine whether there is a portion in the frame that does not overlap with another frame other than the frame. The map generation device according to claim 1.

3. The filtering unit determines for each frame whether there is another frame around the frame using the orientation and position of the moving body for each obtained frame. When there is no other frame around the frame, it is determined that there is a portion in the frame that does not overlap with another frame other than the frame. The map generation device according to claim 2.

4. Acquire image data output from a depth sensor with a fixed viewing angle in units of frames, Extract feature points from the image data for each frame, For each frame, it is determined whether there is a portion in the frame that does not overlap with another frame other than the frame. As a result of the determination, if it exists, the feature points extracted from the non-overlapping portion are excluded, Identify pairs of corresponding feature points between frames, further calculate the three-dimensional coordinates of the feature points identified as a pair, and generate a loop map composed of the set of the feature points identified as a pair using the calculation result, A map generation method characterized by this.

5. The depth sensor is mounted on a moving body, In the determination, based on the information of the moving object, for each frame, the traveling direction and position of the moving object are obtained, and using the traveling direction and position of the moving object for each obtained frame, it is determined whether there is a portion in the frame that does not overlap with another frame other than the frame. The map generation method according to claim 4.

6. In the determination, for each frame, using the orientation and position of the moving object for each obtained frame, it is determined whether there is another frame existing around the frame, and when there is no other frame existing around the frame, it is determined that there is a portion in the frame that does not overlap with another frame other than the frame. The map generation method according to claim 5.

7. Causing a computer to acquire image data output from a depth sensor with a fixed viewing angle in units of frames, extract feature points from the image data for each frame, for each frame, cause the frame to determine whether there is a portion in the frame that does not overlap with another frame other than the frame, and if it is determined as existing as a result of the determination, cause the feature points extracted from the non-overlapping portion to be excluded, specify a set of corresponding feature points between frames, and further calculate the three-dimensional coordinates of the feature points specified as a set, and generate an environmental map composed of the set of the feature points specified as a set using the calculation result. Program.

8. The depth sensor is mounted on a moving object, In the determination, based on the information of the moving object, for each frame, the traveling direction and position of the moving object are obtained, and using the traveling direction and position of the moving object for each obtained frame, cause the frame to determine whether there is a portion in the frame that does not overlap with another frame other than the frame. The program according to claim 7.

9. In the determination, for each frame, using the orientation and position of the moving object for each obtained frame, cause the frame to determine whether there is another frame existing around the frame, and when there is no other frame existing around the frame, cause the frame to be determined that there is a portion in the frame that does not overlap with another frame other than the frame. The program according to claim 8.

Images