Map generation device, map generation method, and program

Abstract

This map generation device 10 comprises: a data acquisition unit 11 which acquires, in units of frames, image data from depth sensors; a feature point extraction unit 12 which extracts, for each frame, feature points from the image data; a filtering unit 13 which selects, for each of the frames, a specific feature point, and excludes the selected specific feature point when the depth of the selected specific feature point is greater by a threshold value or more than a minimum value of the depths of a plurality of other feature points that are arrayed in a specific direction of the selected specific feature point and the image; and an environment map generation unit 14 which identifies a group of feature points corresponding between the frames, calculates three-dimensional coordinates of the feature points identified as the group, uses the calculation result, and generates an environment map composed of a set of the feature points identified as the group.

Description

[Technical Field]

[0001] The present disclosure relates to a map generation device and a map generation method for generating an environmental map constructed from point cloud data, and further to a program for realizing these. [Background technology]

[0002] In recent years, advances in sensing technology have led to the development of SLAM (Simultaneous Localization and Mapping). SLAM is a technology in which a mobile object equipped with a camera simultaneously estimates its own position and creates an environmental map (see, for example, Patent Document 1). SLAM eliminates the need for an autonomously traveling mobile object, such as a robot, to move randomly, but rather allows it to move along an autonomous traveling map obtained from an environmental map, thereby improving its movement efficiency.

[0003] Specifically, the moving body acquires image data output from the camera for each frame, searches for feature points in the latest frame that correspond to feature points extracted from past frames, and extracts pairs of feature points each consisting of corresponding feature points.The moving body then calculates a camera matrix for the latest frame using the pair of feature points, and calculates the three-dimensional coordinates of each pair of feature points using the camera matrix and the two-dimensional coordinates of the feature points in the frame.

[0004] After calculating the camera matrix, the mobile body calculates the camera position from the camera matrix and performs self-location estimation. Furthermore, the mobile body generates or updates an environmental map (hereinafter referred to as a "3D environmental map") composed of a 3D point cloud using the feature points whose 3D coordinates have been calculated. Thereafter, the mobile body generates a map for autonomous driving by converting the 3D environmental map into 2D, and performs autonomous driving using the map for autonomous driving.

[0005] In addition to cameras, depth sensors that can measure the distance (depth) to an object for each pixel of image data are also used. Compared to using cameras, depth sensors can increase the density of the point cloud that makes up the environmental map, enabling highly accurate autonomous driving. A specific example of a depth sensor is a LiDAR (light detection and ranging) sensor (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2020-76714 [Patent Document 2] Japanese Patent Publication No. 2021-124496 Summary of the Invention [Problem to be solved by the invention]

[0007] According to the above-mentioned SLAM, when an object is present in front of the camera, its feature points are extracted and a point cloud of the object is constructed in the environmental map. However, if there is an obstruction smaller than the object between the object and the camera, incorrect feature points may be extracted at the boundary of the area on the object that is shaded by the obstruction. In such cases, an environmental map with low accuracy is generated, resulting in a significant decrease in the accuracy of autonomous driving.

[0008] An example of an objective of the present disclosure is to suppress a decrease in accuracy of an environmental map even when an obstruction exists between a sensor and an object. [Means for solving the problem]

[0009] In order to achieve the above object, a map generating device according to one aspect of the present disclosure includes: a data acquisition unit that acquires image data output from the depth sensor on a frame-by-frame basis; a feature point extraction unit that extracts feature points from the image data for each frame; a filtering unit that selects a specific feature point from the extracted feature points for each frame, and excludes the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; an environment map generation unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and generates an environment map made up of a collection of the feature points identified as pairs using the calculation results; The present invention is characterized in that it is provided with:

[0010] In order to achieve the above object, a map generation method according to one aspect of the present disclosure includes: a data acquisition step of acquiring image data output from the depth sensor on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the identified pairs of feature points, and using the calculation results to generate an environmental map configured from a collection of the identified pairs of feature points; The present invention is characterized by having the following:

[0011] Furthermore, in order to achieve the above object, a program according to one aspect of the present disclosure includes: On the computer, a data acquisition step of acquiring image data output from the depth sensor on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the identified pairs of feature points, and using the calculation results to generate an environmental map configured from a collection of the identified pairs of feature points; The method is characterized in that: [Effects of the Invention]

[0012] As described above, according to the present disclosure, even when an obstruction exists between the sensor and the object, it is possible to suppress a decrease in the accuracy of the environmental map. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a map generating device. [Figure 2] FIG. 2 is a diagram specifically illustrating the configuration of the map generating device. [Figure 3] FIG. 3 is an explanatory diagram for explaining the function of the filtering unit. [Figure 4] FIG. 4 is a diagram showing an example of a three-dimensional environmental map. [Figure 5] FIG. 5 is a flow diagram showing the operation of the map generating device. [Figure 6] FIG. 6 is a block diagram showing an example of a computer that realizes the map generating device. DETAILED DESCRIPTION OF THE INVENTION

[0014] (Embodiment 1) A map generating device, a map generating method, and a program according to an embodiment will be described below with reference to FIGS.

[0015] [Device configuration] First, the schematic configuration of a map generating device according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the schematic configuration of a map generating device.

[0016] 1, a map generating device 10 according to an embodiment is a device for generating an environmental map constructed using point cloud data. As shown in FIG. 1, the map generating device 10 includes a data acquiring unit 11, a feature point extracting unit 12, a filtering unit 13, and an environmental map generating unit 14.

[0017] The data acquisition unit 11 acquires image data output from the depth sensor on a frame-by-frame basis. The feature point extraction unit 12 extracts feature points from the image data acquired by the data acquisition unit 11 for each frame.

[0018] The filtering unit 13 selects a specific feature point from the extracted feature points for each frame. Furthermore, if the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold or more, the filtering unit 13 excludes the selected feature point.

[0019] The environmental map generation unit 14 identifies pairs of corresponding feature points between frames, calculates the three-dimensional coordinates of the feature points identified as pairs, and uses the calculation results to generate an environmental map made up of a collection of feature points identified as pairs.

[0020] Thus, in this embodiment, the map generating device 10 selects a specific feature point, for example, a feature point that is an edge of an object, and excludes the selected feature point if the depth of the selected feature point is greater than the minimum depth of other feature points that are horizontally adjacent to the selected feature point by a certain amount. In this case, the selected feature point is a feature point that is created by the shadow of an obstruction between the sensor and the object. As a result, this embodiment can prevent a decrease in the accuracy of the environmental map even when an obstruction exists between the sensor and the object.

[0021] Next, the configuration and functions of the map generating device in the embodiment will be specifically described with reference to Figures 2 and 3. Figure 2 is a configuration diagram specifically showing the configuration of the map generating device. Figure 3 is an explanatory diagram for explaining the functions of the filtering unit.

[0022] 2, in this embodiment, the map generating device 10 is mounted on an autonomously moving body 100 such as a robot. In addition to the map generating device 10, the moving body 100 is equipped with a depth sensor 20, a control device 30, a steering device 40, and a powertrain 50.

[0023] In this embodiment, the depth sensor 20 is attached to the moving body 100 so as to capture an image in the moving direction of the moving body 100. The depth sensor 20 is a sensor capable of capturing images with depth information and outputs the image data with depth information at a set frame rate. In the example of Fig. 2, a 3D LiDAR is used as the depth sensor 20.

[0024] The control device 30 is constructed by a computer mounted on the moving body 100. The control device 30 controls the traveling direction and traveling speed of the moving body 100 using the environmental map generated by the map generating device 10. For example, the control device 30 uses the environmental map to set a route from the present to the destination, determines the traveling direction and traveling speed of the moving body 100 so that the moving body 100 moves along the set route, and controls the power train 50 and the steering device 40.

[0025] The powertrain 50 is made up of an electric motor for driving, a power transmission mechanism, etc. Tires, caterpillar tracks, etc. are connected to the powertrain 50. The powertrain 50 rotates the tires, caterpillar tracks, etc. in response to instructions from the control device 30.

[0026] The steering device 40 includes a mechanism for controlling the direction of the steering wheels of the moving body 100. The steering device 40 determines the direction of the steering wheels in response to instructions from the control device 30. The steering device 40 may also include a mechanism for controlling the direction of movement by controlling the torque of the left and right drive wheels.

[0027] In the embodiment, the map generating device 10 is constructed by a program in the embodiment described below on a computer mounted on the mobile body 100. The map generating device 10 may also be constructed by a device (for example, an electronic circuit) separate from the computer mounted on the mobile body 100.

[0028] In this embodiment, the data acquisition unit 11 acquires image data with depth information frame by frame from the depth sensor 20. The data acquisition unit 11 inputs the acquired image data (frames) to the feature point extraction unit 12 in sequence.

[0029] In this embodiment, when a frame is input, the feature point extraction unit 12 extracts feature points of an object for each frame using, for example, a general FAST algorithm. Then, the feature point extraction unit 12 inputs information specifying the feature points extracted for each frame to the filtering unit 13.

[0030] In the embodiment, when feature points for each frame are input, the filtering unit 13 selects a specific feature point from the extracted feature points for each frame. In the embodiment, the filtering unit 13 calculates the curvature of a trajectory connecting a feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in a specific direction of the image, and selects the feature point of interest as the specific feature point if the calculated curvature is equal to or greater than a threshold. In other words, as shown in FIG. 3, the filtering unit 13 selects feature points on parts that form the edges of an object as specific feature points (hereinafter referred to as "edge feature points").

[0031] In the example of Figure 3, feature points indicated by "●" are selected. Feature points indicated by "●" are feature points (edge ​​feature points) of the edge of the object, and the curvature there is above a threshold. Feature points indicated by "○" are feature points of the plane of the object (hereinafter referred to as "plane feature points"). In Figure 3, 21 indicates the imaging area of ​​the depth sensor 20. 22 indicates a wall existing in real space, and 23 indicates an obstruction existing between the depth sensor 20 and the wall (object) 22.

[0032] Specifically, the specific direction is the scanning direction of the depth sensor 20 near the feature point of interest. If the depth sensor 20 is a LiDAR, the scanning direction is determined by the specifications of the LiDAR. The filtering unit 13 first focuses on one of the extracted feature points and identifies multiple other feature points aligned with this feature point (hereinafter referred to as the "feature point of interest") in the scanning direction. For example, the filtering unit 13 identifies five feature points before and five feature points after the feature point of interest (a total of 10 feature points) in the scanning direction.

[0033] Next, the filtering unit 13 calculates the curvature c of the locus connecting the feature point of interest and a plurality of other feature points, for example, using the following equation 1. In the following equation 1, i indicates the feature point of interest, j indicates the other feature points adjacent to the feature point of interest, and S indicates the number of the other feature points adjacent to the feature point of interest. X L (k、i) denotes the coordinate of the feature point i of interest in frame k, and X L (k、j) denotes the coordinates of feature point j in frame k.

[0034]

number

[0035] If the calculated value of the curvature c is equal to or greater than a threshold value (curvature c i≧threshold), the feature point of interest is selected as an edge feature point. Furthermore, the filtering unit 13 can select an edge feature point only if the number of feature points in the same frame whose curvature c is equal to or greater than the threshold does not exceed an upper limit. This is because if the number of feature points exceeds the upper limit, the feature point of interest is likely not a feature point created by a shadow.

[0036] Next, the filtering unit 13 calculates the depth d of each selected edge feature point. i is greater than the minimum value of the depths of the other feature points aligned in the scanning direction by a threshold or more. Specifically, the filtering unit 13 determines whether the following equation 2 is satisfied. If the result of the determination is that the depth is greater than the threshold or more (depth d i -MIN(d j )≧threshold), the filtering unit 13 removes the selected edge feature point.

[0037] The environmental map generation unit 14 first uses the feature points of each frame that were not excluded by the filtering unit 13 to associate, for each frame, the feature points extracted from that frame with the feature points extracted from previous frames, thereby identifying pairs of feature points.

[0038] Next, for each frame, the environment map generation unit 14 calculates a camera matrix for that frame using the set of feature points identified in that frame.The environment map generation unit 14 then calculates the three-dimensional coordinates of the feature points using the calculated camera matrix and the two-dimensional coordinates of the feature points in the frame.

[0039] Next, after calculating the camera matrix, the environmental map generation unit 14 calculates the camera position from the camera matrix and performs self-location estimation. Furthermore, the mobile object generates or updates an environmental map (hereinafter referred to as "3D environmental map") composed of a 3D point cloud using the feature points whose 3D coordinates have been calculated. Figure 4 is a diagram showing an example of a 3D environmental map.

[0040] The environmental map generation unit 14 also inputs the generated three-dimensional environmental map to the control device 30. Furthermore, the environmental map generation unit 14 can also generate a two-dimensional environmental map by converting the three-dimensional environmental map into two dimensions. In this case, the environmental map generation unit 14 passes the generated two-dimensional environmental map to the control device 30. The control device 30 then uses the obtained environmental map to control the traveling direction and moving speed of the moving object.

[0041] In the above example, a 3D LiDAR capable of measuring depth in a 3D space is used as depth sensor 20, but in the embodiment, depth sensor 20 is not limited to this. In the embodiment, other examples of depth sensor 20 include a 2D LiDAR with a fixed angle of view that measures depth in a specific plane, and a TOF (Time Of Flight) camera. Furthermore, the LiDAR used as depth sensor 20 may be a fixed type with a fixed angle of view, or a rotating type with an open angle of view. [Device operation] Next, the operation of the map generation device 10 in the embodiment will be described with reference to FIG. 5. FIG. 5 is a flow diagram showing the operation of the map generation device. In the following description, reference will be made to FIGS. 1 to 4 as appropriate. In addition, in the embodiment, a map generation method is implemented by operating the map generation device 10. Therefore, the description of the map generation method in the embodiment will be replaced by the following description of the operation of the map generation device 10.

[0042] 5, first, when depth-added image data is output frame by frame from the depth sensor 20, the data acquisition unit 11 acquires the output image data (step A1). Furthermore, when the data acquisition unit 11 acquires image data for the set number of frames, it inputs the acquired image data to the feature point extraction unit 12.

[0043] Next, when the image data acquired in step A1 is input, the feature point extraction unit 12 extracts feature points for each frame (step A2). Furthermore, the feature point extraction unit 12 outputs the extracted feature points for each frame to the filtering unit 13.

[0044] Next, when the feature points for each frame are input in step A2, the filtering unit 13 selects edge feature points from the extracted feature points for each frame (step A3).

[0045] In step A3, the filtering unit 13 calculates, for each frame, the curvature of a trajectory connecting a feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the scanning direction of the depth sensor 20, and selects the feature point of interest as an edge feature point if the calculated curvature is greater than or equal to a threshold value.

[0046] Next, for each edge feature point selected in step A3, the filtering unit 13 determines whether the depth of the edge feature point is greater than the minimum value of the depths of other feature points aligned with the edge feature point in the scanning direction by a threshold value or more (step A4).

[0047] Next, the filtering unit 13 removes, for each frame, edge feature points determined to be equal to or greater than the threshold in step A4 (step A5).

[0048] Next, the environment map generation unit 14 uses the feature points that were not excluded in step A5 to associate, for each frame, the feature points extracted from that frame with the feature points extracted from previous frames, and identifies pairs of feature points (step A6).

[0049] Next, the environmental map generation unit 14 calculates the three-dimensional coordinates of the feature points identified as pairs in step A6, and uses the calculation results to generate an environmental map made up of a collection of the feature points identified as pairs (step A7).

[0050] Thereafter, the environmental map generating unit 14 inputs the environmental map generated in step A7 to the control device 30. As a result, the control device 30 uses the obtained environmental map to control the traveling direction and moving speed of the moving object.

[0051] As described above, in the embodiment, feature points caused by the shadow of an obstruction between the depth sensor and the object are eliminated, thereby preventing the extraction of an incorrect set of feature points. Therefore, according to the embodiment, even if an obstruction exists between the depth sensor and the object, it is possible to suppress a decrease in the accuracy of the environment map.

[0052] [program] The program in the embodiment may be any program that causes a computer to execute steps A1 to A7 shown in Fig. 5. By installing and executing this program in a computer, the map generation device 10 and map generation method in the embodiment can be realized. In this case, the processor of the computer functions as a data acquisition unit 11, a feature point extraction unit 12, a filtering unit 13, and an environmental map generation unit 14, and performs processing.

[0053] The computer may be a computer mounted on the mobile object 100. Other examples of the computer include a general-purpose PC, a smartphone, and a tablet terminal device.

[0054] The program in the embodiment may be executed by a computer system constructed by a plurality of computers, in which case, for example, each computer may function as one of the data acquisition unit 11, feature point extraction unit 12, filtering unit 13, and environmental map generation unit 14.

[0055] [Physical configuration] Here, a computer that realizes the map generating device 10 by executing a program in the embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of a computer that realizes the map generating device.

[0056] 7, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0057] Furthermore, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this aspect, the GPU or FPGA can execute the programs in the embodiments.

[0058] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0059] The program in the embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

[0060] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0061] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0062] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0063] Note that the map generating device 10 in the embodiments can be realized not by a computer on which a program is installed, but by hardware corresponding to each unit, such as an electronic circuit. Furthermore, the map generating device 10 may be partially realized by a program and the remaining unit by hardware. In the embodiments, the computer is not limited to the computer shown in FIG. 7.

[0064] Some or all of the above-described embodiments can be expressed by (Supplementary Note 1) to (Supplementary Note 12) described below, but are not limited to the following descriptions.

[0065] (Appendix 1) a data acquisition unit that acquires image data output from the depth sensor on a frame-by-frame basis; a feature point extraction unit that extracts feature points from the image data for each frame; a filtering unit that selects a specific feature point from the extracted feature points for each frame, and excludes the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; an environment map generation unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and generates an environment map made up of a collection of the feature points identified as pairs using the calculation results; Equipped with A map generating device characterized by:

[0066] (Appendix 2) the filtering unit calculates a curvature of a locus connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction, and selects the feature point of interest as the specific feature point when the calculated curvature is equal to or greater than a threshold; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold or more; 2. The map generating device of claim 1.

[0067] (Appendix 3) the specific direction is a scanning direction of the depth sensor, the filtering unit calculates a curvature of a locus connecting the feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction, with the feature point of interest as a center; 3. The map generating device of claim 2.

[0068] (Appendix 4) The depth sensor is mounted on a moving body. 4. A map generating device according to any one of appendices 1 to 3.

[0069] (Appendix 5) a data acquisition step of acquiring image data output from the depth sensor on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the identified pairs of feature points, and using the calculation results to generate an environmental map configured from a collection of the identified pairs of feature points; having A map generating method comprising:

[0070] (Appendix 6) in the filtering step, a curvature of a locus connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction is calculated, and if the calculated curvature is equal to or greater than a threshold, the feature point of interest is selected as the specific feature point; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold or more; 6. The map generation method according to claim 5.

[0071] (Appendix 7) the specific direction is a scanning direction of the depth sensor, In the filtering step, a curvature of a locus connecting the feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction with the feature point of interest as a center is calculated. 7. The map generation method according to claim 6.

[0072] (Appendix 8) The depth sensor is mounted on a moving body. 8. A map generation method according to any one of appendixes 5 to 7.

[0073] (Appendix 9) On the computer, a data acquisition step of acquiring image data output from the depth sensor on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the identified pairs of feature points, and using the calculation results to generate an environmental map configured from a collection of the identified pairs of feature points; A program that executes.

[0074] (Appendix 10) in the filtering step, a curvature of a locus connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction is calculated, and if the calculated curvature is equal to or greater than a threshold, the feature point of interest is selected as the specific feature point; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold or more; 10. The program described in Appendix 9.

[0075] (Appendix 11) the specific direction is a scanning direction of the depth sensor, In the filtering step, a curvature of a locus connecting the feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction with the feature point of interest as a center is calculated. 10. The program described in Appendix 10.

[0076] (Appendix 12) The depth sensor is mounted on a moving body. 12. The program of any one of appendices 9 to 11.

[0077] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0078] This application claims priority based on Japanese Patent Application No. 2022-91129, filed on June 3, 2022, the disclosure of which is incorporated herein by reference in its entirety. [Industrial Applicability]

[0079] As described above, according to the present disclosure, even when an obstruction exists between the sensor and the object, it is possible to suppress a decrease in the accuracy of the environmental map. The present disclosure is useful in fields where SLAM is used. [Explanation of symbols]

[0080] 10 Map Generation Device 11 Data Acquisition Section 12 Feature point extraction unit 13 Filtering section 14 Environmental map generation unit 20 Depth Sensor 30 Control device 40 Steering Gear 50 Powertrain 100 Mobile 110 Computer 111 CPU 112 main memory 113 Storage device 114 Input Interface 115 Display Controller 116 Data Reader / Writer 117 Communication Interface 118 Input Devices 119 Display Device 120 Recording Media 121 Bus

Claims

1. a data acquisition unit that acquires image data output from the depth sensor on a frame-by-frame basis; a feature point extraction unit that extracts feature points from the image data for each frame; a filtering unit that selects a specific feature point from the extracted feature points for each frame, and excludes the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; an environment map generation unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and generates an environment map made up of a collection of the feature points identified as pairs using the calculation results; Equipped with A map generating device characterized by:

2. the filtering unit calculates a curvature of a locus connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction, and selects the feature point of interest as the specific feature point when the calculated curvature is equal to or greater than a threshold; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold or more; The map generating device according to claim 1 .

3. the specific direction is a scanning direction of the depth sensor, the filtering unit calculates a curvature of a locus connecting the feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction, with the feature point of interest as a center; The map generating device according to claim 2 .

4. The depth sensor is mounted on a moving body. The map generating device according to claim 1 .

5. The image data output from the depth sensor is acquired frame by frame, extracting feature points from the image data for each frame; selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; Identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environment map composed of the sets of feature points identified as pairs using the calculation results. A map generating method comprising:

6. in the selection, a curvature of a locus connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction is calculated, and if the calculated curvature is equal to or greater than a threshold, the feature point of interest is selected as the specific feature point; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold or more; The map generating method according to claim 5 .

7. the specific direction is a scanning direction of the depth sensor, In the calculation, a curvature of a locus that connects the feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the specific direction with the feature point of interest as a center is calculated. The map generating method according to claim 6.

8. The depth sensor is mounted on a moving body. The map generating method according to claim 5 .

9. On the computer, The image data output from the depth sensor is acquired frame by frame, extracting feature points from the image data for each frame; selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; Identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environmental map composed of the sets of feature points identified as pairs using the calculation results. program.

10. in the selection, a curvature of a locus connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction is calculated, and if the calculated curvature is equal to or greater than a threshold, the feature point of interest is selected as the specific feature point; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold or more; The program according to claim 9.

11. the specific direction is a scanning direction of the depth sensor, In the calculation, a curvature of a locus that connects the feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the specific direction with the feature point of interest as a center is calculated. The program according to claim 10.

12. The depth sensor is mounted on a moving body. The program according to claim 9.

Images