Map generation device, map generation method, and program

Abstract

This map generation device 10 comprises: a data acquisition unit 11 which acquires, in units of frames, image data from depth sensors; a feature point extraction unit 12 which extracts, for each frame, feature points from the image data; a filtering unit 13 which selects, for each of the frames, a specific feature point, and excludes the selected specific feature point when the depth of the selected specific feature point is greater by a threshold value or more than a minimum value of the depths of a plurality of other feature points that are arrayed in a specific direction of the selected specific feature point and the image; and an environment map generation unit 14 which identifies a group of feature points corresponding between the frames, calculates three-dimensional coordinates of the feature points identified as the group, uses the calculation result, and generates an environment map composed of a set of the feature points identified as the group.

Description

Map generation device, map generation method, and computer-readable recording medium

[0001] The present disclosure relates to a map generation device and a map generation method for generating an environmental map constructed from point cloud data, and further to a program for realizing these.

[0002] In recent years, advances in sensing technology have attracted attention to SLAM (Simultaneous Localization and Mapping). SLAM is a technology in which a mobile object equipped with a camera simultaneously estimates its own position and creates an environmental map (see, for example, Patent Document 1). With SLAM, a mobile object such as an autonomously traveling robot does not need to move randomly but can move along an autonomous traveling map obtained from an environmental map, thereby improving its movement efficiency.

[0003] Specifically, the moving body acquires image data output from the camera for each frame, searches for feature points in the latest frame that correspond to feature points extracted from past frames, and extracts pairs of feature points each consisting of corresponding feature points.The moving body then calculates a camera matrix for the latest frame using the pair of feature points, and calculates three-dimensional coordinates of each pair of feature points using the camera matrix and the two-dimensional coordinates of the feature points in the frame.

[0004] After calculating the camera matrix, the mobile body calculates the camera position from the camera matrix and performs self-localization. Furthermore, the mobile body generates or updates an environmental map (hereinafter referred to as a "3D environmental map") composed of a 3D point cloud using the feature points whose 3D coordinates have been calculated. The mobile body then converts the 3D environmental map into a 2D map to generate a map for autonomous driving, and performs autonomous driving using the map for autonomous driving.

[0005] In addition to cameras, depth sensors that can measure the distance (depth) to an object for each pixel of image data are also used. The use of depth sensors allows for a higher density of the point cloud that constitutes the environmental map compared to when cameras are used, enabling highly accurate autonomous driving. A specific example of a depth sensor is a LiDAR (light detection and ranging) sensor (see, for example, Patent Document 2).

[0006] JP 2020-76714 A JP 2021-124496 A

[0007] According to the above-mentioned SLAM, when an object is present in front of the camera, its feature points are extracted and a point cloud of the object is constructed in the environmental map. However, if an obstruction smaller than the object exists between the object and the camera, incorrect feature points may be extracted at the boundary of the area on the object that is shaded by the obstruction. In such cases, an environmental map with low accuracy is generated, resulting in a significant decrease in the accuracy of autonomous driving.

[0008] An example of an objective of the present disclosure is to suppress a decrease in accuracy of an environmental map even when an obstruction exists between a sensor and an object.

[0009] In order to achieve the above object, a map generation device according to one aspect of the present disclosure comprises: a data acquisition unit that acquires image data output from a depth sensor on a frame-by-frame basis; a feature point extraction unit that extracts feature points from the image data for each frame; a filtering unit that selects a specific feature point from the extracted feature points for each frame, and excludes the selected specific feature point if the depth of the selected specific feature point is greater than the minimum depth of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; and an environmental map generation unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and uses the calculation results to generate an environmental map composed of a collection of the feature points identified as pairs.

[0010] Furthermore, in order to achieve the above object, a map generation method according to one aspect of the present disclosure is characterized by comprising: a data acquisition step of acquiring image data output from a depth sensor on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point if the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; and an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and using the calculation results to generate an environmental map composed of a set of the feature points identified as pairs.

[0011] Furthermore, in order to achieve the above object, a program according to one aspect of the present disclosure causes a computer to execute the following steps: a data acquisition step of acquiring image data output from a depth sensor on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point if the depth of the selected specific feature point is greater than the minimum value of the depths of multiple feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; and an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and using the calculation results, generating an environmental map composed of a collection of the feature points identified as pairs.

[0012] As described above, according to the present disclosure, even when an obstruction exists between the sensor and the object, it is possible to suppress a decrease in the accuracy of the environmental map.

[0013] FIG. 1 is a configuration diagram showing a schematic configuration of a map generating device. FIG. 2 is a configuration diagram showing a specific configuration of the map generating device. FIG. 3 is an explanatory diagram for explaining the function of a filtering unit. FIG. 4 is a diagram showing an example of a three-dimensional environment map. FIG. 5 is a flow diagram showing the operation of the map generating device. FIG. 6 is a block diagram showing an example of a computer that realizes the map generating device.

[0014] First Embodiment A map generating device, a map generating method, and a program according to an embodiment will be described below with reference to FIGS.

[0015] [Device Configuration] First, the schematic configuration of a map generating device according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the schematic configuration of a map generating device.

[0016] 1, a map generating device 10 according to an embodiment is a device for generating an environmental map constructed using point cloud data. As shown in FIG. 1, the map generating device 10 includes a data acquiring unit 11, a feature point extracting unit 12, a filtering unit 13, and an environmental map generating unit 14.

[0017] The data acquisition unit 11 acquires image data output from the depth sensor on a frame-by-frame basis. The feature point extraction unit 12 extracts feature points from the image data acquired by the data acquisition unit 11 for each frame.

[0018] The filtering unit 13 selects a specific feature point from the extracted feature points for each frame. If the depth of the selected specific feature point is greater than the minimum value of the depths of multiple other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold or more, the filtering unit 13 excludes the selected feature point.

[0019] The environmental map generation unit 14 identifies pairs of corresponding feature points between frames, calculates the three-dimensional coordinates of the feature points identified as pairs, and uses the calculation results to generate an environmental map made up of a collection of feature points identified as pairs.

[0020] Thus, in this embodiment, the map generating device 10 selects a specific feature point, for example, a feature point that is an edge of an object, and excludes the selected feature point if the depth of the selected feature point is greater than the minimum depth of other feature points that are horizontally adjacent to the selected feature point by a certain amount. In this case, the selected feature point is a feature point that is created by the shadow of an obstruction between the sensor and the object. As a result, this embodiment can prevent a decrease in the accuracy of the environmental map even when an obstruction exists between the sensor and the object.

[0021] Next, the configuration and functions of the map generating device in the embodiment will be specifically described with reference to Figures 2 and 3. Figure 2 is a block diagram specifically showing the configuration of the map generating device. Figure 3 is an explanatory diagram for explaining the functions of the filtering unit.

[0022] 2 , in this embodiment, the map generating device 10 is mounted on an autonomously moving body 100 such as a robot. In addition to the map generating device 10, the moving body 100 is equipped with a depth sensor 20, a control device 30, a steering device 40, and a powertrain 50.

[0023] In this embodiment, the depth sensor 20 is attached to the moving body 100 so as to capture an image in the moving direction of the moving body 100. The depth sensor 20 is a sensor capable of capturing depth-based images and outputs the depth-based image data at a set frame rate. In the example of Fig. 2, a three-dimensional LiDAR is used as the depth sensor 20.

[0024] The control device 30 is constructed by a computer mounted on the mobile body 100. The control device 30 controls the traveling direction and traveling speed of the mobile body 100 using the environmental map generated by the map generation device 10. For example, the control device 30 uses the environmental map to set a route from the present to the destination, determines the traveling direction and traveling speed of the mobile body 100 so that the mobile body 100 moves along the set route, and controls the power train 50 and the steering device 40.

[0025] The power train 50 is made up of an electric motor for driving, a power transmission mechanism, etc. Tires, caterpillar tracks, etc. are connected to the power train 50. The power train 50 rotates the tires, caterpillar tracks, etc. in response to instructions from the control device 30.

[0026] The steering device 40 includes a mechanism for controlling the direction of the steering wheels of the mobile body 100. The steering device 40 determines the direction of the steering wheels in response to instructions from the control device 30. The steering device 40 may also include a mechanism for controlling the direction of travel by controlling the torque of the left and right drive wheels.

[0027] In the embodiment, the map generating device 10 is constructed by a program in the embodiment described below on a computer mounted on the mobile body 100. The map generating device 10 may also be constructed by a device (for example, an electronic circuit) separate from the computer mounted on the mobile body 100.

[0028] In this embodiment, the data acquisition unit 11 acquires depth-added image data frame by frame from the depth sensor 20. The data acquisition unit 11 sequentially inputs the acquired image data (frames) to the feature point extraction unit 12.

[0029] In this embodiment, when frames are input, the feature point extraction unit 12 extracts feature points of an object for each frame using, for example, a general FAST algorithm. Then, the feature point extraction unit 12 inputs information specifying the feature points extracted for each frame to the filtering unit 13.

[0030] In the embodiment, when feature points for each frame are input, the filtering unit 13 selects a specific feature point from the extracted feature points for each frame. In the embodiment, the filtering unit 13 calculates the curvature of a trajectory connecting a feature point of interest and multiple other feature points that are aligned with the feature point of interest in a specific direction of the image, and selects the feature point of interest as the specific feature point if the calculated curvature is equal to or greater than a threshold. In other words, as shown in FIG. 3 , the filtering unit 13 selects feature points on parts that form the edges of an object as specific feature points (hereinafter referred to as "edge feature points").

[0031] In the example of FIG. 3, feature points indicated by "●" are selected. Feature points indicated by "●" are feature points of the edge portion of the object (edge ​​feature points), and the curvature there is equal to or greater than a threshold. Feature points indicated by "○" are feature points of the plane portion of the object (hereinafter referred to as "plane feature points"). In FIG. 3, 21 indicates the imaging area of ​​the depth sensor 20. 22 indicates a wall existing in real space, and 23 indicates an obstruction existing between the depth sensor 20 and the wall (object) 22.

[0032] Specifically, the specific direction is the scanning direction of the depth sensor 20 near the feature point of interest. If the depth sensor 20 is a LiDAR, the scanning direction is determined by the specifications of the LiDAR. The filtering unit 13 first focuses on one of the extracted feature points and identifies multiple other feature points aligned with this feature point (hereinafter referred to as the "feature point of interest") in the scanning direction. For example, the filtering unit 13 identifies five feature points before and five feature points after the feature point of interest (a total of 10 feature points) in the scanning direction.

[0033] Next, the filtering unit 13 calculates the curvature c of a trajectory connecting the feature point of interest and a plurality of other feature points, for example, using the following equation 1. In the following equation 1, i indicates the feature point of interest, j indicates the other feature points adjacent to the feature point of interest, and S indicates the number of the other feature points adjacent to the feature point of interest. X Ｌ （ｋ、ｉ） denotes the coordinates of the feature point i of interest in frame k, and X Ｌ （ｋ、ｊ） denotes the coordinates of feature point j in frame k.

[0034]

[0035] If the calculated value of the curvature c is equal to or greater than a threshold value (curvature c ｉ≧threshold), the filtering unit 13 selects the feature point of interest as an edge feature point. Furthermore, the filtering unit 13 can select an edge feature point only if the number of feature points in the same frame whose curvature c is equal to or greater than the threshold does not exceed an upper limit. This is because if the number of feature points exceeds the upper limit, the feature point of interest is likely not a feature point created by a shadow.

[0036] Next, for each selected edge feature point, the filtering unit 13 calculates the depth d ｉ is greater than the minimum value of the depths of the other feature points aligned in the scanning direction by a threshold or more. Specifically, the filtering unit 13 determines whether the following equation 2 is satisfied. If the result of the determination is greater than the threshold or more (depth d ｉ -MIN(d ｊ )≧threshold), the filtering unit 13 removes the selected edge feature point.

[0037] The environment map generation unit 14 first uses the feature points of each frame that were not excluded by the filtering unit 13 to associate, for each frame, the feature points extracted from that frame with the feature points extracted from previous frames, and identifies pairs of feature points.

[0038] Next, for each frame, the environment map generation unit 14 calculates a camera matrix for that frame using the set of feature points identified in that frame. Then, the environment map generation unit 14 calculates the three-dimensional coordinates of the feature points using the calculated camera matrix and the two-dimensional coordinates of the feature points in the frame.

[0039] Next, after calculating the camera matrix, the environmental map generation unit 14 calculates the camera position from the camera matrix and performs self-location estimation. Furthermore, the mobile object generates or updates an environmental map (hereinafter referred to as a "3D environmental map") composed of a 3D point cloud using the feature points whose 3D coordinates have been calculated. Figure 4 is a diagram showing an example of a 3D environmental map.

[0040] The environmental map generation unit 14 also inputs the generated three-dimensional environmental map to the control device 30. Furthermore, the environmental map generation unit 14 can also generate a two-dimensional environmental map by converting the three-dimensional environmental map into two dimensions. In this case, the environmental map generation unit 14 passes the generated two-dimensional environmental map to the control device 30. The control device 30 then uses the obtained environmental map to control the traveling direction and moving speed of the moving object.

[0041] In the above example, a 3D LiDAR capable of measuring depth in a three-dimensional space is used as the depth sensor 20. However, in the embodiment, the depth sensor 20 is not limited to this. In the embodiment, other examples of the depth sensor 20 include a 2D LiDAR with a fixed angle of view that measures depth in a specific plane, and a TOF (Time of Flight) camera. The LiDAR used as the depth sensor 20 may be a fixed type with a fixed angle of view, or a rotating type with an open angle of view. [Device Operation] Next, the operation of the map generation device 10 in the embodiment will be described using FIG. 5. FIG. 5 is a flow diagram showing the operation of the map generation device. In the following description, reference will be made to FIGS. 1 to 4 as appropriate. In the embodiment, a map generation method is implemented by operating the map generation device 10. Therefore, the description of the map generation method in the embodiment will be replaced with the following description of the operation of the map generation device 10.

[0042] 5 , first, when depth-added image data is output frame by frame from the depth sensor 20, the data acquisition unit 11 acquires the output image data (step A1). Furthermore, when the data acquisition unit 11 acquires image data for the set number of frames, the data acquisition unit 11 inputs the acquired image data to the feature point extraction unit 12.

[0043] Next, when the image data acquired in step A1 is input, the feature point extraction unit 12 extracts feature points for each frame (step A2). The feature point extraction unit 12 also outputs the extracted feature points for each frame to the filtering unit 13.

[0044] Next, when the feature points for each frame are input in step A2, the filtering unit 13 selects edge feature points from the extracted feature points for each frame (step A3).

[0045] In step A3, the filtering unit 13 calculates, for each frame, the curvature of the trajectory connecting the feature point of interest and multiple other feature points that are aligned with the feature point of interest in the scanning direction of the depth sensor 20, and if the calculated curvature is greater than or equal to a threshold value, selects the feature point of interest as an edge feature point.

[0046] Next, for each edge feature point selected in step A3, the filtering unit 13 determines whether the depth of the edge feature point is greater than the minimum value of the depths of the other feature points aligned with the edge feature point in the scanning direction by a threshold value or more (step A4).

[0047] Next, the filtering unit 13 removes, for each frame, edge feature points determined to be greater than or equal to the threshold value in step A4 (step A5).

[0048] Next, the environment map generation unit 14 uses the feature points that were not excluded in step A5 to associate, for each frame, the feature points extracted from that frame with the feature points extracted from previous frames, and identifies pairs of feature points (step A6).

[0049] Next, the environmental map generation unit 14 calculates the three-dimensional coordinates of the feature points identified as a group in step A6, and uses the calculation results to generate an environmental map composed of a collection of feature points identified as a group (step A7).

[0050] Thereafter, the environmental map generating unit 14 inputs the environmental map generated in step A7 to the control device 30. As a result, the control device 30 uses the obtained environmental map to control the traveling direction and moving speed of the moving object.

[0051] As described above, in the embodiment, feature points caused by the shadow of an obstruction between the depth sensor and the object are eliminated, thereby preventing the extraction of an incorrect set of feature points. Therefore, according to the embodiment, even if an obstruction exists between the depth sensor and the object, it is possible to suppress a decrease in the accuracy of the environment map.

[0052] [Program] The program in the embodiment may be any program that causes a computer to execute steps A1 to A7 shown in Figure 5. By installing and executing this program on a computer, the map generation device 10 and map generation method in the embodiment can be realized. In this case, the processor of the computer functions as the data acquisition unit 11, feature point extraction unit 12, filtering unit 13, and environmental map generation unit 14 and performs processing.

[0053] The computer may be a computer mounted on the mobile object 100. Other examples of the computer include a general-purpose PC, a smartphone, and a tablet terminal device.

[0054] The program in the embodiment may be executed by a computer system constructed by a plurality of computers, in which case, for example, each computer may function as one of the data acquisition unit 11, feature point extraction unit 12, filtering unit 13, and environment map generation unit 14.

[0055] [Physical Configuration] A computer that implements the map generating device 10 by executing a program according to the embodiment will now be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of a computer that implements the map generating device.

[0056] 7, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0057] Furthermore, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this aspect, the GPU or FPGA can execute the programs in the embodiments.

[0058] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0059] The program in the embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

[0060] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0061] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0062] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0063] Note that the map generation device 10 in the embodiment can be realized not by a computer on which a program is installed, but by hardware corresponding to each unit, such as an electronic circuit. Furthermore, the map generation device 10 may be partially realized by a program and the remaining unit by hardware. In the embodiment, the computer is not limited to the computer shown in FIG. 7.

[0064] Some or all of the above-described embodiments can be expressed by (Supplementary Note 1) to (Supplementary Note 12) described below, but are not limited to the following descriptions.

[0065] (Supplementary Note 1) A map generating device comprising: a data acquiring unit that acquires image data output from a depth sensor on a frame-by-frame basis; a feature point extracting unit that extracts feature points from the image data for each frame; a filtering unit that selects a specific feature point from the extracted feature points for each frame, and excludes the selected specific feature point if the depth of the selected specific feature point is greater than the minimum value of the depths of multiple other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; and an environmental map generating unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and uses the calculation results to generate an environmental map made up of a collection of the feature points identified as pairs.

[0066] (Supplementary Note 2) The map generating device according to Supplementary Note 1, wherein the filtering unit calculates a curvature of a trajectory connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction, and selects the feature point of interest as the specific feature point if the calculated curvature is equal to or greater than a threshold, and excludes the selected specific feature point if a depth of the selected specific feature point is greater than the minimum value of depths of the other plurality of feature points by equal to or greater than a threshold.

[0067] (Supplementary Note 3) The map generating device according to Supplementary Note 2, wherein the specific direction is a scanning direction of the depth sensor, and the filtering unit calculates a curvature of a trajectory connecting the feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the specific direction, with the feature point of interest at the center.

[0068] (Supplementary Note 4) The map generating device according to any one of Supplementary Notes 1 to 3, wherein the depth sensor is mounted on a moving body.

[0069] (Supplementary Note 5) A map generation method comprising: a data acquisition step of acquiring image data output from a depth sensor on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point if the depth of the selected specific feature point is greater than the minimum value of the depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; and an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and using the calculation results to generate an environmental map made up of a collection of the feature points identified as pairs.

[0070] (Supplementary Note 6) The map generating method according to Supplementary Note 5, wherein in the filtering step, a curvature of a trajectory connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction is calculated, and if the calculated curvature is equal to or greater than a threshold, the feature point of interest is selected as the specific feature point, and if a depth of the selected specific feature point is greater than the minimum value of depths of the other plurality of feature points by equal to or greater than a threshold, the selected specific feature point is excluded.

[0071] (Supplementary Note 7) The map generation method according to Supplementary Note 6, wherein the specific direction is a scanning direction of the depth sensor, and the filtering step calculates a curvature of a trajectory connecting the feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the specific direction, with the feature point of interest at the center.

[0072] (Supplementary Note 8) The map generating method according to any one of Supplementary Notes 5 to 7, wherein the depth sensor is mounted on a moving body.

[0073] (Supplementary Note 9) A program causing a computer to execute the following steps: a data acquisition step of acquiring image data output from a depth sensor on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point if the depth of the selected specific feature point is greater than the minimum value of the depths of multiple other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; and an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and using the calculation results, generating an environmental map made up of a collection of the feature points identified as pairs.

[0074] (Supplementary Note 10) The program according to Supplementary Note 9, wherein in the filtering step, a curvature of a trajectory connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction is calculated, and if the calculated curvature is equal to or greater than a threshold, the feature point of interest is selected as the specific feature point, and if a depth of the selected specific feature point is greater than the minimum value of depths of the other plurality of feature points by equal to or greater than a threshold, the selected specific feature point is excluded.

[0075] (Supplementary Note 11) The program according to Supplementary Note 10, wherein the specific direction is a scanning direction of the depth sensor, and the filtering step calculates a curvature of a trajectory connecting the feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the specific direction, with the feature point of interest at the center.

[0076] (Supplementary Note 12) The program according to any one of Supplementary Notes 9 to 11, wherein the depth sensor is mounted on a moving object.

[0077] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0078] This application claims priority based on Japanese Patent Application No. 2022-91129, filed on June 3, 2022, the disclosure of which is incorporated herein in its entirety by reference.

[0079] As described above, according to the present disclosure, even when an obstruction exists between the sensor and the object, it is possible to suppress a decrease in the accuracy of the environmental map. The present disclosure is useful in fields where SLAM is used.

[0080] REFERENCE SIGNS LIST 10 Map generation device 11 Data acquisition unit 12 Feature point extraction unit 13 Filtering unit 14 Environmental map generation unit 20 Depth sensor 30 Control device 40 Steering device 50 Power train 100 Mobile body 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. A data acquisition unit that acquires image data output from the depth sensor on a frame-by-frame basis; A feature point extraction unit that extracts feature points from the image data for each frame; a filtering unit that selects a specific feature point from the extracted feature points for each frame, and excludes the selected specific feature point when a depth of the selected specific feature point is greater than a minimum value of depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; an environment map generating unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and generates an environment map composed of a collection of the feature points identified as pairs using a calculation result; Equipped with A map generating device comprising:

2. the filtering unit calculates a curvature of a trajectory connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction, and selects the feature point of interest as the specific feature point when the calculated curvature is equal to or greater than a threshold; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold value or more; The map generating device according to claim 1 .

3. the specific direction is a scanning direction of the depth sensor, the filtering unit calculates a curvature of a locus that connects the feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the specific direction with the feature point of interest as a center; The map generating device according to claim 2 .

4. The depth sensor is mounted on a moving body. The map generating device according to claim 1 .

5. The image data output from the depth sensor is acquired frame by frame. Extracting feature points from the image data for each frame; selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point when the depth of the selected specific feature point is greater than a minimum value of depths of a plurality of other feature points that are aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; Identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environment map composed of the sets of the feature points identified as pairs using the calculation results. A map generating method comprising:

6. in the selection, a curvature of a trajectory connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction is calculated, and when the calculated curvature is equal to or greater than a threshold, the feature point of interest is selected as the specific feature point; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold value or more; The map generating method according to claim 5.

7. the specific direction is a scanning direction of the depth sensor, In the calculation, a curvature of a trajectory that connects the feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the specific direction with the feature point of interest as a center is calculated. The map generating method according to claim 6.

8. The depth sensor is mounted on a moving body. The map generating method according to claim 5.

9. On the computer, The image data output from the depth sensor is acquired frame by frame. extracting feature points from the image data for each frame; selecting a specific feature point from the extracted feature points for each frame, and excluding the selected specific feature point when the depth of the selected specific feature point is greater than a minimum value of depths of a plurality of other feature points aligned with the selected specific feature point in a specific direction of the image by a threshold value or more; Identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environmental map composed of a set of the feature points identified as pairs using the calculation results. program.

10. in the selection, a curvature of a trajectory connecting a feature point of interest and a plurality of other feature points aligned with the feature point of interest in the specific direction is calculated, and when the calculated curvature is equal to or greater than a threshold, the feature point of interest is selected as the specific feature point; excluding the selected specific feature point when the depth of the selected specific feature point is greater than the minimum value of the depths of the other plurality of feature points by a threshold value or more; The program according to claim 9.

11. the specific direction is a scanning direction of the depth sensor, In the calculation, a curvature of a trajectory that connects the feature point of interest and a plurality of other feature points that are aligned with the feature point of interest in the specific direction with the feature point of interest as a center is calculated. The program according to claim 10.

12. The depth sensor is mounted on a moving body. The program according to claim 9.