Map generation device, map generation method, and program

Abstract

This map generation device 10 comprises: a data acquisition unit 11 that acquires image data in units of frames, the image data being outputted from a depth sensor for which the view angle is fixed; a feature point extraction unit 12 that extracts feature points from the image data for each frame; a filtering unit 13 that assesses, for each frame, whether a section is present in the given frame that does not overlap another frame other than the given frame, and, when it is found as a result of the assessment that such a section is present, eliminates any feature points extracted from the non-overlapping section; and an environment map generation unit 14 that identifies a set of feature points that correspond between the frames, furthermore calculates three-dimensional coordinates of the feature points identified as the set, and generates, using the result of calculation, an environment map configured from aggregations of feature points identified as the set.

Description

Map generation device, map generation method, and computer-readable recording medium

[0001] The present disclosure relates to a map generation device and a map generation method for generating an environmental map constructed from point cloud data, and further to a program for realizing these.

[0002] In recent years, advances in sensing technology have attracted attention to SLAM (Simultaneous Localization and Mapping). SLAM is a technology in which a mobile object equipped with a sensor simultaneously estimates its own position and creates an environmental map (see, for example, Patent Document 1). With SLAM, a mobile object such as an autonomously traveling robot does not need to move randomly but can move along an autonomous traveling map obtained from an environmental map, thereby improving its movement efficiency.

[0003] Specifically, the moving body acquires image data output from the camera for each frame, searches for feature points in the latest frame that correspond to feature points extracted from past frames, and extracts pairs of feature points each consisting of corresponding feature points.The moving body then calculates a camera matrix for the latest frame using the pair of feature points, and calculates three-dimensional coordinates of each pair of feature points using the camera matrix and the two-dimensional coordinates of the feature points in the frame.

[0004] After calculating the camera matrix, the mobile body calculates the camera position from the camera matrix and performs self-localization. Furthermore, the mobile body generates or updates an environmental map (hereinafter referred to as a "3D environmental map") composed of a 3D point cloud using the feature points whose 3D coordinates have been calculated. The mobile body then converts the 3D environmental map into a 2D map to generate a map for autonomous driving, and performs autonomous driving using the map for autonomous driving.

[0005] In addition to cameras, depth sensors that can measure the distance (depth) to an object for each pixel of image data are also used. The use of depth sensors allows for a higher density of the point cloud that constitutes the environmental map compared to when cameras are used, enabling highly accurate autonomous driving. A specific example of a depth sensor is a LiDAR (light detection and ranging) sensor (see, for example, Patent Document 2).

[0006] Patent Document 2 discloses a three-dimensional LiDAR sensor. The three-dimensional LiDAR sensor disclosed in Patent Document 2 includes a light source, a light receiver, a mirror, and a control device. Laser light emitted from the light source is reflected by the mirror and then irradiated onto an object. The laser light reflected by the object then enters the light receiver. The control device calculates the depth to the object from the time it takes for the laser light to be emitted from the light source and incident on the light receiver.

[0007] Furthermore, the 3D LiDAR sensor disclosed in Patent Document 2 is a rotary 3D LiDAR sensor that also includes a rotation mechanism for rotating a mirror. Therefore, the 3D LiDAR sensor disclosed in Patent Document 2 enables scanning (searching for feature points) in a 360-degree range or a range close to that.

[0008] JP 2020-76714 A JP 2021-124496 A

[0009] While the above-mentioned rotational 3D LiDAR sensor is capable of scanning over a wide area, it requires a rotation mechanism, which results in high manufacturing costs. For this reason, 3D LiDAR with a fixed angle of view that does not include a rotation mechanism (hereinafter referred to as "fixed angle of view 3D LiDAR") has also been proposed for SLAM.

[0010] However, with fixed-angle 3D LiDAR, the fixed angle of view can cause a situation in which the latest frame does not overlap with previous frames in SLAM. When this happens, an incorrect set of feature points is extracted, resulting in an inaccurate environment map. This significantly reduces the accuracy of autonomous driving.

[0011] An example of an objective of the present disclosure is to improve the accuracy of an environmental map when a depth sensor with a fixed angle of view is used.

[0012] In order to achieve the above object, a map generation device according to one aspect of the present disclosure comprises: a data acquisition unit that acquires image data output from a depth sensor with a fixed angle of view on a frame-by-frame basis; a feature point extraction unit that extracts feature points from the image data for each frame; a filtering unit that determines for each frame whether or not there is a portion in the frame that does not overlap with other frames, and if the determination result indicates that there is such a portion, removes the feature points extracted from the non-overlapping portion; and an environmental map generation unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and uses the calculation results to generate an environmental map composed of a collection of the feature points identified as pairs.

[0013] Furthermore, in order to achieve the above object, a map generation method according to one aspect of the present disclosure is characterized by comprising: a data acquisition step of acquiring image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of determining, for each frame, whether or not there is a portion in the frame that does not overlap with another frame other than the frame in question, and, if the determination result indicates that there is such a portion, excluding the feature points extracted from the non-overlapping portion; and an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and using the calculation results to generate an environmental map composed of a collection of the feature points identified as pairs.

[0014] Furthermore, in order to achieve the above object, a program according to one aspect of the present disclosure causes a computer to execute the following steps: a data acquisition step of acquiring image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of determining, for each frame, whether or not there is a portion in the frame that does not overlap with other frames, and, if the determination result indicates that there is such a portion, excluding the feature points extracted from the non-overlapping portion; and an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and using the calculation results, generating an environmental map composed of a collection of the feature points identified as pairs.

[0015] As described above, according to the present disclosure, when a depth sensor with a fixed angle of view is used, the accuracy of an environmental map can be improved.

[0016] FIG. 1 is a diagram showing a schematic configuration of a map generating device. FIG. 2 is a diagram showing a specific configuration of the map generating device. FIG. 3 is an explanatory diagram for explaining the angle of view of a depth sensor. FIG. 4 is a diagram showing a determination process in a filtering unit of the map generating device. FIG. 5 is a diagram showing an example of a three-dimensional environment map. FIG. 6 is a flow diagram showing the operation of the map generating device. FIG. 7 is a block diagram showing an example of a computer that realizes the map generating device.

[0017] (Embodiments) Hereinafter, a map generating device, a map generating method, and a program according to embodiments will be described with reference to FIGS.

[0018] [Device Configuration] First, the schematic configuration of the map generating device will be described with reference to Fig. 1. Fig. 1 is a diagram showing the schematic configuration of the map generating device.

[0019] The map generating device 10 shown in Fig. 1 is a device for generating an environmental map constructed from point cloud data. As shown in Fig. 1, the map generating device 10 includes a data acquiring unit 11, a feature point extracting unit 12, a filtering unit 13, and an environmental map generating unit 14.

[0020] The data acquisition unit 11 acquires image data output from a depth sensor with a fixed angle of view on a frame-by-frame basis. The feature point extraction unit 12 extracts feature points from the image data acquired by the data acquisition unit 11 for each frame.

[0021] The filtering unit 13 determines, for each frame, whether or not there is a portion in the frame that does not overlap with another frame other than the frame itself, and if the result of the determination indicates that there is such a portion, the filtering unit 13 eliminates the feature points extracted from the non-overlapping portion.

[0022] The environment map generation unit 14 identifies pairs of corresponding feature points between frames, calculates the three-dimensional coordinates of the identified pairs of feature points, and uses the calculation results to generate an environment map made up of a collection of the identified pairs of feature points.

[0023] In this manner, in the embodiment, the map generating device 10 excludes extracted feature points in portions of each frame that do not overlap with other frames. This prevents the extraction of an incorrect set of feature points, which occurs when the angle of view of the depth sensor is fixed. In other words, according to the embodiment, the accuracy of the environmental map can be improved when a depth sensor with a fixed angle of view is used.

[0024] Next, the configuration and functions of the map generating device according to the embodiment will be specifically described with reference to Figures 2 and 3. Figure 2 is a diagram specifically showing the configuration of the map generating device. Figure 3 is an explanatory diagram for explaining the angle of view of the depth sensor.

[0025] 2 , in this embodiment, the map generating device 10 is mounted on an autonomously moving body 100 such as a robot. In addition to the map generating device 10, the moving body 100 is equipped with a depth sensor 20, a control device 30, a steering device 40, and a powertrain 50.

[0026] In this embodiment, the depth sensor 20 is attached to the moving body 100 so as to capture an image in the moving direction of the moving body 100. The depth sensor 20 is a sensor capable of capturing depth-based images and outputs the depth-based image data at a set frame rate. In the example of Fig. 2, a fixed-angle 3D LiDAR is used as the depth sensor 20.

[0027] As shown in Fig. 3, the angle of view of the depth sensor 20 is fixed, and the imaging area 21 thereof changes depending on the orientation of the moving body 100. In the example of Fig. 3, the change in the imaging area 21 from time (t-2) to time (t) is shown.

[0028] The control device 30 is constructed by a computer mounted on the mobile body 100. The control device 30 controls the traveling direction and traveling speed of the mobile body 100 using the environmental map generated by the map generation device 10. For example, the control device 30 uses the environmental map to set a route from the present to the destination, determines the traveling direction and traveling speed of the mobile body 100 so that the mobile body 100 moves along the set route, and controls the power train 50 and the steering device 40.

[0029] The power train 50 is made up of an electric motor for driving, a power transmission mechanism, etc. Tires, caterpillar tracks, etc. are connected to the power train 50. The power train 50 rotates the tires, caterpillar tracks, etc. in response to instructions from the control device 30.

[0030] The steering device 40 includes a mechanism for controlling the direction of the steering wheels of the mobile body 100. The steering device 40 determines the direction of the steering wheels in response to instructions from the control device 30. The steering device 40 may also include a mechanism for controlling the direction of travel by controlling the torque of the left and right drive wheels.

[0031] In the embodiment, the map generating device 10 is constructed by a program in the embodiment described below on a computer mounted on the mobile body 100. The map generating device 10 may also be constructed by a device (for example, an electronic circuit) separate from the computer mounted on the mobile body 100.

[0032] In this embodiment, the data acquisition unit 11 acquires depth-added image data frame by frame from the depth sensor 20. The data acquisition unit 11 sequentially inputs the acquired image data (frames) to the feature point extraction unit 12.

[0033] In this embodiment, when frames are input, the feature point extraction unit 12 extracts feature points of an object for each frame using, for example, a general FAST algorithm. Then, the feature point extraction unit 12 inputs information specifying the feature points extracted for each frame to the filtering unit 13.

[0034] In this embodiment, when feature points for each frame are input, the filtering unit 13 first determines the traveling direction and position of the moving body 100 for each input frame based on the information about the moving body 100. Specifically, the filtering unit 13 determines the position of the moving body 100 for each frame based on the information about the moving body 100 at the time the frame was output, and then determines the absolute attitude as the traveling direction. The absolute attitude is expressed by the translation vector T(x, y, z) and rotation components R (roll, pitch, yaw) of the moving body 100.

[0035] The information about the moving body 100 includes information indicating the position and direction of travel of the moving body 100. The position information may be obtained from a GPS (Global Positioning System) receiver mounted on the moving body 100, or may be obtained by self-position estimation, which will be described later. The information indicating the direction of travel can be obtained, for example, from an angular velocity sensor mounted on the moving body 100.

[0036] Next, the filtering unit 13 determines, for each frame, whether or not there is a portion in that frame that does not overlap with other frames, using the traveling direction and position of the moving body 100. In the embodiment, the filtering unit 13 determines, for each frame, whether or not there is another frame in the vicinity of that frame, using the traveling direction and position of the moving body 100. Then, if there is no other frame in the vicinity of that frame, the filtering unit 13 determines that there is a portion in that frame that does not overlap with other frames.

[0037] Specifically, the filtering unit 13 identifies frames located at the edges on the right, left, upper, or lower side of the moving object. In the example of Fig. 3, the frame output at time t is the frame located at the edge. The frame located at the edge may have a portion that does not overlap with other frames.

[0038] Next, the filtering unit 13 calculates the amount of change in the posture of the moving body 100 between the frame located at the edge and the frame outputted before it. Specifically, the time when the frame located at the edge is outputted is set to t, the time when the frame immediately before that is set to t-1, and the rotation component at each time is set to R ｔ , R ｔ－１ In this case, the amount of change γ in the attitude of the moving body is calculated by the following equation 1.

[0039] (Math. 1) γ=R ｔ -R ｔ－１

[0040] Next, the filtering unit 13 determines whether or not there is a portion of the frame located at the edge that does not overlap with another frame, based on the calculated amount of change γ in the orientation, and if the result of the determination indicates that there is such a portion, eliminates the feature points extracted from the non-overlapping portion. Here, the process of determining whether or not frames overlap will be described with reference to Figure 4. Figure 4 is a diagram showing the determination process in the filtering unit of the map generating device.

[0041] In Figure 4, 21 indicates the imaging area when the frame located at the edge is captured (at time t). In Figure 4, the depth direction is the z-axis direction, the horizontal direction of the image is the x-axis direction, and the vertical direction of the image is the y-axis direction. Furthermore, the angle of view of the depth sensor 20 is H, the extracted feature point is P(x, y, z), and the angle in the zx plane between the normal line passing through the center of the imaging surface of the depth sensor 20 and the line extending from feature point P that also passes through the center is θ.

[0042] In this case, if the angle θ of the feature point P is greater than the angle ((H / 2)-abs(γ)) obtained by subtracting the amount of change in posture γ from half the value of the angle of view H, the feature point P exists in an area where the frames do not overlap.

[0043] Therefore, the filtering unit 13 determines whether or not each feature point extracted from a frame at time t satisfies the following equation 2. If there is a feature point that satisfies the following equation 2, the filtering unit 13 determines that a frame located at an edge has a portion that does not overlap with another frame.

[0044] (Math. 2) (H / 2)-abs(γ)>θ

[0045] Thereafter, the filtering unit 13 eliminates feature points present in the corresponding portions of the frames determined not to overlap with other frames. Specifically, the filtering unit 13 eliminates feature points present in the portions of the frames located at the edges that do not overlap with other frames (feature points that satisfy the above equation 2).

[0046] The environment map generation unit 14 first uses the feature points of each frame that were not excluded by the filtering unit 13 to associate, for each frame, the feature points extracted from that frame with the feature points extracted from previous frames, and identifies pairs of feature points.

[0047] Next, for each frame, the environment map generation unit 14 calculates a camera matrix for that frame using the set of feature points identified in that frame. Then, the environment map generation unit 14 calculates the three-dimensional coordinates of the feature points using the calculated camera matrix and the two-dimensional coordinates of the feature points in the frame.

[0048] Next, after calculating the camera matrix, the environmental map generation unit 14 calculates the camera position from the camera matrix and performs self-location estimation. Furthermore, the mobile object generates or updates an environmental map (hereinafter referred to as a "3D environmental map") composed of a 3D point cloud using the feature points whose 3D coordinates have been calculated. Figure 5 is a diagram showing an example of a 3D environmental map.

[0049] The environmental map generation unit 14 also inputs the generated three-dimensional environmental map to the control device 30. Furthermore, the environmental map generation unit 14 can also generate a two-dimensional environmental map by converting the three-dimensional environmental map into two dimensions. In this case, the environmental map generation unit 14 passes the generated two-dimensional environmental map to the control device 30. The control device 30 then uses the obtained environmental map to control the traveling direction and moving speed of the moving object.

[0050] In the above example, a three-dimensional LiDAR capable of measuring depth in a three-dimensional space is used as the depth sensor 20, but in the embodiment, the depth sensor 20 is not limited to this. In the embodiment, other examples of the depth sensor 20 include a two-dimensional LiDAR with a fixed angle of view that measures depth in a specific plane, and a TOF (Time Of Flight) camera.

[0051] [Device Operation] Next, the operation of the map generation device 10 will be described using FIG. 6. FIG. 6 is a flow diagram showing the operation of the map generation device. In the following description, reference will be made to FIGS. 1 to 5 as appropriate. In addition, in the embodiment, a map generation method is implemented by operating the map generation device 10. Therefore, the description of the map generation method in the embodiment will be replaced by the following description of the operation of the map generation device 10.

[0052] 6 , first, when depth-added image data is output frame by frame from the depth sensor 20, the data acquisition unit 11 acquires the output image data (step A1). Furthermore, when the data acquisition unit 11 acquires image data for the set number of frames, the data acquisition unit 11 inputs the acquired image data to the feature point extraction unit 12.

[0053] Next, when the image data acquired in step A1 is input, the feature point extraction unit 12 extracts feature points for each frame (step A2). The feature point extraction unit 12 also outputs the extracted feature points for each frame to the filtering unit 13.

[0054] Next, when the feature points for each frame are input in step A2, the filtering unit 13 determines the traveling direction and position of the moving object based on the information about the moving object 100 for each input frame (step A3).

[0055] Next, the filtering unit 13 uses the traveling direction and position of the moving body 100 to determine for each frame whether or not there is a portion in that frame that does not overlap with other frames (step A4).

[0056] Specifically, in step A4, the filtering unit 13 identifies a frame located at the edge on the right, left, upper, or lower side of the moving body 100. Furthermore, the filtering unit 13 uses the above-mentioned equation 1 to calculate the amount of change in the posture of the moving body 100 between the frame located at the edge and the frame output immediately before it.

[0057] Then, using the calculated posture change amount γ, filtering unit 13 determines whether each of the feature points extracted from the edge frames satisfies the above equation 2. If there is a feature point that satisfies the above equation 2, filtering unit 13 determines that the edge frame has a portion that does not overlap with another frame.

[0058] Next, in a frame that has a portion that does not overlap with another frame, the filtering unit 13 eliminates feature points in that portion (step A5). Specifically, in A5, the filtering unit 13 eliminates feature points that satisfy the above equation 2 from frames located at the edges.

[0059] Next, the environment map generation unit 14 uses the feature points that were not excluded in step A5 to associate, for each frame, the feature points extracted from that frame with the feature points extracted from previous frames, and identifies pairs of feature points (step A6).

[0060] Next, the environmental map generation unit 14 calculates the three-dimensional coordinates of the feature points identified as a group in step A6, and uses the calculation results to generate an environmental map composed of a collection of feature points identified as a group (step A7).

[0061] Thereafter, the environmental map generating unit 14 inputs the environmental map generated in step A7 to the control device 30. As a result, the control device 30 uses the obtained environmental map to control the traveling direction and moving speed of the moving object.

[0062] As described above, in the embodiment, even if the angle of view of the depth sensor 20 is fixed, feature points are excluded from portions of frames located at the edges that do not overlap with other frames, thereby avoiding the extraction of incorrect sets of feature points. Therefore, according to the embodiment, the accuracy of the environmental map can be improved when a depth sensor with a fixed angle of view is used.

[0063] [Program] The program in the embodiment may be any program that causes a computer to execute steps A1 to A7 shown in Figure 6. By installing and executing this program on a computer, the map generation device 10 and map generation method in the embodiment can be realized. In this case, the processor of the computer functions as the data acquisition unit 11, feature point extraction unit 12, filtering unit 13, and environmental map generation unit 14 and performs processing.

[0064] The computer may be a computer mounted on the mobile object 100. Other examples of the computer include a general-purpose PC, a smartphone, and a tablet terminal device.

[0065] The program in the embodiment may be executed by a computer system constructed by a plurality of computers, in which case, for example, each computer may function as one of the data acquisition unit 11, feature point extraction unit 12, filtering unit 13, and environment map generation unit 14.

[0066] [Physical Configuration] A computer that implements the map generating device 10 by executing a program according to the embodiment will now be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of a computer that implements the map generating device.

[0067] 7, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0068] Furthermore, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this aspect, the GPU or FPGA can execute the programs in the embodiments.

[0069] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0070] The program in the embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

[0071] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0072] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0073] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0074] Note that the map generation device 10 in the embodiment can be realized not by a computer on which a program is installed, but by hardware corresponding to each unit, such as an electronic circuit. Furthermore, the map generation device 10 may be partially realized by a program and the remaining unit by hardware. In the embodiment, the computer is not limited to the computer shown in FIG. 7.

[0075] Some or all of the above-described embodiments can be expressed by (Supplementary Note 1) to (Supplementary Note 9) described below, but are not limited to the following descriptions.

[0076] (Supplementary Note 1) A map generating device comprising: a data acquisition unit that acquires image data output from a depth sensor with a fixed angle of view on a frame-by-frame basis; a feature point extraction unit that extracts feature points from the image data for each frame; a filtering unit that determines for each frame whether or not there is a portion in the frame that does not overlap with other frames, and if the determination result indicates that there is such a portion, removes the feature points extracted from the non-overlapping portion; and an environmental map generation unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and uses the calculation results to generate an environmental map made up of a collection of the feature points identified as pairs.

[0077] (Supplementary Note 2) The map generating device according to Supplementary Note 1, wherein the depth sensor is mounted on a moving body, and the filtering unit determines a direction of travel and a position of the moving body for each frame based on information about the moving body, and determines whether or not a portion of the frame exists that does not overlap with other frames using the determined direction of travel and position of the moving body for each frame.

[0078] (Supplementary Note 3) The map generating device according to Supplementary Note 2, wherein the filtering unit determines, for each frame, whether or not another frame exists around the frame using the orientation and position of the moving body determined for each frame, and if no other frame exists around the frame, determines that the frame has a portion that does not overlap with other frames other than the frame.

[0079] (Supplementary Note 4) A map generation method comprising: a data acquisition step of acquiring image data output from a depth sensor with a fixed angle of view on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of determining, for each frame, whether or not there is a portion in the frame that does not overlap with another frame other than the frame in question, and, if the determination result indicates that there is such a portion, excluding the feature points extracted from the non-overlapping portion; and an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and using the calculation results to generate an environmental map composed of a collection of the feature points identified as pairs.

[0080] (Supplementary Note 5) The map generation method according to Supplementary Note 4, wherein the depth sensor is mounted on a moving body, and in the filtering step, a direction of travel and a position of the moving body are determined for each frame based on information about the moving body, and the determined direction of travel and position of the moving body for each frame are used to determine whether or not there is a part in the frame that does not overlap with other frames other than the frame in question.

[0081] (Supplementary Note 6) The map generation method according to Supplementary Note 5, wherein in the filtering step, for each frame, it is determined whether or not another frame exists around the frame using the orientation and position of the moving body found for each frame, and if no other frame exists around the frame, it is determined that the frame has a portion that does not overlap with other frames other than the frame.

[0082] (Supplementary Note 7) A program causing a computer to execute the following steps: a data acquisition step of acquiring image data output from a depth sensor with a fixed angle of view on a frame-by-frame basis; a feature point extraction step of extracting feature points from the image data for each frame; a filtering step of determining, for each frame, whether or not a portion of the frame exists that does not overlap with another frame other than the frame in question, and, if the determination result indicates that such a portion exists, excluding the feature points extracted from the non-overlapping portion; and an environmental map generation step of identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and using the calculation results to generate an environmental map made up of a collection of the feature points identified as pairs.

[0083] (Supplementary Note 8) The program according to Supplementary Note 7, wherein the depth sensor is mounted on a moving body, and in the filtering step, the direction of travel and position of the moving body are determined for each frame based on information about the moving body, and the determined direction of travel and position of the moving body for each frame are used to determine whether or not there is a part in the frame that does not overlap with other frames other than the frame in question.

[0084] (Supplementary Note 9) The program according to Supplementary Note 8, wherein in the filtering step, for each frame, it is determined whether or not another frame exists around the frame using the orientation and position of the moving body found for each frame, and if no other frame exists around the frame, it is determined that there is a part of the frame that does not overlap with other frames other than the frame itself.

[0085] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0086] This application claims priority based on Japanese Patent Application No. 2022-91128, filed on June 3, 2022, the disclosure of which is incorporated herein by reference in its entirety.

[0087] As described above, according to the present disclosure, it is possible to improve the accuracy of an environmental map when using a depth sensor with a fixed angle of view. The present disclosure is useful in fields where SLAM is used.

[0088] REFERENCE SIGNS LIST 10 Map generation device 11 Data acquisition unit 12 Feature point extraction unit 13 Filtering unit 14 Environmental map generation unit 20 Depth sensor 30 Control device 40 Steering device 50 Power train 100 Mobile body 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. A data acquisition unit that acquires image data output from a depth sensor having a fixed angle of view on a frame-by-frame basis; A feature point extraction unit that extracts feature points from the image data for each frame; a filtering unit that determines, for each frame, whether or not a portion of the frame that does not overlap with another frame other than the frame in question exists, and, if the result of the determination indicates that a portion of the frame exists, removes the feature points extracted from the non-overlapping portion; an environment map generating unit that identifies pairs of corresponding feature points between frames, calculates three-dimensional coordinates of the feature points identified as pairs, and generates an environment map composed of a collection of the feature points identified as pairs using a calculation result; Equipped with A map generating device comprising:

2. The depth sensor is mounted on a moving body, the filtering unit determines, for each frame, a moving direction and a position of the moving body based on information about the moving body, and determines, using the determined moving direction and position of the moving body for each frame, whether or not a portion of the frame does not overlap with another frame other than the frame; The map generating device according to claim 1 .

3. the filtering unit determines, for each frame, whether or not another frame exists around the frame using the orientation and position of the moving object obtained for each frame, and if no other frame exists around the frame, determines that the frame has a portion that does not overlap with any other frame other than the frame itself; The map generating device according to claim 2 .

4. Image data output from a depth sensor with a fixed angle of view is acquired frame by frame. Extracting feature points from the image data for each frame; determining, for each frame, whether or not there is a portion in the frame that does not overlap with another frame other than the frame in question, and if the result of the determination indicates that there is a portion in the frame, excluding the feature points extracted from the non-overlapping portion; Identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environment map composed of the sets of the feature points identified as pairs using the calculation results. A map generating method comprising:

5. The depth sensor is mounted on a moving body, In the determination, a moving direction and a position of the moving body are obtained for each frame based on information of the moving body, and using the obtained moving direction and position of the moving body for each frame, it is determined whether or not a portion of the frame does not overlap with another frame other than the frame. The map generating method according to claim 4.

6. In the determination, for each frame, using the orientation and position of the moving object determined for each frame, it is determined whether or not another frame exists around the frame, and if no other frame exists around the frame, it is determined that the frame has a portion that does not overlap with any other frame other than the frame. The map generating method according to claim 5.

7. On the computer, Image data output from a depth sensor with a fixed angle of view is acquired frame by frame, extracting feature points from the image data for each frame; determining, for each frame, whether or not there is a portion in the frame that does not overlap with another frame other than the frame in question, and, if the result of the determination indicates that there is a portion, excluding the feature points extracted from the non-overlapping portion; Identifying pairs of corresponding feature points between frames, calculating three-dimensional coordinates of the feature points identified as pairs, and generating an environmental map composed of a set of the feature points identified as pairs using the calculation results. program.

8. The depth sensor is mounted on a moving body, In the determination, a moving direction and a position of the moving body are obtained for each frame based on information of the moving body, and using the obtained moving direction and position of the moving body for each frame, it is determined whether or not a portion of the frame does not overlap with another frame other than the frame. The program according to claim 7.

9. In the determination, for each frame, the orientation and position of the moving body obtained for each frame are used to determine whether or not another frame exists around the frame, and if no other frame exists around the frame, the frame is determined to have a portion that does not overlap with any other frame other than the frame itself. The program according to claim 8.