Map utilization device, method, and program

Abstract

To provide a map using device which can set a route with a high self-position estimation accuracy based on an image without requiring a mobile body to travel in advance.SOLUTION: The present invention performs first processing 31 of creating a two-dimensional map for a movable range for a mobile body from a three-dimensional map of a field; and second processing 32 for setting a route from a place of start to a destination as a route which goes through a plurality of way points determined to be highly accurate when the mobile body compares an image taken by a camera with the three-dimensional map and estimates the self position of the mobile body in the two-dimensional map.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a map utilization device, method, program, and movement control method for a mobile object to move by performing self-location estimation using a VPS. [Background technology]

[0002] When an autonomous mobility service using a mobile object such as a robot is introduced, the autonomous vehicle will travel autonomously with reference to a map, and it is desirable for the self-location estimation accuracy during the autonomous travel to be high. This is because if the self-location estimation accuracy is low, there is a risk of deviating from the planned travel route. In this regard, for example, Patent Documents 1 to 3 have been considered as technologies for planning a travel route depending on the accuracy of the self-location estimation.

[0003] Patent Document 1 discloses a program for providing appropriate driving support for a moving object. The accuracy of self-location estimation is evaluated at the position where the camera image was acquired, and whether or not to use the self-location estimation result for driving support can be switched based on the accuracy. Here, if it is determined that the accuracy of self-location estimation using the camera image is low, the program switches to self-location estimation using odometry.

[0004] Patent Document 2 discloses a device that shortens the time it takes for a moving object to reach a destination. A self-location estimation unit based on the detection result of a laser sensor evaluates that the self-location estimation accuracy has decreased, and the frequency with which a low-evaluation area in which the self-location estimation accuracy is evaluated as low is selected as a route decreases. This suppresses the moving object from moving to a low-evaluation area, thereby reducing unnecessary time.

[0005] Patent Document 3 discloses a method for evaluating the accuracy of self-location estimation by a laser sensor, which can easily evaluate the accuracy of self-location estimation of a moving object. A schematic diagram showing a map of the moving object's driving environment is generated, and the positions where the accuracy of self-location estimation is likely to decrease are colored, allowing the user to visually judge. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2021-105832 A [Patent Document 2] Patent Publication No. 2021-117893 [Patent Document 3] JP 2022-037736 A [Non-patent literature]

[0007] [Non-Patent Document 1] "Komorita et al., "Research on Image-Based Positioning Technology and System Development and Demonstration," Institute of Image Information and Television Engineers, 2021." Summary of the Invention [Problem to be solved by the invention]

[0008] However, in all of the techniques disclosed in Patent Documents 1 to 3, in order to evaluate the accuracy of self-location estimation, it is necessary for the robot to travel in advance at least through the field for which the map is to be generated, which poses the problem of requiring the robot to travel in that field incurring the effort of such travel.

[0009] As an example of a method that does not require the robot to run through the field in advance and perform measurements for generating a map, the Visual Positioning System (VPS) described in Non-Patent Document 1 can generate a map by processing camera images of the environment taken by a person or other person (i.e., images from a camera viewpoint that does not necessarily coincide with the camera viewpoint of the robot actually moving), so it does not require the robot to actually run through the field in advance.

[0010] In this regard, in Patent Document 1, self-location is estimated using a map based on image capture, but since the map needs to be generated from images captured by the robot itself, it is not possible to enjoy the advantage of not having to limit the imaging means used in map generation to the actual robot, as in VPS. Also, in Patent Document 1, the accuracy of self-location estimation can only be evaluated at the stage when the robot actually travels and estimates its self-location, and it is not possible to evaluate the accuracy before travel.

[0011] As described above, Patent Documents 2 and 3 require the robot to move at least in advance, and use laser sensor measurement for self-position estimation, which is different from image-based self-position estimation like VPS.

[0012] In view of the problems with the conventional technology described above, a first object of the present invention is to provide a map utilization device, method, and program that can set a route with high accuracy in self-location estimation based on images, without requiring the moving object to travel in advance.

[0013] A second object of the present invention is to provide a movement control method for controlling the movement of a moving object on a route set by a map utilization device that achieves the first object. [Means for solving the problem]

[0014] In order to achieve the first object, the present invention is characterized in that it is a map utilization device that executes a first process of creating a two-dimensional map of a range in which a moving object can move from a three-dimensional map of a field, and a second process of setting a route from a starting point to a destination in the two-dimensional map as a route that follows a plurality of waypoints that are determined to have high accuracy when estimating the moving object's self-position by comparing an image captured by a camera of the moving object with the three-dimensional map. The present invention is also characterized in that it is a method and a program corresponding to the device.

[0015] In order to achieve the second object, the present invention is a movement control method in which a computer controls in real time to move a moving object while passing through the waypoints that are predetermined based on the first process and the second process that are executed in advance by the map using device, and has a second feature in that the moving object's own position is estimated in real time by comparing an image captured by a camera of the moving object in real time with the 3D map, and an instruction is generated to move the moving object so that the self-position estimated in real time passes through the waypoints in order. Effect of the Invention

[0016] According to the first feature, it is possible to set a route with high accuracy of self-location estimation based on an image without requiring the moving object to travel in advance. According to the second feature, it is possible to control the movement of the moving object on the route set by the map using device. [Brief description of the drawings]

[0017] [Figure 1] 1 is a configuration diagram of a map using system according to an embodiment. [Diagram 2] FIG. 2 is a functional block diagram of each device in the map utilization system according to an embodiment. [Diagram 3] 1 is a flowchart showing an overall schematic operation of a map use system according to an embodiment. [Figure 4] FIG. 1 is a diagram showing an example of a 3D map as a 3D point cloud generated from captured footage of an environment using the VPS technique. [Diagram 5] 4 is a flowchart showing a 2D map creation process by a 2D map creation unit according to one embodiment, as details of step S2 in FIG. 3. [Figure 6] FIG. 13 is a diagram showing an example of a processed 3D map (processed by extracting only the height range in which the robot can move). [Figure 7] FIG. 7 is a diagram showing a general case of machining that does not depend on the individual origin positions in FIG. 6. [Figure 8]FIG. 13 is a diagram showing a schematic example of length conversion (reduction) in the height Z-axis direction for generating a polygon. [Figure 9] FIG. 13 is a table showing examples of setting values ​​of three thresholds and examples of coloring rules based on threshold determination. [Figure 10] This is an example of a group of polygons when the color intensity is set according to the coloring rules in the table of FIG. [Figure 11] FIG. 11 is a diagram showing a two-dimensional map generated by compressing the polygon group in FIG. 10 in the height direction. [Figure 12] 4 is a flowchart of a route setting process according to an embodiment, showing details of step S3 in FIG. 3. [Figure 13] FIG. 10 is a diagram showing an example of a point table in which point values ​​corresponding to the three shades of red shown in FIG. 9 are preset. [Figure 14] 13A and 13B are diagrams showing schematic examples of each process of the path generating unit, as examples EX1 and EX2. [Figure 15] 13A and 13B are diagrams showing schematic examples of each process of the path generation unit, as examples EX3 and EX4. [Figure 16] FIG. 13 is a diagram showing a schematic example of the camera field of view determined in a cell (NX, NY) for each angle θ. [Figure 17] FIG. 13 is a diagram showing an example of recording in JSON format. [Figure 18] FIG. 2 is a diagram showing a schematic example of a waypoint. [Figure 19] 4 is a flowchart showing the process of driving control and map generation according to an embodiment, as detailed steps S4 in FIG. 3. [Figure 20] 11A to 11C are diagrams illustrating examples of movement control of a robot, as examples EX11 to EX14. [Figure 21] FIG. 1 is a diagram illustrating a hardware configuration of a typical computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] 1 is a configuration diagram of a map utilization system 100 according to an embodiment. The map utilization system 100 includes an environment capture device 1 as a camera, a map utilization device 3 and a VPS device 6 as server devices, and a robot 4 as a mobile body, and these elements can communicate with each other via a network NW such as the Internet or a LAN (local area network). Here, the environment capture device 1 and the robot 4 can move within a field F consisting of indoors and / or outdoors, which is the range for which the map utilization system 100 creates a map.

[0019] The following description will be given based on the device configuration in Fig. 1, but this configuration can be modified in various ways. For example, in Fig. 1, the map use device 3 and the VPS device 6 are configured separately from the robot 4, but the map use device 3 and / or the VPS device 6 may be mounted in the robot 4 (the map use device 3 and / or the VPS device 6 may be realized by a processor or the like provided in the robot 4).

[0020] 2 is a functional block diagram of each device of the map utilization system 100 according to an embodiment. As shown in the figure, the map utilization device 3 includes a 2D map creation unit 31 including a 3D map processing unit 311, a polygon generation unit 312, a cell generation unit 313, and a 2D map storage unit 314, a route setting unit 32 including a departure point / destination setting unit 321, a route generation unit 322, and a route storage unit 323, and a travel control unit 33 including an image cropping unit 331, a position confirmation unit 332, and an operation planning unit 333.

[0021] The robot 4 includes a sensor unit 41, a map generation unit 42, an environment photographing unit 43, and an operation control unit 44. The VPS device 6 includes a 3D map creation unit 21, a 3D map storage unit 22, and a position estimation unit 51. As shown in the figure, in the VPS device 6, the 3D map creation unit 21 and the 3D map storage unit 22 function as a 3D map creation device 2 that creates a 3D map, and the 3D map storage unit 22 and the position estimation unit 51 function as a self-position estimation device 5 that estimates the self-position of the robot 4.

[0022] Fig. 3 is a flowchart showing an overall schematic operation of the map utilization system 100 according to an embodiment. Below, the details of each device and each functional unit in Fig. 2 will be described while explaining each step S1 to S4 in Fig. 3.

[0023] In step S1, images captured comprehensively within field F by environmental imaging device 1 are input, and 3D map creation device 2 creates a 3D map of field F. In step S2, 2D map creation unit 31 creates a 2D map of field F using the 3D map created in step S1 as input.

[0024] In step S3, for example, for a robot 4 that delivers goods such as merchandise, input of a starting point and destination within field F is accepted, and the route setting unit 32 sets a route for the robot 4, which is a mobile body, to travel from the starting point to the destination within field F.

[0025] In step S4, while the robot 4 takes images on the path (global path) set in step S3, the self-position estimation device 5 analyzes the images to estimate the current position of the robot 4, and the travel control unit 33 determines a movement path (local path) of the robot 4 according to the current position and performs travel control processing of the robot 4 in accordance with the determination, and the robot 4 moves according to the travel control. In this way, the robot 4 takes images of the field F at each time, updates its own position estimated from the results with movement by travel according to the travel control, and moves sequentially along the local path, thereby reaching the starting point to the destination within the field F.

[0026] Steps S1 to S4 will be further described in detail below.

[0027] <Creating a 3D map...Step S1> In step S1, the 3D map generating unit 2 creates a 3D map and stores the 3D map in the 3D map storage unit 22. In this embodiment, a case where a 3D map is created by the VPS method disclosed in the above-mentioned Non-Patent Document 1 will be described as an example.

[0028] First, the environment (field F) is manually photographed using the environment imaging device 1. The camera constituting the environment imaging device 1 is, for example, a 360° camera such as RICOH THETA (registered trademark, product name), and images are captured moving around the environment. At this time, in order to improve the self-location estimation accuracy of the VPS, it is desirable for the 360° camera to maintain the same height as the robot's image capturing unit, and to move gently around corners avoiding right-angle movements, and to go around in a circle.

[0029] Note that by using a 360° camera with an ultra-wide angle of view, the effort required to point the camera in various directions when taking pictures while moving around the environment can be reduced; however, it is also possible to use a normal camera with an angle of view limited to the surrounding forward area to take pictures in various directions while moving around the environment.

[0030] Next, the captured image is input to the 3D map creation unit 21 of the VPS device 6, a 3D map is created, and the 3D map is stored in the 3D map storage unit 22. For each frame image in the image, moving objects such as people become noise and are removed using object recognition processing or the like, and then feature points are extracted for stationary objects such as walls that serve as obstacles that prevent the robot 4 from moving when it moves, or as objects that serve as landmarks for movement, and the feature points are matched between different images, and a 3D point cloud of stationary objects in the environment can be reconstructed using principles such as stereo parallax. At this time, the length of a portion of the map may be measured separately using a surveying instrument, and a physical quantity conversion may be performed to scale the point cloud accordingly.

[0031] As a result, a 3D map that holds the point cloud and coordinates (the 3D coordinates of each point constituting the 3D point cloud) as exemplified in FIG. 4 can be generated by the VPS method. In the example of FIG. 4, the environment (field F) is an indoor space such as a conference room, and the X and Y axes are horizontal and the Z axis is vertical in the 3D spatial coordinates (X, Y, Z). In FIG. 4, each point constituting the point cloud is shown as a black dot, and the distribution of the point cloud is shown as seen from a direction close to the vertical Z axis. As a result, the point cloud distributed vertically on the surface of the wall, etc. is drawn darkly and densely on the paper, and conversely, the point cloud distributed horizontally on the surface of the ceiling, floor, etc. is drawn lightly and densely on the paper. The origin (X, Y, Z)=(0, 0, 0) shown in FIG. 4 can be appropriately defined when generating the point cloud. Note that each diagram used as an example in the following description also relates to an indoor environment similar to the example of FIG. 4.

[0032] <Creating a 2D map...Step S2> In step S2, the units 311 to 315 of the two-dimensional map creation unit 31 perform the following processes in this order to create a two-dimensional map.

[0033] The 3D map processing unit 311 processes the 3D map into a 2D map while retaining the characteristics of the environment. The polygon generation unit 312 generates polygons that divide the space of the 2D map while retaining the characteristics of the environment, and determines the attributes of each polygon (attributes regarding the passability of moving objects). The cell generation unit 313 determines unit cells (hereinafter, cells) and the attributes of each cell in two-dimensional grid coordinates for the 2D map divided by the polygons, thereby obtaining the final created 2D map. The 2D map storage unit 314 stores the created 2D map and provides it for reference during processing by the route setting unit 32.

[0034] Fig. 5 is a flowchart showing the process of creating a 2D map by the 2D map creation unit 31 as detailed step S2 in Fig. 3. Among steps S101 to S120 in Fig. 5, steps S101 to S106 are processes by the 3D map processing unit 311, steps S107 to S118 are processes by the polygon generation unit 312, step S119 is processes by the cell generation unit 313, and step S120 is processes by the 2D map storage unit 314. Therefore, in the following explanation of each step, reference to the units 311 to 315 that are the main operators will be omitted, and only the contents of the processes will be explained.

[0035] First, in steps S101, S102, and S103, a 3D map and a floor height (H f >0[m]) and the total height of the robot (H r >0[m]) as input. f is a numerical value obtained by visually observing a 3D map (the height of the floor is the z-axis coordinate value of the floor, and depending on whether it is positive or negative, it is expressed as ±H f The user input value that confirms that the robot total length H r may use a user-entered value that confirms the catalog value of the robot 4.

[0036] Next, in steps S104 to S106, the input height H f and total length H r All values ​​of height z between f ≦z≦H r -H f ), in step S105, the 3D map (point cloud) at the height z is extracted, and the points that are not extracted (those with coordinates (x, y, z) whose height z is z<-H f or z>H r -H f The 3D map is processed by deleting all points within the height range z(-H f ≦z≦H r -H f), and similar repetition steps will appear in the following explanations.

[0037] Figure 6 shows an example of a processed 3D map. In the 3D map, the floor height z = -H f From the height of robot 4, z=H r -H f Length H in the height direction z r Only the area R2 within the range is left, and areas R1 and R3 other than this area R2 are deleted as areas that the robot will not pass through. The significance of this process is that by limiting the data to only the areas where the robot 4 can pass through the environment, rather than using all the data of the 3D map generated as a point cloud as is, it becomes easier to determine whether the robot 4 can travel.

[0038] Specifically, using an application capable of editing 3D data such as Unity (Non-Patent Document 2), the height z (-H f ≦z≦Hr-H f ) and delete the other points (the area to be deleted in Figure 6). [Non-Patent Document 2] "The world's leading platform for real-time content creation, https: / / unity.com / ja, (Accessed 2022-08-01)."

[0039] In the example of Figure 6, the height of the origin z=0 is the negative value of the floor surface z=-H f <0 height and positive value for the top of the robot standing on the floor r -H f > 0. In general, as shown in Figure 7, the floor height coordinate value z = H f And this floor height z=H f Robot height H r >0 added to the robot height coordinate z=H f +H r It is necessary to extract only the points located at heights between and .

[0040] In this embodiment, the premise for making it possible to extract steps S104 to S106 as described above is that the ground or floor surface within field F, which is the subject of the creation of the 2D map and in which the robot 4 moves using a moving mechanism such as wheels, has a generally constant height, does not have any large inclined areas such as steep slopes, and can be considered to be a generally horizontal surface.

[0041] Next, in step S107, a polygon is generated, which is a unit lattice that divides the 3D map as the 3D point cloud two-dimensionally and is used to count the number of points in the unit lattice. That is, the 3D map is divided into a polygon having a side d P The height Z is divided into H r [m] to d P Convert to [m] and set the unit size of the horizontal X and vertical Y to the same as the side d P In Fig. 8, we generate polygons with size relationship H r >d p Since this is the usual relationship, the positional relationship of the points is maintained, and the length of one side d at each position of horizontal X and vertical Y is P The number of points in the square grid of [m] is maintained while the length H r is the unit size d p As a result, the image is reduced in size in the Z-axis direction.

[0042] In this embodiment, a polygon is formed with a side d P In the following description, a cube with a size of 10 [cm] = 0.01 [m] is assumed, but the size is not limited to this and may be changed according to the accuracy required for the movement of the robot 4 in the environment (field F). For example, the required accuracy of self-location estimation may change depending on the environment in which the robot 4 runs. For example, while rough accuracy of self-location estimation may be acceptable in a wide outdoor area, it is expected that high accuracy of self-location estimation is required when passing through a narrow passage in an apartment building or the like. For this reason, the size of the polygon may be dynamically changed depending on the environment in which the robot runs.

[0043] Size of one side d P [m] setting, the number of polygons in the horizontal direction N Xmax The distance d in the X-axis direction of the map is calculated as follows: mapx [m] and one side of the polygon d P If there is a fraction less than 10 cm, you can round it up to 10 cm. Similarly, the number of polygons in the vertical direction, N Ymax The distance d in the Y-axis direction of the map is calculated as follows: mapy [m] and one side of the polygon d P [m] is the quotient.

[0044]

number

[0045] Next, in steps S110 to S118, the number of points included in each polygon is counted. To this end, a variable (N x ,N y ) in steps S108 and S109, the initial value (N X ,N Y ) = (1,1).

[0046] Note that steps S110 and S118 are the same as steps S111 to S117 that they enclose, except that all values ​​of the variable Nx are N X = 1, 2, …, N Xmax Similarly, steps S111 and S116 each represent a first loop that repeats steps S112 to S115 that they enclose with a variable N Y All values ​​of N Y = 1, 2, …, N Ymax In this manner, in steps S110 to S118 involving double loop processing of the first and second loops, the processing of steps S112, S113, and S114 is performed for all polygon positions {(N x ,N y )| N x = 1, 2, …, NXmax , N Y = 1, 2, …, N Ymax} will be performed exhaustively in raster scan order.

[0047] In addition, in step S115, Y to the next value N Y +1, and in step S117, X to the next value N X Update to +1 and N Y Set the initial value N Y = 1, the above-mentioned raster scan order processing is realized. Similarly, in step S116, the current N Y The value of is N Ymax If it is +1, proceed to step S117; if not (N Y ≦N Ymax If so, the process returns to step S111. In step S118, the current N X The value of is N Xmax If it is +1, proceed to step S119; if not (N X ≦N Xmax If so, the process returns to step S110, whereby the above-mentioned processing in the raster scan order is realized.

[0048] Hereinafter, steps S112, S113, and S114 will be described as the actual processes repeated by the double loop process described above.

[0049] In step S112, the position (N x ,N y ), the number of points contained in the polygon is counted, and then the process proceeds to step S113. x ,N y ) is written as

[0050] Since the point cloud may contain noise, in step S113, the number of points in the cloud, num (N x ,N y) determines the amount of noise and features according to it. Specifically, a threshold Th1 is set for the number num(N x ,N y ) of the point cloud. A polygon (N x ,N y ) with the number of point clouds less than the threshold Th1 is determined to be noise. That is, by going from step S113 (skipping step S114) to step S115, as will be described later with reference to FIG. 9, a polygon (N x ,N y ) with the number of point clouds less than the threshold Th1 is not colored and is treated as colorless (white) as the initial color.

[0051] On the other hand, for a polygon (N x ,N y ) having the number num(N x ,N y ) of the point cloud exceeding the threshold Th1, it proceeds from step S113 to step S114, and coloring is performed in step S114. Here, two more thresholds Th2 and Th3 are set according to the number num(N x ,N y ) of the point cloud, and the color intensity (and the presence or absence of color) is adjusted in four levels by the three thresholds. (Note that for the three thresholds to be set, the magnitude relationship is Th1 < Th2 < Th3.) This is performed for all polygons (N x ,N y ).

[0052] FIG. 9 is a diagram showing an example of the set values of the three thresholds Th1 < Th2 < Th3, the coloring rule by threshold determination, and an example of the polygon attribute represented by the coloring. FIG. 10 is an example of a polygon group when coloring is performed according to the coloring rule in the table of FIG. 9 for the color intensity.

[0053] As shown in FIG. 9, with the three thresholds Th1 = 20, Th2 = 25, and Th3 = 40, the attribute of the polygon (N x ,N y ) position as a two-dimensional map is evaluated in four levels, and the number of point clouds num(N x ,N y) is equal to or smaller than the minimum threshold value Th1, it is evaluated that no object is present at the polygon position and that the path is one through which the robot 4 can move.

[0054] On the other hand, the number of points num(N x ,N y ) is greater than the minimum threshold value Th1, it is evaluated that there is an object (such as a wall) at the polygon position that is an obstacle that the robot 4 cannot move to, and the reliability is divided into three levels. x ,N y If the number of points num(N x ,N y If the number of points num(N x ,N y ) is greater than Th3, it corresponds to an object and its reliability is high.

[0055] The three shades of red shown in Figure 9, with or without color (white) and with color, are an example for visualizing the polygon attributes shown in Figure 9 on a two-dimensional map by distinguishing them. In the following, the explanation will be based on the example of Figure 9, with the difference in color representing the polygon attributes.

[0056] When color is present, it is presumed that there is an object at the polygon position. As mentioned above, this object may constitute an obstacle that robot 4 cannot move past, and at the same time, it may serve as a landmark for robot 4 to estimate its own position using images as it moves along the passageway.

[0057] 9, the cases with color (objects) are divided into three stages, but they may be divided into any stage of one or more stages. If they are divided into two or more stages, the polygon attributes are evaluated such that the more points there are, the higher the reliability that they correspond to an object, and they can be distinguished by appropriate color coding, etc.

[0058] All polygons (N x ,N y ) was colored. Next, in step S119, the polygon height direction information is compressed to 0 [cm] while maintaining the coloring, thereby generating cells, thereby generating a two-dimensional map as shown in FIG. 11. The two-dimensional map in FIG. 11 is obtained by compressing the polygon height Z-axis direction in FIG. 10 to 0 [cm], and the two-dimensional grid coordinates (N x ,N y ) to specify the position of each, and the size of each side is d p The 2D map is composed of a unit cell (cell) having a square shape of [m] and attributes based on its coloring. In step S120, the 2D map generated in step S119 is stored in the 2D map storage unit 314, and the flow in FIG. 5 (the processing of the 2D map generation unit 31) is terminated.

[0059] <Route setting...Step S3> In step S3, each unit 321 to 323 of the route setting unit 32 performs the following processes in this order to set a route, and the set route is stored in the route storage unit 323. Here, the route derives a route to reach the destination while avoiding places with low accuracy of self-location estimation.

[0060] Fig. 12 is a flowchart of a route setting process according to an embodiment, as details of step S3 in Fig. 3. Of all steps S201 to S222 in the flow in Fig. 12, the first steps S201 and S202 are processes of the departure point / destination setting unit 321, and the subsequent steps S203 to S222 are processes of the route generating unit 322. Therefore, in the following description of each step, reference to the subject of operation will be omitted.

[0061] 14 and 15 are diagrams showing schematic examples of each process of the path generating unit 322 as examples EX1, EX2 and examples EX3, EX4, respectively, and will be referred to as appropriate in the following description.

[0062] First, in step S201, input of a two-dimensional map stored in the two-dimensional map storage unit 314 is accepted, and in step S202, a starting point and a destination are accepted as user input and set. Here, the cell position (N X ,N Y ) and accepts the input of the departure point and destination. X ,S Y ) and destination (G X ,G Y ) (i.e., the starting point is cell position (N X ,N Y )=(S X ,S Y ), and the destination is the cell position (N X ,N Y )=(G X ,G Y ) shall be.

[0063] Next, in step S203, an object is determined from the two-dimensional map. The reason for determining the object in step S203 is that the object is a feature of the environment, and the feature can be used to predict the accuracy of self-location estimation.

[0064] Specifically, a total point value according to the color of the cells that are not white but are colored in the 2D map is calculated according to the point table in Fig. 13. Note that Fig. 13 shows an example of a point table that is preset as point values ​​according to the three shades of red in Fig. 9. Note that, without being limited to the example in Fig. 13, a setting may be used in which, for a cell that is determined to contain an object rather than a passageway, the higher the count number of the original point cloud is, the higher the reliability that the cell corresponds to an object is estimated to be.

[0065] As described above, points are determined for all colored cells from the point table, and further, a set of colored cells that are connected on the 2D map and whose total point value is equal to or greater than a threshold value is regarded as an object, thereby making it possible to perform object determination in step S203. In other words, colored cells that meet the conditions are determined to be objects, and colored cells that do not meet the conditions and uncolored (white) cells are determined to not be objects.

[0066] For example, if the threshold value for the point sum is set to 10, object determination can be performed for each of the four colored cell connections C1 to C4, as shown in example EX1 in Fig. 14. Note that the threshold value is not limited to 10, and can be changed by an administrator or the like by specifying an appropriate value according to the environment.

[0067] In the example EX1 of FIG. 14, the point sums sum(C1) to sum(C4) are calculated for the four colored cell connections C1 to C4 as follows, and a comparison with a threshold value of 10 is made to determine whether or not to regard them as objects.

[0068] ● The connection point C1 is composed of the connected area of ​​two dark red cells, Since sum(C1)=2×4=8<10, the connection point C1 is not regarded as an object. ● The connection point C2 is also composed of the connection area of ​​two dark red cells, Since sum(C2)=2×4=8<10, the connection point C2 is also not regarded as an object. ● Connection point C3 is composed of a connection area of ​​two light red cells, two red cells, and one dark red cell. Since sum(C3)=2×1+2×2+1×4=10≧10, the connection point C3 is regarded as an object. ● Connection point C4 is composed of three dark red cells. Since sum(C4)=3×4=12≧10, the connection point C4 is regarded as an object.

[0069] In the following explanation, examples EX2 to EX4 in Figures 14 and 15 will be referred to as being in the same situation as example EX1 in Figure 14, where connected areas C1 to C4 exist in a local range of 8 cells horizontally by 8 cells vertically = 64 cells.

[0070] In step S204, camera information (angle of view, focal length) is input, and a camera field of view is generated from this value. This camera information may be obtained as information about the camera constituting the environment photographing unit 43 of the robot 4 by user input or the like.

[0071] Next, in steps S205 to S219, the processes of steps S211, S212, and S213 are repeated in a triple loop to cover the entire two-dimensional map. X Vertical N Y This is carried out for all parameters of the direction θ. For this purpose, first, in step S205, an integer variable (N x ,N y ) with respect to the initial value (N X ,N Y )=(1,1). Also in step S205, the direction θ is set to an initial value θ=0°.

[0072] Steps S206 and S219 repeat steps S207 to S218 surrounding them for all values ​​of the variable Nx, N X = 1, 2, …, N Xmax Similarly, steps S207 and S218 indicate a first loop in which steps S208 to S217 which they surround are repeated for a variable N Y All values ​​of N Y = 1, 2, …, N Ymax In this way, in steps S206 to S219 involving double loop processing of the first and second loops, the processing of steps S209 to S215 is performed for all polygon positions {(N x ,N y )| N x = 1, 2, …, N Xmax , N Y = 1, 2, …, NYmax} will be performed exhaustively in raster scan order.

[0073] In addition, in step S216, Y to the next value N Y +1, and in step S218, X to the next value N X Update to +1 and N Y Set the initial value N Y = 1, the above-mentioned raster scan order processing is realized. Similarly, in step S217, the current N Y The value of is N Ymax If it is +1, proceed to step S218; if not (N Y ≦N Ymax If so, the process returns to step S207. In step S219, the current N X The value of is N Xmax If it is +1, proceed to step S220; otherwise (N X ≦N Xmax If so, the process returns to step S206, whereby the above-mentioned processing in the raster scan order is realized.

[0074] Hereinafter, steps S208 to S215 will be described as the actual process repeated by the double loop process described above. (Note that steps S208 to S215 further include a process that exhaustively repeats the process for the direction θ for the colored cells, so that a triple loop process is performed overall.)

[0075] In step S208, the cell (N X ,N Y ) and prepare information on the camera's field of view, assuming that a camera whose angle of view and field of view are determined by the camera information is located at the specified position. This camera's field of view is determined as a sector-shaped, triangular, or other shaped area on a two-dimensional map. Note that the distance in the camera's field of view can be set to be limited to within a certain range from the camera position, rather than to infinity.

[0076] In step S208, information on the shape of the area in the camera's field of view is prepared, and then in step S210 described later, the cell (N X ,N Y ) and the camera is assumed to be facing in a direction of angle θ (a direction on the 2D map where 0°≦θ<360°, and the angle θ is set in the repeated loop processing for that θ), and the process superimposes the camera's field of view on the 2D map.

[0077] In step S209, the cell (N X ,N Y ) is colored, and if it is not colored, X ,N Y ) is white, and if it corresponds to an aisle, it is processed in step S210. X ,N Y ) corresponds to an object and is likely to be an unmovable location), the process proceeds to step S216 to skip it from the processing target.

[0078] In steps S210 to S215, the non-colored cell (N X ,N Y 12, the direction θ of the camera field of view in step S208 is comprehensively set to 0° to 360°, and steps S211, S212, and S213 are repeated for each value of θ. In this embodiment, the increment width of the angle during the repetition is set to 10° in step S214, but other increments may be set. That is, in steps S210 to S215 of FIG. 12, the non-colored cell (N X ,N YFor the angle θ, the processes of steps S211, S212, and S213 are performed comprehensively for each of the 36 angles θ=0°, 10°, 20°, ..., 340°, 350° in increments of 10°. (For this reason, in step S215, it is determined whether the process has been completed for all angles θ from 0°≦θ<360° with the increment width set in step S214, and if completed, the process proceeds to step S216, and if not, the process returns to step S210 and is repeated.)

[0079] In Fig. 16, the number of cells (N X ,N Y 16 shows a schematic example of a camera field of view determined in the above-mentioned embodiment. In the example of Fig. 16, for a cell in the center of a 7 x 7 cell range shown on the left, a 360° range is comprehensively covered by an angle increment width of 45°, and sector-shaped camera fields of view CV0 to CV7 facing eight angles θ = 0°, 45°, 90°, ..., 270°, 315° are shown on the right.

[0080] Comprehensive processing for each camera view direction θ as in the example of FIG. 16 is performed in steps S211, S212, and S213 of FIG. 12. For this purpose, in step S210, a fan-shaped area showing the camera view facing the direction θ is superimposed on the two-dimensional map, and non-colored cells (N X ,N Y The process proceeds to step S211 to identify a place (cell) with high accuracy of self-location estimation among the above. A cell determined to have high accuracy of self-location estimation is determined by grasping a certain amount of characteristics.

[0081] Specifically, in this embodiment, if at least one of the two condition determinations in step S211 or step S212 is satisfied, the process proceeds to step S213, and the cell (N X ,N Y If neither of the two condition determinations in step S211 and step S212 is satisfied, step S213 is skipped and the cell (N X ,N Y) and the direction θ are those with high self-location estimation accuracy, is not stored.

[0082] As the first determination, in step S211, the cell position (N X ,N Y ) the number of objects in the camera field of view in the direction θ is threshold Th o If the accuracy is higher than this, it is determined that the self-location estimation accuracy is high.

[0083] The example EX2 in FIG. 14 is an example of the first determination. o With the setting of =2, the cell position (N X ,N Y )=(6,8), the number of objects in the camera field of view CV(6,8,90°) with angle θ=90° is 2, which is above the threshold. Therefore, X ,N Y The angle θ=90° at CV(6,8) is determined to be high accuracy for self-location estimation. Here, the camera field of view CV(6,8,90°) includes a part of connected area C3 determined to be an object and the whole of connected area C4 determined to be an object, so the number of objects is 2.

[0084] In addition, as shown in the connected area C3 in Example EX2, when only a part of the connected area determined to be an object is included, a certain percentage or more (e.g., more than half) of the connected area is located at the cell position (N X ,N Y If the number of objects is less than a certain percentage, they are not counted as objects.

[0085] In each example of FIG. 14 and FIG. 15, the coordinates of the upper left corner of a 8×8 cell are (N X ,N Y ) = (1,1), and the bottom right coordinate is (N X ,N Y ) = (8,8).

[0086] As a second determination method, in step S212, the cell position (N X ,N Y) The total number of points of the cells included in the camera field of view in the direction θ is the threshold value Th p If the accuracy is higher than this, it is determined that the self-location estimation accuracy is high.

[0087] The example EX3 in FIG. 15 is an example of the second determination, and the threshold value Th p At the setting of =20, the cell position (N X ,N Y )=(2,6), the total number of points in the camera field of view CV(2,6,90°) with angle θ=90° is 22 as shown below. Since this total number of points is greater than the threshold, the cell position (N X ,N Y )=(2,6), the angle θ=90° is determined to have high accuracy in self-location estimation.

[0088] In other words, since there are two light red cells, one red cell, and four dark red cells within the camera's field of view CV (2,6,90°), the total point value can be calculated as follows: 2×1+1×2+4×4=22

[0089] Note that, like the red cell at position (4,4) in the camera field of view CV (2,6,90°) in Example EX3, cells that are only partially contained within the camera field of view, but not entirely, may be excluded from the points calculation. Alternatively, for cells that are only partially contained within the field of view, like this cell, they may be used in the points calculation if a certain percentage or more of the cell's area is contained within the field of view.

[0090] The cell positions that do not fall under either of the two judgments in steps S211 and S212 (N X ,N Y ) and the angle θ can be treated as having a low self-location estimation accuracy because step S213 is skipped and information indicating that the self-location estimation accuracy is high is not recorded.

[0091] Example EX4 in FIG. 15 is an example that does not fall under either of the two judgments. X ,N YFor the camera field of view CV(8,8,90°) with θ=(8,8) and angle θ=90°, the number of objects included is only one, the connected region C4 that has been determined as an object (the connected region C3 that has been determined as an object is excluded from the object count because only a portion of it is included in the field of view below a certain percentage). Therefore, the threshold Th o By setting = 2, the number of objects below the threshold is one, and the first judgment is not established.

[0092] Similarly, for this camera field of view CV(8,8,90°), the total number of points in the region is 12 as calculated below, and the threshold Th p The second judgment is also invalid because the total point value is less than the threshold value due to the setting of = 20. In other words, there are three dark red cells within the camera's field of view CV (8,8,90°) (the two red cells that are only partially included in the field of view are excluded from the point calculation using the method described above), so the total point value can be calculated as follows: 3×4=12

[0093] As described above, the camera field of view CV (8,8,90°) of example EX4 does not meet either of the two judgment conditions, so information indicating that the self-location estimation accuracy is high is not stored, and the self-location estimation accuracy can be treated as being low.

[0094] In steps S211, S212, and S213, the cell position (N X ,N Y When it is determined that the self-location estimation accuracy is high for all combinations of the coordinates {(N X ,N Y ), θ} can be recorded in a well-known data notation format such as JSON (JavaScript Object Notation), as illustrated in example D1 of FIG.

[0095] FIG. 17 is a diagram showing an example of data D1 to D3 handled by the route setting unit 32 in a JSON format, and will be referred to as appropriate in the following description.

[0096] Next, in step S220, a route is planned. In route planning, the white cells (non-colored cells) on the 2D map are regarded as the area in which the robot can move, and an optimal route is calculated based on random sampling (Non-Patent Document 3) from the starting point and the destination. This results in the coordinates of the route plan in JSON format, such as example D2 in FIG. 17. [Non-Patent Document 3] "Yuki Hirase et al., Path Planning for Mobile Robots Considering Dynamic Environments", Robotics and Mechatronics Conference, 2019.

[0097] This route planning information may be obtained, for example, by tracing adjacent white cells on a two-dimensional map from a starting point to a destination, as shown below: Alternatively, the route may be obtained not in cell coordinates, but in physical location coordinates within the environment on the two-dimensional map, such as in meters: (1,1)→(1,2)→(1,3)→(1,4)→(2,4)→(3,4)→…

[0098] For the calculation of the optimal route, any existing algorithm for solving the optimal route problem, such as that described in Non-Patent Document 3, can be used. A method based on deep learning, etc. may also be used.

[0099] Next, the shortest route is matched with the coordinates of the cell with the highest accuracy of self-location estimation. The distance d between the two coordinates is calculated. Distance d is the Euclidean distance between the coordinates and is calculated using the following formula.

[0100]

number

[0101] In the above formula, (x pp ,y pp ,θ pp ) are the coordinates on the optimal route, of which (x pp ,y pp ) is the two-dimensional coordinate position, and θ pp is the position (xpp ,y pp ) on the path (the position (x pp ,y pp ) in the tangent direction at (x hl ,y hl ,θ hl ) is the coordinate that is judged to have high accuracy of self-localization. Note that, for cell coordinates, (N x ,N y ) is the integer coordinate (dimensionless) that indicates the lattice point position, and (x hl ,y hl ) is a coordinate that represents an actual physical distance on a two-dimensional map, such as in meters. (x hl ,y hl )=(d p ×N x ,d p ×N y ) and so on, they can be converted into each other.

[0102] Then, as shown in the following formula, the distance d (shortest distance) to the coordinates on the optimal route is a preset threshold Th d Coordinates (cells) within a distance that are determined to have high self-location estimation accuracy are determined to be waypoints. d <Th d

[0103] For example, if the coordinate id=1 in the route plan and the coordinate id=1 with high accuracy of self-location estimation, and the coordinate id=2 in the route plan and the coordinate id=2 with high accuracy of self-location estimation are determined to be waypoints, the waypoint is the starting point (S x ,S y ) and destination (G x ,G y ) and save it in the route memory. This will result in the coordinates of the waypoint in JSON format, as shown in data example D3 in Figure 17.

[0104] In this way, a waypoint set can be obtained as a set of points that are determined to be close to the optimal route and have high self-location estimation accuracy, with each waypoint that is an element of the set also being assigned order information corresponding to moving along the optimal route.

[0105] FIG. 18 is a diagram showing a schematic example of a waypoint. x ,S y ) to the destination (Gx, Gy), eight waypoints wp1 to wp8 are obtained in the vicinity of the optimal route BP, in the order of movement on the optimal route BP. In some cases, the waypoints may be obtained as points that match with points on the optimal route BP. In the example of FIG. 18, of the eight waypoints wp1 to wp8, two waypoints wp5 and wp8 are obtained as points on the optimal route BP. (Note that the matches shown here are matches in terms of two-dimensional coordinates on a map. As mentioned above, both the waypoints and points on the optimal route have directional components in addition to two-dimensional coordinates on a map. There may be cases where the matches include the directional components.)

[0106] As described later, since the waypoints are followed by the robot 4 in order to realize movement, it is preferable that the waypoints are set not so densely across adjacent cells, but so sparsely as shown in the schematic example of FIG. 18. For this reason, in the determination of step S221, a threshold value Th d may be set to a smaller value, or in the judgment in step S311 (FIG. 19) described later, the threshold value x may be set to a larger value so that even if waypoints are densely packed across adjacent cells, they can be treated as essentially a single waypoint.

[0107] <Driving control and map generation...Step S4> In step S4, travel control and map generation are performed. A map of the interior of the robot is generated while performing travel control on the robot 4. Fig. 19 is a flowchart showing the process of travel control and map generation according to one embodiment as details of step S4 in Fig. 3.

[0108] First, in step S301, the sensor unit 41 is operated inside the robot 4, and the map generation unit 42 processes the output of the sensor unit 41 to generate a map, and the process proceeds to step S302. The map may be generated by executing LiDAR-SLAM (Non-Patent Document 4) that generates a map by sensing the environment using a LiDAR sensor mounted on the robot 4 as the sensor unit 41, but is not limited to this and any algorithm for generating a map may be used. [Non-Patent Document 4] "Tomono et al., "Current Status and Future Prospects of SLAM," Institute of Systems, Control and Information Engineers, 2020."

[0109] In addition, map creation is started in step S301, and in the subsequent steps S302 to S312, the robot 4 moves from the starting point to the destination using the method of this embodiment. During the movement of the robot 4, the sensor unit 41 continuously acquires sensor data in real time, and the map generation unit 42 uses the acquired data to generate a map (generates a three-dimensional point cloud map). When the robot 4 reaches the destination, in step S313, map creation by the sensor unit 41 and the map generation unit 42 is terminated.

[0110] Note that the map created in step S301 is a map created when the robot 4 has an opportunity to move around the environment, and is not necessarily a map that needs to be stored in the 3D map storage unit 314 of this embodiment. Also, the information that the robot 4 refers to when moving in steps S302 to S312 is a route that has been created in advance and is stored in the route storage unit 323.

[0111] Next, in step S302, the operation memory unit 323 sequentially acquires waypoints from the route memory unit 323 and grasps the driving route. As illustrated in Fig. 18, each waypoint is assigned an order along the optimal route (different from the driving route).

[0112] Next, in step S303, an image is acquired from the environment photographing unit 43 as a camera mounted on the robot 4, and further in step S304, an image is cut out in frame units from the image by the image cutting out unit 331. This is for matching the image with a 3D map using self-position estimation in VPS (Non-Patent Document 1).

[0113] Next, self-location estimation is performed in the self-location estimation device 5 of the VPS device 6. In step S305, the location confirmation unit 332 transmits an image to the location estimation unit 51 using, for example, a REST API (Non-Patent Document 5) as a method for transmitting and receiving data such as an image in real time. [Non-Patent Document 5] "What is a REST API?", https: / / www.redhat.com / ja / topics / api / what-is-a-rest-api, (Accessed 2022-07-26).

[0114] In step S306, the position estimation unit 51 removes noise such as moving objects from the received image and extracts features. After that, the position estimation unit 51 solves the PnP problem (Non-Patent Document 6) that matches the features of the image with a 3D map (point cloud map) to calculate the position and orientation (X, Y, θ) at which the camera of the environment shooting unit 43 captured the image. That is, the position estimation unit 51 compares the image with the 3D map to estimate the self-position (X, Y, θ). After that, the self-position (X, Y, θ) is received by the position confirmation unit 332 in response to the REST API in the position estimation unit 5 on the VPS side. [Non-Patent Document 6] "PE Sarlin et al., "From Coarse to Fine: Robust Hierarchical Localization at Large Scale", CVPR, 2019."

[0115] Next, in steps S307 and S308, the motion control unit 44 geometrically calculates the position difference between its own position and the waypoint acquired by the motion planning unit 333 in step S302 and notified to the motion control unit 44. The purpose is to eliminate the position difference by moving the robot 4. Note that the robot 4 has a movement mechanism such as wheels, and is capable of moving forward, backward, and rotating (changing its orientation on a plane) without changing its position.

[0116] Specifically, in steps S307 and S308, as shown in example EX11 of FIG. 20, the current posture (the direction vector v of the robot 4 moving forward on the two-dimensional map plane) is calculated. r ) and the current waypoint W1, and the distance r between the waypoint W1 and the current position (X, Y). Here, the direction vector v r and vector v with the current position (X, Y) of robot 4 as the starting point and waypoint W1 as the end point. w Using this, the angle θ is calculated as follows:

[0117]

number

[0118] FIG. 20 is a diagram showing an explanatory example of the movement control of the robot 4 as examples EX11 to EX14, and will be further referred to below.

[0119] Next, in step S309, the operation control unit 44 controls the robot 4 to rotate until θ=0 so as to change the state from example EX11 in Fig. 20 to example EX12. Next, in step S310, the operation control unit 44 controls the robot 4 to move forward so as to change the state from example EX12 in Fig. 20 to example EX13.

[0120] During this movement, the position P r =(X,Y) and waypoint position Pw=(X w ,Y wThe distance r between the robot 4 and the target position Pr is calculated in real time as follows: Since the real-time video is analyzed and estimated by the position estimation unit 51, the position Pr=(X, Y) of the robot 4 can also be calculated in real time. r=|P r -P w |

[0121] These steps (moving robot 4 forward while calculating distance r and angle θ in real time) are repeated, and in step S311, it is determined that robot 4 has arrived at waypoint W1 when r falls within a predetermined threshold value x (r≦x), as shown in example EX14 in Figure 20.

[0122] In step S311, if it is determined that the current waypoint has been reached based on the above threshold determination r≦x, the process proceeds to step S312, and if it is determined that the current waypoint has not been reached, the process returns to step S303 and is repeated, thereby achieving the process of moving the robot 4 forward toward the waypoint while calculating the distance r and angle θ in real time.

[0123] In step S312, it is determined whether the robot 4 has reached the destination. If it has reached the destination, the process proceeds to step S313, where the map (the map created in step S310 and created continuously by the map creator 42 along the movement of the robot 4 using a technique such as LiDAR-SLAM) is saved, and the flow of FIG. 19 is terminated. Note that the determination of arrival at the destination in step S312 may be performed, for example, by determining whether the destination (G x ,G y ) is treated as the last waypoint in the sequence, and the determination in step S311 is made. The waypoint determined to have been reached in step S311 is the last destination (G x ,G y ), the determination in step S312 may be regarded as being true.

[0124] If it is determined in step S312 that the robot 4 has not reached the destination, the process returns to step S302, and the next waypoint is acquired (in the example of FIG. 20, if the currently reached waypoint is W1, the next waypoint is W2), and the above process is repeated. X ,S Y )→wp1→wp2→…→wp7→wp8→destination (G X ,G Y ) the robot 4 can move from a starting point to a destination by passing through waypoints in order.

[0125] Here, the waypoints are estimated in advance as having high self-location estimation accuracy with respect to the two-dimensional position and the camera direction. Therefore, according to this embodiment, it is possible to avoid the risk of the robot 4 becoming unable to move midway due to a decrease in the accuracy of self-location estimation by the VPS technique, and the robot 4 can reach the destination without the need to move the robot 4 around the environment in advance. Furthermore, when the robot 4 reaches the destination, the map of the robot 4 is also completed by the processing of the map generation unit 42.

[0126] Various supplementary examples, additional examples, alternative examples, etc. will be described below.

[0127] As an application example of the embodiment of the present invention, it becomes possible to perform work utilizing the robot 4 more smoothly in an environment such as a work site by remote control (centralized management by a remote manager using the map utilization device 3). That is, according to this embodiment, it is not essential to operate the robot 4 in the actual environment in advance, and a movement route with high self-location estimation accuracy by the VPS method can be set, so that the preparation work etc. is reduced and remote work etc. can be performed more smoothly.

[0128] This will eliminate the need for staff to physically go to the work site, and by saving the energy resources required for human transportation, it will be possible to reduce carbon dioxide emissions, thereby contributing to Goal 13 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Take urgent action to combat climate change and its impacts."

[0129] 21 is a diagram showing an example of the hardware configuration of a general computer device 70. Each of the environmental imaging device 1, the VPS device 6, the map utilization device 3, and the robot 4 constituting the map utilization system 100 can be realized as one or more computer devices 70 having such a configuration. When a partial configuration of the map utilization system 100 is realized by two or more computer devices 70, information required for processing may be transmitted and received via a network. The computer device 70 includes a CPU (Central Processing Unit) 71 for executing predetermined instructions, a GPU (Graphics Processing Unit) 72 as a dedicated processor for executing some or all of the instructions executed by the CPU 71 in place of the CPU 71 or in cooperation with the CPU 71, a RAM 73 as a main storage device providing a work area for the CPU 71 (and the GPU 72), a ROM 74 as an auxiliary storage device, a communication interface 75, a display 76, an input interface 77 for receiving user inputs via a mouse, a keyboard, a touch panel, or the like, a camera 81 and a sensor 82 used for the self-position estimation function of the robot 4, an actuator 83 such as a motor and a moving mechanism 84 such as wheels (including a transmission mechanism such as an axle and gears) driven by the actuator 83 for providing an autonomous moving function of the robot 4, and a bus BS for transmitting and receiving data between them. When the inputs in the above-mentioned steps S102, S103 (FIG. 5) and steps S202, S204 (FIG. 12) are received as user inputs, they may be received via the input interface 77.

[0130] Each functional unit of the map utilization system 100 can be realized by a CPU 71 and / or a GPU 72 that reads a predetermined program corresponding to the function of each unit from a ROM 74 and executes it. Both the CPU 71 and the GPU 72 are a type of computing device (processor). Here, when display-related processing is performed, a display 76 configured as a monitor such as a liquid crystal display or a lighting device such as an LED also operates in conjunction with the above, and when communication-related processing related to data transmission and reception is performed, a communication interface 75 also operates in conjunction with the above.

[0131] In the robot 4, the environmental capture unit 4 and sensor unit 41 are realized using inputs from a camera 81 and a sensor 82, and actuators 83 such as motors are driven by control output from the operation control unit 44, and movement mechanisms 84 such as wheels are operated, thereby enabling the robot 4 to move. [Explanation of symbols]

[0132] 100...map utilization system, 1...environmental photography device, 2...3D map creation device, 3...map utilization device, 4...robot, 5...self-location estimation device, 6...VPS device 21...3D map creation unit, 22...3D map storage unit, 31...2D map creation unit, 32...route setting unit, 33...driving control unit, 41...sensor unit, 42...map generation unit, 43...environmental photography unit, 44...operation control unit, 51...position estimation unit 311: 3D map processing unit, 312: polygon generation unit, 313: cell generation unit, 314: 2D map storage unit, 321: departure point / destination setting unit, 322: route generation unit, 323: route storage unit, 331: image extraction unit, 332: position confirmation unit, 333: motion planning unit

Claims

1. a first process for generating a two-dimensional map of a range in which the moving object may move from a three-dimensional map of the field; a second process of setting a route from a departure point to a destination in the two-dimensional map as a route that follows a plurality of waypoints that are determined to have high accuracy when estimating a self-position of the moving body by comparing an image captured by a camera of the moving body with the three-dimensional map; In the first process, the two-dimensional map is created by providing, for each unit cell obtained by dividing an area of ​​the two-dimensional map, information on whether the moving body can move and, if the moving body cannot move, information on the reliability of the existence of the object; In the second process, for each of the unit cells of the two-dimensional map that are classified as movable, the reliability information of unit cells that are included within the field of view of the camera of the moving body at the position of the unit cell is added up for each camera direction, thereby determining a unit cell and camera direction for which the accuracy of the self-position estimation is determined to be high.

2. a first process for generating a two-dimensional map of a range in which the moving object may move from a three-dimensional map of the field; a second process of setting, on the two-dimensional map, a route from a starting point to a destination as following a plurality of waypoints that are determined to have high accuracy when the moving body estimates its own position by comparing an image captured by a camera of the moving body with the three-dimensional map, and that are determined by determining that the accuracy of the self-position estimation at the position and camera orientation at the waypoint is high.

3. 3. The map utilization device according to claim 2, characterized in that in the second process, an optimal route for reaching the destination from the starting point by tracing a movable area on the two-dimensional map by solving an optimal route problem is determined as a series of positions and camera orientations, and points that are determined to have a high accuracy of the self-location estimation and are determined to have a position and camera orientation close to a point on the optimal route are arranged in order along the progression of the optimal route and are determined as the waypoints.

4. The map utilization device according to claim 1, characterized in that in the first process, a range from the height of the moving surface of the moving body to the height of the moving body is extracted from the three-dimensional map, and a distinction is made between whether the moving body can move or cannot move within that range, thereby creating the two-dimensional map.

5. 2. The map utilization device according to claim 1, wherein in the second process, an optimal route is determined by solving an optimal route problem to trace a movable area on the two-dimensional map from the starting point to the destination, and unit cells that are determined to be close to a point on the optimal route and for which the accuracy of the self-location estimation is determined to be high are arranged in order along the progression of the optimal route and are then determined as the waypoints.

6. The map utilization device according to claim 5, characterized in that in the second process, unit cells and camera directions that are determined to have high accuracy of the self-location estimation and that are located close to a point on the optimal route and whose camera direction is determined to be close to the direction of the optimal route at that point are arranged in order along the progression of the optimal route, and are then determined as the waypoints.

7. 2. The map utilization device according to claim 1, further comprising a third process for controlling the moving object to move from the departure point to the destination by passing through the waypoints in order.

8. In the third process, The moving object estimates its own position in real time by comparing an image captured by a camera of the moving object in real time with the three-dimensional map; 8. The map utilization device according to claim 7, further comprising: a command for moving the mobile unit so that the self-position estimated in real time passes through the waypoints in order.

9. a first step of creating a two-dimensional map of a range in which the moving object may move from a three-dimensional map of the field; a second step of setting, in the two-dimensional map, a route from a departure point to a destination through a plurality of waypoints that are determined to have high accuracy when estimating a self-position of the moving body by comparing an image captured by a camera of the moving body with the three-dimensional map; In the first step, the two-dimensional map is created by providing, for each unit cell obtained by dividing an area of ​​the two-dimensional map, information on whether the moving object can move and, if the moving object cannot move, information on the reliability of the object's existence; The second step is a map usage method characterized in that, for each unit cell of the two-dimensional map that is classified as movable, a unit cell and camera direction for which the accuracy of the self-location estimation is determined to be high is determined by adding up the reliability information of unit cells that are included within the field of view of the camera of the moving body for each camera direction at the position of the unit cell.

10. a first step of creating a two-dimensional map of a range in which the moving object may move from a three-dimensional map of the field; a second step of setting, on the two-dimensional map, a route from a starting point to a destination as following a plurality of waypoints that are determined to have high accuracy when the moving body estimates its own position by comparing an image captured by a camera of the moving body with the three-dimensional map, and that are determined by determining that the accuracy of the self-position estimation at the position and camera orientation at each waypoint is high.

11. The method of using a map according to claim 10, characterized in that in the second step, an optimal route for reaching the destination from the starting point by tracing a movable area on the two-dimensional map by solving an optimal route problem is defined as a series of positions and camera orientations, and points that are determined to have a position and camera orientation close to a point on the optimal route and for which the accuracy of the self-location estimation is determined to be high are arranged in order along the progression of the optimal route and are defined as the waypoints.

12. 9. A program for causing a computer to function as the map utilization device according to claim 1.

Images