Information processor, information processing method, and computer program

Abstract

To provide an information processor capable of calculating an activation location posture according to a type of operation.SOLUTION: An information processor includes: operation setting information acquisition means for acquiring a type of operation performed by a moving body, information related to an operation region, and map information for estimating a previously created self location; and activation location posture calculation means for calculating an activation location posture upon start of the operation based on the type of the operation, the operation region, and the map information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a computer program, and the like. [Background technology]

[0002] Autonomous mobile objects are commonly used, such as automated guided vehicles used in factories and warehouses and autonomous mobile floor-cleaning robots used in commercial facilities, which estimate their own position and orientation based on data obtained from various sensors and travel autonomously to perform specified tasks.

[0003] When an autonomous mobile vehicle performs a task such as cleaning or transportation, a restricted area may be designated in addition to the work area. In such cases, a method has been proposed to determine the starting point of the task based on the relationship between the work area and the restricted area of ​​the autonomous mobile vehicle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2023-501831 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the method of Patent Document 1, when starting self-location estimation, there is a possibility that the self-location estimation to estimate which position and orientation the current position and orientation is on the map may fail, resulting in a problem that the task cannot be started.

[0006] An object of the present invention is to provide an information processing device that can calculate an activation position and posture according to the type of work. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present invention includes: a task setting information acquisition means for acquiring information on the type of task to be performed by the mobile object, information on the task area, and map information for self-position estimation created in advance; an activation position and orientation calculation means for calculating an activation position and orientation at the start of the work based on the type of work, the work area, and the map information; The present invention is characterized by having the following. [Effects of the Invention]

[0008] According to the present invention, it is possible to realize an information processing device that can calculate a startup position and orientation according to the type of work. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a conceptual diagram of a usage scene of a moving body 200 equipped with an information processing device according to a first embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an example of the hardware configuration of an information processing device 100 according to a first embodiment. [Figure 3] 1 is a functional block diagram showing an example of the logical configuration of an information processing device 100 according to a first embodiment and a moving object 200 on which the information processing device 100 is mounted. [Figure 4] 4 is a flowchart showing a processing example of an information processing method using the information processing device according to the first embodiment. [Figure 5] 5 is a flowchart showing an example of a process for determining a startup position and attitude in the information processing method according to the first embodiment. [Figure 6] 10 is a flowchart showing a detailed example of step S203 for determining an activation position and attitude in the first embodiment. [Figure 7] FIG. 13 is a diagram showing an example of an occupancy grid map created by LiDAR in the fourth embodiment. [Figure 8] 13 is a flowchart illustrating in detail an example of a process for determining the start-up position and attitude (step S203) in the fourth embodiment. [Figure 9] FIG. 13 is a diagram showing an example of an operation screen in the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each drawing, the same members or elements are designated by the same reference numerals, and duplicate descriptions will be omitted or simplified.

[0011] <Embodiment 1> In the first embodiment, an autonomous mobile floor cleaning robot (mobile body) that automatically cleans floor surfaces in commercial facilities, etc., will be described as the mobile body according to the embodiment. However, the mobile body according to the embodiment may be a robot that performs other tasks, or may be a vehicle or drone that performs automatic driving or driving assistance.

[0012] In this embodiment, the target of self-localization of a moving body is the position and orientation in real space of a reference coordinate system fixed to the moving body. The moving body estimates the position and orientation of the sensor based on information acquired from the sensor, and calculates the position and orientation of the moving body from the relationship between the reference coordinate system and the sensor mounting position and orientation. In the following embodiments, the position and orientation refers to both the position and orientation.

[0013] In this embodiment, when starting work by an autonomous mobile body, a candidate that is likely to be successful in global self-localization and that is close to the movement start point for cleaning work is preferentially selected from a group of position and posture candidates, and this is set as the starting position and posture. This makes it possible to reduce unnecessary movement other than work and increase the possibility of starting work.

[0014] The mobile body described in this embodiment creates a map for estimating its own position in advance using SLAM (Simultaneous Localization and Mapping). SLAM is a technology that places sensors such as distance sensors and stereo cameras on the mobile body and estimates the position and orientation of the sensors and a map of the surrounding environment.

[0015] When performing tasks such as floor cleaning, the autonomous vehicle estimates its own position based on a map created using SLAM (hereafter referred to as the SLAM map) and the sensor values ​​of the sensors installed on the autonomous vehicle.

[0016] In this embodiment, the moving body is equipped with a stereo camera facing in the direction of travel, and an information processing device mounted on the moving body detects feature points from images acquired from the left and right lenses of the stereo camera, and performs 3D geometric measurement by stereo matching to estimate the 3D position and orientation of the stereo camera, thereby calculating the position and orientation of the moving body.

[0017] A SLAM map created using a stereo camera includes information on multiple spatially discrete positions and orientations, image data captured at each position and orientation, image coordinate information of groups of feature points detected from each image data using the AKAZE algorithm, and image feature information, etc. AKAZE is an abbreviation for Accelerated KAZE Features.

[0018] Here, a set of image data, the position and orientation of the moving object when the image data was captured, image coordinate information of feature points detected from the image data, image feature information, and 3D position and orientation information of feature points calculated by stereo vision, etc. is called a keyframe. In this way, map information includes point cloud feature information based on point cloud data acquired from a ranging sensor such as a stereo camera.

[0019] When a mobile object in this embodiment starts estimating its own position by referencing a SLAM map to perform a task such as floor cleaning, it first extracts image features from the current camera image using the AKAZE algorithm, and then searches the SLAM map for keyframes containing scenes similar to the extracted image features using the Bag of Words (BoW) algorithm.

[0020] The current position and orientation in the map coordinate system is estimated by referencing the position and orientation of key frames with similar scenes found on the map. Then, each time the stereo camera image frame is updated, the results of self-localization are updated by calculating the relative position and orientation change based on the previous position and orientation.

[0021] 1 is a conceptual diagram of a usage scene of a mobile object 200 equipped with an information processing device according to the first embodiment of the present invention. Management of the mobile object 200 and the work content to be performed by the mobile object 200 are set via a mobile object control terminal 400.

[0022] The work setting information set in the mobile body 200 when setting the work content includes a map name for identifying a map to be referred to by the mobile body 200 while traveling, type information for identifying the type of work, coordinate information according to the type of work, and information on the work area. Note that the coordinate information is information including at least one XY coordinate in the map coordinate system on which the mobile body 200 travels.

[0023] Here, the type of work refers to cleaning work using teaching playback, cleaning work using zigzag travel, etc. Coordinate information for cleaning work using teaching playback is a record of multiple X and Y coordinates and direction θ of the travel of the mobile object 200 in chronological order. Coordinate information for cleaning work using zigzag travel is a record of the X and Y coordinates of two diagonal vertices when the work area is represented as a rectangle.

[0024] Hereinafter, the information set in the mobile body 200 using the mobile body control terminal 400 will be referred to as "work setting information." The mobile body 200 autonomously moves through a work area set by the mobile body operator via the mobile body control terminal 400, and performs cleaning work while autonomously traveling by driving the cleaning unit.

[0025] When setting a travel route for cleaning work of the mobile body 200, the mobile body operator displays the SLAM map stored in the mobile body 200 on the mobile body control terminal 400 and drags the pointing device to set the desired travel route, thereby performing travel route teaching.

[0026] The mobile object control terminal 400 superimposes on the map an icon image corresponding to the mobile object 200 indicating the startup position and attitude determined by the information processing device 100. This allows the mobile object operator to confirm the startup position and attitude of the mobile object 200 determined by the information processing device 100.

[0027] 2 is a diagram showing an example of the hardware configuration of the information processing device 100 according to embodiment 1. The information processing device 100 is configured with a CPU 150, a RAM 151, a storage unit 152 such as an HDD or SSD, a communication unit 153, a system bus 154, and the like.

[0028] The CPU 150 uses the RAM 151 as a work memory, executes an operating system (OS) and various computer programs stored in the storage unit 152, and controls each unit via the system bus 154. The computer programs executed by the CPU 150 include a computer program for proposing a work start position for the mobile object 200, etc.

[0029] Fig. 3 is a functional block diagram showing an example of a logical configuration of the information processing device 100 according to the first embodiment and a mobile object 200 equipped with the information processing device 100. Note that some of the functional blocks shown in Fig. 3 are realized by causing a CPU or the like serving as a computer included in the mobile object or the information processing device 100 to execute a computer program stored in a memory serving as a storage medium.

[0030] However, some or all of these functions may be implemented by hardware. Examples of hardware that can be used include dedicated circuits (ASICs) and processors (reconfigurable processors, DSPs). Furthermore, the functional blocks shown in Fig. 3 do not have to be built into the same housing, and may be configured as separate devices connected to each other via signal paths.

[0031] In the first embodiment, the mobile object 200 includes an information processing device 100, a communication unit 201, a map storage unit 202, a route generation unit 204, a self-position estimation unit 205, a control unit 206, etc. However, some of the functional blocks shown in Fig. 3 (for example, the information processing device 100) may be provided in a server provided outside the mobile object, a mobile object control terminal 400, etc.

[0032] The information processing device 100 includes an activation position / posture priority condition setting unit 102, an activation position / posture determination unit 103, an activation position / posture candidate information acquisition unit 104, and the like, and determines an activation position / posture of the moving object 200.

[0033] The communication unit 201 receives work setting information from the mobile object control terminal 400 via wireless communication, and notifies the work setting information to the startup position / attitude priority condition setting unit 102 of the information processing device 100. Based on the work setting information from the mobile object operator input via the communication unit 201, the information processing device 100 starts processing to determine the startup position / attitude, which is the position / attitude at the start of startup of self-location estimation, such as when starting work on the mobile object 200.

[0034] The map storage unit 202 stores a SLAM map necessary for the mobile body 200 to estimate its own position. The route generation unit 204 generates a travel route for the mobile body 200 based on input from the mobile body operator and the starting position and attitude determined by the information processing device 100.

[0035] The control unit 206 has a built-in CPU and controls a self-position estimation unit 205 that estimates the self-position of the moving body 200, and various actuators based on the self-position estimation result of the self-position estimation unit 205 in order to travel along the route generated by the route generation unit 204.

[0036] The activation position / orientation priority condition setting unit 102 sets which position / orientation is to be preferentially selected when determining the activation position / orientation. That is, the activation position / orientation priority condition setting unit 102 sets a priority for each of a plurality of activation position / orientation candidates. The activation position / orientation candidate information acquisition unit 104 acquires activation positions / orientations that are candidates for the activation position / orientation and information related thereto.

[0037] The activation position and attitude determination unit 103 determines an activation position and attitude at the start of work based on the information acquired by the activation position and attitude candidate information acquisition unit 104 and the priority set by the activation position and attitude priority condition setting unit 102. Then, the determined activation position and attitude is transmitted to the path generation unit 204 and also transmitted (notified) to the mobile object control terminal 400.

[0038] Here, the processing flow will be explained using Fig. 4 to Fig. 6. First, Fig. 4 is a flowchart showing a processing example of an information processing method using the information processing device according to embodiment 1. Note that the operations of the steps in the flowcharts of Fig. 4 to Fig. 6 are performed sequentially by the CPU or the like serving as a computer within the information processing device 100 or the control unit 206 executing a computer program stored in memory.

[0039] The sequence of the entire system of the mobile body 200 when a cleaning job is specified will be described in Fig. 4. Next, a detailed flow of determining the start position and attitude will be described in Fig. 5. Fig. 4 shows the processing when a mobile body operator sets a job such as cleaning, the information processing device 100 calculates start position and attitude candidate information, and the mobile body 200 starts performing the job.

[0040] The mobile body 200 waits until the mobile body operator inputs work setting information via the communication unit 201, and starts the subsequent processing when it receives the work setting information such as the work type in step S101.

[0041] The SLAM map used is created in advance by the mobile operator using the mobile unit 200, and the type of work is a cleaning operation recorded using the teaching playback method. Note that the work setting information here includes at least information on the type of work to be performed by the mobile unit and the work area, as well as map information for self-position estimation created in advance.

[0042] Step S101 also functions as a task setting information acquisition step (task setting information acquisition means) that acquires information on the type of task to be performed by the mobile body and the task area, and map information for self-position estimation created in advance.

[0043] Upon receiving the task setting information, the information processing device 100 calculates and determines the activation position and orientation in step S102. Here, step S102 functions as an activation position and orientation calculation step (activation position and orientation calculation means) that calculates the activation position and orientation at the time of starting the task based on the task type, task area, and map information.

[0044] After the information processing device 100 calculates the start-up position and attitude, the moving body 200 transmits the start-up position and attitude to the moving body operator's moving body control terminal 400 and the path generating unit 204 via the communication unit 201 in step S103.

[0045] Next, in step S104, the path generation unit 204 generates a path for performing a predetermined task based on the startup position and posture determined by the information processing device 100. Next, in step S105, upon receiving a startup command from the communication unit 201, the moving body 200 starts the self-position estimation unit 205 and starts estimating its own position.

[0046] Finally, in step S106, the control unit 206 controls an actuator (e.g., a motor) of the moving object 200 based on the travel route generated by the route generation unit 204 and the position and orientation estimated by the self-position estimation unit 205. This causes, for example, a cleaning mop to rotate, and cleaning work begins.

[0047] If the start position and posture of the work is specified as in the teaching playback method, a travel path from the starting position and posture to the work start position and posture is generated in the travel path generation process of step S104. Then, after moving to the work start position, playback operation based on the teaching content is started.

[0048] Fig. 5 is a flowchart showing an example of processing for determining the activation position and attitude in the information processing method according to the first embodiment, and shows details of step S102 in Fig. 4. When the information processing device 100 receives work setting information set by the mobile body operator using the mobile body control terminal 400, it starts processing for determining the activation position and attitude as shown in Fig. 5.

[0049] First, in step S201, the activation position and orientation candidate information acquisition unit 104 projects the three-dimensional position and orientation of a key frame included in the SLAM map to be used onto a two-dimensional position and orientation, which are the position on the XY plane on which the moving object 200 travels and the orientation θ of the moving object 200. Then, the two-dimensional position and orientation obtained by the projection are acquired as activation position and orientation candidate information.

[0050] In the following embodiments, the position of the moving body 200 is represented by coordinates on an XY plane in the map coordinate system on which the moving body 200 travels, and the attitude of the moving body 200 is represented by a direction θ in the map coordinate system that indicates the traveling direction of the moving body 200. The X-axis direction in the map coordinate system is set to θ=0.

[0051] In step S202, the activation position / posture priority condition setting unit 102 sets priority conditions based on the work setting information (e.g., work type, etc.) set by the mobile body operator and received via the communication unit 201. That is, in step S202, the priority is set based on the work type, etc.

[0052] In addition, the work type in this embodiment is a cleaning operation recorded using a teaching playback method, and the cleaning operation data includes the XY coordinates and direction θ of the mobile body 200 in the map coordinate system recorded in chronological order according to the route traveled by the mobile body 200.

[0053] In this embodiment, the priority condition is set so that the closer the start position and posture candidate information is to the work start position and posture and the smaller the difference in posture is, the higher the priority is. Note that the work start position and posture is the position and posture recorded first in the cleaning operation data.

[0054] In step S203, the activation position / posture determination unit 103 determines the activation position / posture based on the priority condition and image information and the like included in the key frames corresponding to each of the activation position / posture candidates acquired by the activation position / posture candidate information acquisition unit 104.

[0055] FIG. 6 is a flowchart showing a detailed example of step S203 for determining the activation position and attitude in the first embodiment.

[0056] In step S301 of FIG. 6, the start-up position and attitude determination unit 103 evaluates the start-up position and attitude candidates acquired in step S201 from the perspective of the success probability of global self-localization.

[0057] The evaluation is performed based on the number and distribution of feature points included in the key frames used to calculate each candidate launch position and orientation. That is, the evaluation is performed based on the number and distribution of feature points included in the key frames corresponding to each candidate launch position and orientation. Here, it is determined that the greater the number of detected feature points and the wider the distribution of the detected points in the image coordinates, the higher the probability of success of global position estimation.

[0058] Specifically, in step S301, the number of image features included in the camera image of the key frame is counted, and the camera image is divided into, for example, a 6 × 8 grid, and the number of grids including feature points is counted and recorded in association with the candidate activation position and orientation. This process is performed for all candidate activation positions and orientations.

[0059] In step S302, the start-up position / posture determination unit 103 determines, from the start-up position / posture candidates, positions / postures in which the number of recorded feature points is less than, for example, 30, or the number of grids including recorded feature points is less than, for example, 20, as unsatisfied positions / postures. Then, the start-up position / posture determination unit 103 excludes the unsatisfied positions / postures from the start-up position / posture candidates.

[0060] Next, in step S303, the position and posture are evaluated based on the task priority. That is, the activation position and posture determination unit 103 evaluates the priority of the activation position and posture candidates based on the conditions set by the activation position and posture priority condition setting unit 102, and calculates an evaluation value p. This evaluation is performed on the activation position and posture candidates that were not excluded in step S302.

[0061] Here, the evaluation value p related to priority is calculated using, for example, the following formula 1. The smaller the value of p, the higher the priority. Here, the first term on the right side is the evaluation of the position, and the second term is the evaluation of the posture. x and y are the coordinates on the XY plane along which the moving body moves, and x0 and y0 are the start coordinates of the task on the XY plane.

[0062] The angdiff function takes two attitude angles as arguments and returns the angular difference between those attitude angles. θ is the attitude of the moving body, and θ0 is the attitude of the moving body at the start of the task. k1 and k2 are weighting coefficients, respectively.

number

[0063] In step S304, the activation position / posture determination unit 103 determines the position / posture with the highest priority (i.e., the smallest evaluation value p) from among the activation position / posture candidates whose priorities have been calculated, as the activation position / posture.

[0064] As described above, according to this embodiment, it is possible to determine a starting position and attitude that can satisfy both work and traffic constraints and ensure successful global self-localization at the start of self-localization, such as when starting work.

[0065] <Variation 1-1> In this embodiment, the camera image is divided into a 6 × 8 grid to evaluate the distribution of feature points. Also, positions and orientations corresponding to key frames in which the number of feature points is 30 or less or the number of grids including feature points is 20 or less are deleted from the start-up position and orientation candidates, but these values ​​are merely examples and are not limited to these values ​​and any values ​​may be used.

[0066] <Variation 1-2> Furthermore, in the above embodiment, cleaning of a filled-in area and cleaning of a designated area by teaching playback were described, but area designation is not limited to this. For example, an area that the mobile body operator wants to designate as a starting position and posture, separate from the work area, may be set via the mobile body control terminal 400. Also, in the case of an autonomous mobile transport robot, a method may be used in which only the start point and end point of movement are designated.

[0067] <Variation 1-3> Furthermore, in the above embodiment, positions and postures corresponding to key frames in which the number of feature points is 30 or less or the number of grids including feature points is 20 or less are deleted from the start-up position and posture candidates, and then the position and posture in which the value of the evaluation value p related to priority is smallest is determined as the start-up position and posture.

[0068] However, it is also possible to calculate an overall evaluation value e from the number, distribution, and priority of feature points using, for example, the following formula 2, and determine the start-up position and posture based on the overall evaluation value e. Here, Pf is the number of feature points detected at the detected position and posture, Pg is the number of grids containing feature points, and k3, k4, and k5 are weighting coefficients, respectively.

number

[0069] <Variation 1-4> In this embodiment, when obtaining candidate information for the startup position and posture in step S201, the position and posture information of the keyframes stored in the SLAM map is used. However, candidate information for the startup position and posture may be obtained by other methods as long as the likelihood of success of self-location estimation and the task priority can be calculated from the position and posture.

[0070] That is, in this modified example 1-4, the method of obtaining candidate information for the activation position and orientation in step S201 and the method of evaluating the number and distribution of feature points during activation position and orientation determination in step S203 (step S301) are different.

[0071] When acquiring candidate information for the start-up position and orientation in step S201, the entire area of ​​the SLAM map is divided into grids of a predetermined size. The center coordinates of each grid and the combination of the orientations obtained by rotating a predetermined angle around each center coordinate are set as candidates for the start-up position and orientation.

[0072] Next, we will explain the method for evaluating the number and distribution of feature points in step S301 in this modification 1-4. In the position and posture that are candidates for the activation position and posture, feature points that are within a predetermined distance and that fall within the camera's angle of view when photographed in that posture are extracted from the SLAM map.

[0073] The extracted feature points are then projected onto an image plane based on the focal length and resolution of the camera. The number of feature points projected onto the image plane and the number of grids containing feature points by dividing the image plane into grids of a predetermined size are counted, and these are recorded in association with the target candidate launch position and orientation. This process is performed for all candidate launch positions and orientations. After that, the launch position and orientation are determined by performing the same process from step S302 onwards.

[0074] By doing as described above, according to this embodiment, it is possible to determine a startup position and orientation suitable for startup from a wider range, regardless of the route traveled when the map was saved.

[0075] <Embodiment 2> In the first embodiment, the startup position and attitude priority condition setting unit 102 determines that the larger the number of feature points included in a key frame and the wider the area in which they are distributed, the more suitable it is for global self-location estimation.

[0076] However, the method for setting the initial position and posture priority conditions is not limited to this. In the second embodiment, an example will be described in which, when setting the priority conditions in the startup position and posture priority condition setting unit 102 in step S202, the priority is determined by focusing on the ease of self-position estimation.

[0077] For example, the appearance of architectural structures such as ceilings and pillars generally does not change easily depending on the date or time. On the other hand, the position of objects such as chairs and cardboard boxes may change on a daily basis. Therefore, if feature points are detected for a chair or cardboard box before it is moved, the feature points detected before the movement cannot be used if the chair or cardboard box is then moved, making it unsuitable for determining the startup position and posture.

[0078] Therefore, in the second embodiment, the start-up position / posture priority condition setting unit 102 sets conditions so that the position / posture in which more architectural structures with invariable arrangements of ceilings, pillars, etc. are observed is evaluated with a higher priority. At this time, the start-up position / posture determination unit 103 applies semantic segmentation to images included in key frames corresponding to each start-up position / posture candidate to detect walls and ceilings.

[0079] Then, feature points are detected in the image areas of the detected walls and ceilings, and the greater the number of feature points and the wider the area over which they are distributed, the higher the evaluation priority is. Note that the start-up position and orientation priority condition setting unit 102 may identify movable or moving objects such as doors and curtains by object recognition, and set conditions such that a position and orientation in which an image including such an object is obtained is evaluated as inappropriate as the start-up position and orientation.

[0080] That is, the activation position / posture determination unit 103 performs object recognition on images included in key frames associated with each activation position / posture candidate, and recognizes movable or moving objects such as doors and curtains. It calculates the area of ​​the image including the recognized object, and evaluates it so that the larger the calculated area, the lower the priority.

[0081] The startup position and orientation priority condition setting unit 102 may also set conditions to lower the priority of environments that are prone to erroneous position and orientation recognition. For example, checkerboard patterns, large lattice windows, and fully tiled walls have similar detectable image features that tend to lead to erroneous self-position recognition. Such environments may be set as conditions that are inappropriate for the startup position and orientation.

[0082] That is, the activation position and orientation determination unit 103 detects a periodic structure from an image included in a key frame associated with each activation position and orientation candidate using a machine learning algorithm or frequency analysis, calculates the area of ​​the image region of the detected periodic structure, and evaluates the area so that the larger the calculated area, the lower the priority.

[0083] According to the second embodiment as described above, it is also possible to determine an actuation position and attitude suitable for actuation at a location where a moving object is placed.

[0084] <Embodiment 3> In the first embodiment, the startup position and attitude priority condition setting unit 102 determines that the larger the number of feature points included in a key frame and the wider the area in which they are distributed, the more suitable it is for global self-location estimation.

[0085] However, the method for setting the initial position and posture priority conditions is not limited to this. In the third embodiment, when the start position and posture priority condition setting unit 102 sets priority conditions in step S202, the priority is determined by focusing on whether the position and posture are appropriate as the work start position and posture. That is, the priority is set based on the shape (circular, square, etc.) or layout (center of gravity position, etc.) or movement pattern (zigzag, spiral, etc.) of the work area.

[0086] In setting the operation of the work, it is possible to perform cleaning work by moving in a zigzag pattern or a spiral pattern in the work area specified by the mobile body operator on the mobile body control terminal 400. In this case, in the third embodiment, the robot may start moving from the periphery of the work area.

[0087] In this case, the activation position / orientation priority condition setting unit 102 sets conditions to increase the priority of areas closer to the periphery of the working area. To achieve this, the activation position / orientation determination unit 103 calculates, for example, the distance between the position of the activation position / orientation candidate and the periphery of the working area, and evaluates the shorter the distance, the higher the priority.

[0088] On the other hand, when setting the operation of the work, if the mobile body operator performs cleaning work by traveling in a spiral pattern from the center of the work area toward the periphery of the work area specified by the mobile body control terminal 400, traveling starts near the center of the work area. In this case, the start position / posture priority condition setting unit 102 may set a condition to increase the priority of an area close to the centroid of the work area.

[0089] In this case, the activation position / orientation determination unit 103 calculates the coordinates of the center of gravity of the working area. Then, the distance between the position of the activation position / orientation candidate and the center of gravity of the working area is calculated, and the shorter the distance, the higher the evaluation priority. Note that the working area may be approximated by a quadrangle or an ellipse, and the center of the quadrangle or ellipse may be used as the center of gravity.

[0090] Alternatively, in setting up the operation of a task, the mobile body operator may specify the task content and also directly specify the desired starting position and attitude using the mobile body control terminal 400. In this case, the mobile body 200 will start up in the starting position and attitude desired by the mobile body operator, then autonomously travel to the task area, and then start the task.

[0091] At this time, it is desirable that the activation position / attitude priority condition setting unit 102 sets conditions such that the priority is increased as the position / attitude is closer to the activation position / attitude desired by the mobile body operator set via the mobile body control terminal 400. In other words, the priority may be set based on the activation position / attitude of the mobile body.

[0092] Alternatively, the activation position / posture priority condition setting unit 102 may set the priority condition so that the activation position / posture that minimizes the total travel distance or the required time from activation to completion of the work has a higher priority.

[0093] In this case, the activation position / posture determination unit 103 evaluates the route from each of the activation position / posture candidate positions / postures generated by the route generation unit 204 to the completion of the task so that the shorter the travel route or the shorter the travel time, the higher the priority.

[0094] <Embodiment 4> In the first embodiment, the start-up position and orientation are determined based on the distribution of image features detected from camera images. In contrast, in the fourth embodiment, when the self-position and orientation of a moving object are estimated using shape information measured by a range sensor as input, the start-up position and orientation are determined based on the distribution of objects in the working space.

[0095] Fig. 7 is a diagram showing an example of an occupancy grid map created by LiDAR in embodiment 4. That is, Fig. 7 shows an occupancy grid map created from position data of a point cloud on the surface of a surrounding object obtained by measuring the working area using, for example, 2D-LiDAR (Light Detection and Ranging).

[0096] Here, the cells in the grid of the occupancy grid map 300 that are filled in gray are cells that have been determined to contain an object or to be unknown areas by map creation using the 2D-LiDAR sensor and the SLAM algorithm.

[0097] On the other hand, white cells (e.g., 303) are areas where no objects exist, and in this embodiment, the entire area indicated by these white cells or a part of it is assumed to be the cleaning area 301 designated by the mobile operator. Also, points where the curvature of the curve connecting the point cloud scanned by the 2D-LiDAR exceeds a threshold and can be detected as corners are corner points 302a to 302k.

[0098] Here, when describing the corner points 302a to 302k without identifying which point is being referred to, the point will simply be referred to as corner point 302.

[0099] Fig. 8 is a flowchart illustrating in detail a processing example of determining the start-up position and attitude (step S203) in embodiment 4. Note that the CPU or the like serving as a computer in the information processing device 100 executes a computer program stored in a memory, thereby sequentially performing the operations of the steps in the flowchart in Fig. 8.

[0100] In step S401, the start-up position and attitude determination unit 103 detects corner points from the occupancy grid map. Here, since the 2D-LiDAR in this embodiment scans an area 180 degrees forward, for example, from cell 303 in Fig. 7, four corner points, 302a to 302d, can be detected in a position and attitude in the upward direction in Fig. 7. On the other hand, ten corner points, 302b to 302k, can be detected in a position and attitude in the right direction in Fig. 7 from cell 303.

[0101] In step S402, the start-up position and attitude determination unit 103 evaluates the position and attitude based on the number of corner points detected. That is, the start-up position and attitude determination unit 103 evaluates the position and attitude in which the number of corner points detected in step S401 exceeds, for example, five, as being suitable for self-location estimation.

[0102] In step S403, the activation position / attitude determination unit 103 excludes the position / attitude that is not evaluated as a position / attitude suitable for self-location estimation in step S402 from the activation position / attitude candidates as an unsatisfied position / attitude.

[0103] Next, in step S404, the activation position and attitude determination unit 103 evaluates whether or not the information acquired by the 2D-LiDAR for each of the activation position and attitude candidates is suitable for the activation position and attitude, based on the priority information and the detected corner points.

[0104] That is, in step S404, the activation position / posture determination unit 103 calculates the priority for each of the activation position / posture candidates based on the calculation formula for the evaluation value p described in the first embodiment, and determines the position / posture with the highest priority (lowest evaluation value p) as the activation position / posture.

[0105] As described above, according to the fourth embodiment, it is also possible to determine a starting position and attitude that satisfies both work or traffic restrictions and the success of global self-localization at the start of self-localization, such as when starting work.

[0106] <Variation 4-1> In the fourth embodiment, a method using corner points on a two-dimensional plane obtained by a 2D-LiDAR that measures two-dimensional distances has been described. However, the present invention is not limited to this, and three-dimensional corner points may be calculated from a three-dimensional point cloud obtained by a 3D-LiDAR that measures three-dimensional distances.

[0107] The starting position and orientation may be determined by evaluating whether the position and orientation are suitable for global self-localization based on the three-dimensional corner points. Note that the sensor for detecting the corner points is not limited to LiDAR, and any sensor capable of measuring shape features may be used.

[0108] <Variation 4-2> In step S402, a position and orientation with more than five corner points is evaluated as a position and orientation suitable for self-position estimation, but this value is merely an example and is not limited to this value and any value may be used.

[0109] <Embodiment 5> In the first embodiment, an example has been described in which a starting position and attitude is determined that satisfies both work or traffic constraints and the success of global self-localization at the start of self-localization when starting work, etc. In the fifth embodiment, an example will be described in which the operator of the mobile body can select the starting position and attitude.

[0110] FIG. 9 is a diagram showing an example of an operation screen in the fifth embodiment, and shows an example of an operation screen of the mobile object control terminal 400 used when the mobile object operator controls the mobile object 200 immediately after the processing of step S302.

[0111] The mobile object control terminal 400 has a display unit 401 that can display the travel route generated by the information processing device 100 and the operating status of the mobile object 200. The display unit 401 has a touch panel input function, and can display information and also allow a user such as a mobile object operator to input information.

[0112] The contents displayed on the display unit 401 include a mobile body travel area map 410, a setting menu 420, an execute button 431, a start position change button 432, and a cancel button 433. The travel area map 410 displays the SLAM map No. 3, for example, that the mobile body operator selected in the setting menu 420.

[0113] The travel area map 410 displays a moving body marker 412 indicating the start position and attitude determined by the information processing device 100, and a travel route 411 according to the work content set by the moving body operator in the setting menu 420. 413 to 415 indicate non-travelable areas, which are areas outside the map range or areas with pillars or the like, and indicate areas where the moving body 200 cannot travel.

[0114] Here, when the mobile body operator presses the execute button 431, the mobile body starts estimating its own position. When the start-up position change button 432 is pressed, the start-up position and attitude with the next highest start-up position and attitude evaluation value is redrawn as the mobile body marker 412 in the travel area map 410. When the cancel button 433 is pressed, the information processing device 100 interrupts the start-up position and attitude determination process.

[0115] As described above, according to the fifth embodiment, the operator of the mobile body can select the activation position and attitude while referring to the mobile body control terminal 400.

[0116] <Variation 5-1> In the above example, when the activation position change button 432 is selected, the activation position and orientation with the next highest activation position and orientation evaluation value is sequentially redrawn, but multiple options may also be displayed simultaneously.

[0117] That is, multiple startup position and orientation candidates may be displayed sequentially or simultaneously, and in this case, the startup position and orientation determination unit 103 functions as a display control means for displaying the multiple startup position and orientation candidates, for example, on the display unit 401 of the mobile control terminal 400.

[0118] Furthermore, the amount of deviation to the activation position and attitude and the estimated time required to reach the travel route 411 may be displayed based on the travel route 411 corresponding to the work content set in the setting menu 420. This allows the mobile body operator to select a desired activation position and attitude from multiple candidates.

[0119] <Variation 5-2> Although the mobile object control terminal 400 has been described as an example of a tablet terminal equipped with a touch panel type liquid crystal display, the present invention is not limited to this as long as it allows the mobile object operator to check and input information. For example, the mobile object control terminal 400 may be an LCD display mounted on the mobile object 200, or input may be made using an input terminal such as a keyboard or mouse.

[0120] In the above example, the display unit 401 of the mobile object control terminal 400 displays a travel area map 410, a setting menu 420, an execute button 431, a start position change button 432, a cancel button 433, etc. However, the display is not limited to these, and may display information necessary or useful for the mobile object operator to control the mobile object 200. For example, the remaining battery charge of the mobile object 200 or an image captured by a camera may also be displayed.

[0121] The processes of the first to fifth embodiments may be executed at a timing other than the task setting (e.g., during mapping or localizing). Although the first to fifth embodiments have been described with reference to examples in which the start-up position and posture are determined during the task setting of the floor cleaning robot, the present invention is not limited to this. For example, when a mobile operator is manually pushing the floor cleaning robot for map creation, it may be determined whether the current position is suitable as the start-up position and posture.

[0122] Furthermore, in the first to fifth embodiments, an example of an autonomous cleaning robot that cleans floors has been described, but the present invention can be applied to any mobile object that moves autonomously to perform some kind of work, such as an autonomous mobile robot for security surveillance, an autonomous mobile robot for serving food in restaurants, etc. Alternatively, the present invention can be applied to autonomously mobile drones, self-driving vehicles, etc.

[0123] In the first to fifth embodiments, if the task cannot be started from the position of the activation position and orientation calculated by the activation position and orientation determination unit 103, the path generation unit 204 may regenerate the path.

[0124] Furthermore, in the first to fifth embodiments, an example has been described in which the activation position and attitude determination unit 103 calculates an activation position and attitude for starting a predetermined task, but the activation position and attitude determination unit 103 may calculate an activation position and attitude for an autonomous moving body to start self-location estimation.

[0125] Although the present invention has been described in detail above based on the preferred embodiments, the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The present invention also includes the following combinations.

[0126] (Configuration 1) An information processing device characterized by having a work setting information acquisition means for acquiring information regarding a type of work to be performed by a mobile body, information regarding a work area, and map information for self-position estimation created in advance, and a start-up position and attitude calculation means for calculating a start-up position and attitude when starting the work based on the type of work, the work area, and the map information.

[0127] (Configuration 2) The information processing device according to Configuration 1, wherein the launch position / posture calculation means evaluates the launch position / posture candidates based on the number and distribution of feature points included in key frames corresponding to each of the plurality of launch position / posture candidates.

[0128] (Configuration 3) The information processing device according to configuration 1 or 2, characterized in that the activation position and orientation at the time of starting the work, calculated by the activation position and orientation calculation means, is notified to a predetermined terminal.

[0129] (Configuration 4) The information processing device according to any one of configurations 1 to 3, wherein the activation position / posture calculation means sets a priority for each of a plurality of activation position / posture candidates.

[0130] (Configuration 5) The information processing device according to configuration 4, wherein the startup position and orientation calculation means determines the startup position and orientation based on at least the priority.

[0131] (Configuration 6) The information processing device according to either one of configurations 4 or 5, wherein the start-up position and attitude calculation means sets the priority based on the type of the work.

[0132] (Configuration 7) The information processing device according to any one of configurations 4 to 6, wherein the startup position and orientation calculation means sets the priority based on the shape, arrangement, or movement pattern of the working area.

[0133] (Configuration 8) The information processing device according to any one of configurations 4 to 7, wherein the start-up position and attitude calculation means sets the priority based on the start-up position and attitude of the moving body.

[0134] (Configuration 9) The information processing device according to any one of configurations 1 to 8, wherein the map information is generated using SLAM.

[0135] (Configuration 10) The information processing device according to any one of configurations 1 to 9, wherein the map information includes point cloud characteristic information based on point cloud data acquired from a distance measurement sensor.

[0136] (Configuration 11) The information processing device according to any one of configurations 1 to 10, further comprising a display control means for displaying start-up position and attitude candidates.

[0137] (Method) An information processing method characterized by comprising: a work setting information acquisition step for acquiring a type of work to be performed by a mobile body, information regarding a work area, and map information for self-position estimation created in advance; and a start-up position and attitude calculation step for calculating a start-up position and attitude when starting the work based on the type of work, the work area, and the map information.

[0138] (Program) A computer program for controlling each means of the information processing device according to any one of configurations 1 to 11 by a computer.

[0139] In order to realize part or all of the control in the above-described embodiments, a computer program that realizes the functions of the above-described embodiments may be supplied to an information processing device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the information processing device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0140] 100: Information processing device 200: Mobile 400: Mobile control terminal

Claims

1. a task setting information acquisition means for acquiring information on the type of task to be performed by the mobile object, information on the task area, and map information for self-position estimation created in advance; an activation position and orientation calculation means for calculating an activation position and orientation at the start of the work based on the type of work, the work area, and the map information; An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the activation position and orientation calculation means evaluates the activation position and orientation candidates based on the number and distribution of feature points included in key frames corresponding to each of the plurality of activation position and orientation candidates.

3. 2. The information processing apparatus according to claim 1, wherein the activation position and orientation at the time of starting the work, calculated by the activation position and orientation calculation means, is notified to a predetermined terminal.

4. 2. The information processing apparatus according to claim 1, wherein the activation position and orientation calculation means sets a priority for each of a plurality of activation position and orientation candidates.

5. 5. The information processing apparatus according to claim 4, wherein the startup position and orientation calculation means determines the startup position and orientation based on at least the priority.

6. 5. The information processing apparatus according to claim 4, wherein the activation position and orientation calculation means sets the priority based on the type of the work.

7. 5. The information processing apparatus according to claim 4, wherein the startup position and orientation calculation means sets the priority based on the shape, layout, or movement pattern of the work area.

8. 5. The information processing apparatus according to claim 4, wherein the start-up position and orientation calculation means sets the priority based on the start-up position and orientation of the moving body.

9. 2. The information processing apparatus according to claim 1, wherein the map information is created using SLAM.

10. The information processing apparatus according to claim 1 , wherein the map information includes point cloud feature information based on point cloud data acquired from a distance measurement sensor.

11. 2. The information processing apparatus according to claim 1, further comprising a display control means for displaying candidates for the start-up position and attitude.

12. a task setting information acquisition step of acquiring information on the type of task to be performed by the mobile object, information on the task area, and map information for self-location estimation created in advance; an actuation position and attitude calculation step of calculating an actuation position and attitude at the time of starting the work based on the type of work, the work area, and the map information; An information processing method comprising:

13. A computer program for controlling each means of the information processing apparatus according to any one of claims 1 to 11 by a computer.

Images