Autonomous mobile body control device, autonomous mobile body control method and program

Abstract

An embodiment of the present invention is an autonomous moving body control device comprising: a region boundary distance information acquisition unit for acquiring region boundary distance information, which is information indicating the distance from an autonomous moving body subject to control to each position on the boundary of a subject region, which is a region in which the autonomous moving body is positioned; and a target information acquisition unit for acquiring target information, which is information indicating the relationship between the autonomous moving body and the subject region, on the basis of the region boundary distance information and reference region information indicating candidates for the shape, the orientation, and the position of the subject region.

Description

[Technical Field]

[0001] The present invention relates to an autonomous mobile body control device, an autonomous mobile body control method, and a program. [Background technology]

[0002] There has been active research and development into technologies for improving the accuracy of movement of autonomous mobile objects such as autonomously moving robots. For example, attempts have been made to improve the accuracy of movement using 3DLiDAR (3-dimensional Light Detection and Ranging) or a monocular camera. One such attempt involves setting a trapezoidal region of interest and detecting reflected light from the road surface within that region to detect road edges (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-118889 [Patent Document 2] Japanese Patent Publication No. 2020-154751 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technologies, it is difficult to estimate the deviation between the direction of a road at an intersection or the like and the traveling direction of the autonomous mobile body, or the relative positional relationship between a landmark at an intersection or the autonomous mobile body, and therefore there are cases where the autonomous mobile body is unable to move appropriately. Furthermore, this is not limited to intersections, but can also occur in unpredictable situations, such as when traveling in a situation where there are many pedestrians.

[0005] In view of the above circumstances, an object of the present invention is to provide a technique for improving the accuracy of movement of an autonomous moving body. [Means for solving the problem]

[0006] One aspect of the present invention is a control target autonomous moving body. based on a processing target image captured by an imaging device attached to the processing target image, parameters of the imaging device, and a result of performing area segmentation processing on the processing target image to distinguish a target area appearing in the processing target image from other areas, The autonomous mobile body control device includes: an area boundary distance information acquisition unit that acquires area boundary distance information, which is information indicating the distance to each position on the boundary of a target area, which is the area in which the autonomous mobile body is located; and a destination information acquisition unit that acquires destination information, which is information indicating the relationship between the autonomous mobile body and the target area, based on reference area information that indicates candidates for the position, orientation, and shape of the target area and the area boundary distance information.

[0007] One aspect of the present invention is the above-mentioned autonomous mobile control device, wherein the target information acquisition unit executes a process to determine conditions that minimize an error, which is the difference between a mapping graph representing the reference area information and a mapping graph representing the area boundary distance information, and acquires the target information based on the conditions of the execution result.

[0008] In one aspect of the present invention, in the autonomous mobile control device, the reference area information changes based on a parameter that represents at least a state of the target area as seen from the autonomous mobile body.

[0009] One aspect of the present invention is the above-mentioned autonomous mobile control device, wherein the reference area information includes information indicating the position of the boundary of the target area that is not photographed by an imaging device running parallel to the autonomous mobile body and facing the direction of the autonomous mobile body.

[0010] One aspect of the present invention is the above-mentioned autonomous mobile control device, wherein the target information acquisition unit acquires the target information by executing an error minimization process that determines conditions for minimizing the error, which is the difference between the reference area information and the area boundary distance information, and the error minimization process uses the reference area information using one or more reference area expression functions, which are functions that represent the position, orientation, and shape of the target area and have one or more parameters.

[0011] In one aspect of the present invention, in the autonomous mobile control device, the area boundary distance information acquisition unit acquires the area boundary distance information after deleting information about a moving obstacle, The target information acquisition unit acquires the target information by executing an error minimization process that determines a condition for minimizing an error, which is a difference between the reference area information and the area boundary distance information.

[0012] One aspect of the present invention is a control target autonomous moving body. based on a processing target image captured by an imaging device attached to the processing target image, parameters of the imaging device, and a result of performing area segmentation processing on the processing target image to distinguish a target area appearing in the processing target image from other areas, The autonomous mobile body control method includes a destination information acquisition step of acquiring destination information, which is information indicating the relationship between the autonomous mobile body and a target area, based on reference area information indicating candidates for the position, orientation, and shape of a target area, which is the area in which the autonomous mobile body is located, and area boundary distance information, which is information indicating the distance to each position on the boundary of the target area.

[0013] One aspect of the present invention is a program for causing a computer to function as the autonomous mobile object control device described above. [Effects of the Invention]

[0014] According to the present invention, it is possible to improve the accuracy of movement of an autonomous moving body. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is an explanatory diagram illustrating an overview of an autonomous mobile body control device 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a processing target image in the embodiment. [Figure 3] 10A and 10B are diagrams showing examples of results of area division processing in the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a distance image in the embodiment. [Figure 5] 4 is a diagram showing an example of a result of projecting the segmentation result shown in FIG. 3 onto a VLS plane in the embodiment. [Figure 6] FIG. 10 is a diagram showing an example of a result of an error minimization process in the embodiment. [Figure 7]FIG. 10 is a diagram showing an example of purpose information according to the embodiment. [Figure 8] FIG. 10 is a diagram showing an example of a boundary outside the field of view in the embodiment. [Figure 9] 10A and 10B are diagrams showing an example of a result of error minimization processing when an intersection is present in the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a result of segmenting an intersection in the embodiment. [Figure 11] 10A and 10B are diagrams illustrating an example of a result of projecting the segmentation result at an intersection onto an overhead view and a visual field boundary distance in an embodiment. [Figure 12] FIG. 2 is a diagram showing an example of the functional configuration of an autonomous mobile body control device 1 according to the embodiment. [Figure 13] FIG. 2 is a diagram showing an example of the functional configuration of a control unit 10 according to the embodiment. [Figure 14] FIG. 2 is a diagram showing an example of the flow of processing executed by the autonomous mobile body control device 1 of the embodiment. [Figure 15] 10 is a flowchart showing an example of the flow of processing in which an area boundary distance information acquisition unit 102 acquires area boundary distance information in the embodiment. [Figure 16] FIG. 9 is an explanatory diagram illustrating an example of the relationship between a moving body main body 905 of the autonomous moving body 9, an image capturing device 902, and a horizontal plane in the embodiment. [Figure 17] FIG. 4 is a diagram showing an example of parameters used in an equation expressing a distance in the embodiment. [Figure 18] FIG. 2 is an explanatory diagram illustrating parameters used to formulate the shape of a road whose shape is straight in the embodiment. [Figure 19] FIG. 10 is a diagram showing an example of the propagation of a signal emitted from the center of a virtual lidar when the road shape is a straight line in an embodiment. [Figure 20] FIG. 2 is an explanatory diagram illustrating parameters used to formulate the shape of a curved road in an embodiment. [Figure 21] FIG. 10 is a first diagram showing an example of auxiliary points used for formulating a second distance equation in the embodiment. [Figure 22]FIG. 10 is a second diagram showing an example of auxiliary points used for formulating a second distance equation in the embodiment. [Figure 23] FIG. 10 is a diagram showing an example of the shape of a T-shaped road in the embodiment. [Figure 24] FIG. 10 is a diagram showing an example of auxiliary points used for formulating a third distance equation in the embodiment. [Figure 25] FIG. 10 is a first diagram showing an example of an overhead image and a distance from the center of a virtual lidar to a boundary on a VLS plane in an embodiment. [Figure 26] FIG. 10 is a first diagram showing an example of the results of estimation using equations for a straight line, a right curve, and a left curve in the embodiment. [Figure 27] FIG. 2 is a second diagram showing an example of an overhead image and a distance from the center of the virtual lidar to the boundary on the VLS plane in the embodiment. [Figure 28] FIG. 2 is a second diagram showing an example of the results of estimation using equations for a straight line, a right curve, and a left curve in the embodiment. [Figure 29] FIG. 10 is a first diagram showing an example of an overhead image and a distance from the center of a virtual lidar to a boundary on a VLS plane in an embodiment. [Figure 30] 10A to 10C are diagrams showing examples of results of estimation using equations for a straight line, a right-hand T-junction, a left-hand T-junction, and a T-junction in the embodiment. [Figure 31] 10A and 10B are diagrams showing an example of an overhead image and a distance from the center of a virtual lidar to a boundary on a VLS plane in an embodiment. [Figure 32] 10A to 10C are diagrams showing examples of results of estimation using equations for a straight line, a right-hand T-junction, a left-hand T-junction, and a T-junction in the embodiment. [Figure 33] 10A and 10B are diagrams showing an example of an overhead image and a distance from the center of a virtual lidar to a boundary on a VLS plane in an embodiment. [Figure 34] 10A to 10C are diagrams showing examples of results of estimation using equations for a straight line, a right-hand T-junction, a left-hand T-junction, and a T-junction in the embodiment. [Figure 35] FIG. 10 is a diagram showing an example of the results of error minimization processing performed by the autonomous mobile body control device 1 in a modified example, in which a moving obstacle is ignored when part of the subject is a moving obstacle. [Figure 36] 10 is a flowchart showing an example of the flow of processing executed by the autonomous mobile body control device 1 when part of a subject is a moving obstacle in a modified example. [Figure 37] 10 is a flowchart showing an example of a flow of generating a segmentation model in a modified example. [Figure 38] FIG. 10 is a diagram showing the experimental environment of the first experiment conducted outdoors in a modified example. [Figure 39] FIG. 10 is a first diagram showing the results of a first experiment conducted outdoors in the modified example. [Figure 40] FIG. 2 is a second diagram showing the results of the first experiment conducted outdoors in the modified example. [Figure 41] FIG. 3 is a third diagram showing the results of the first experiment conducted outdoors in the modified example. [Figure 42] FIG. 10 is a diagram showing the experimental environment of the first experiment conducted outdoors in a second modified example. [Figure 43] FIG. 10 is a first diagram showing the results of a first experiment conducted outdoors in a second modified example. [Figure 44] 2 is a second diagram showing the results of the first experiment conducted outdoors in the modified example. FIG. [Figure 45] The third figure showing the results of the first experiment conducted outdoors in the second modified example. [Figure 46] FIG. 10 is a diagram showing the experimental environment of the second experiment conducted in the first indoor location in the modified example. [Figure 47] FIG. 10 is a first diagram showing the results of a second experiment conducted indoors in the modified example. [Figure 48] 2 is a second diagram showing the results of a second experiment conducted indoors in the first modified example. [Figure 49] FIG. 3 is a third diagram showing the results of a second experiment conducted indoors in the first modified example. [Figure 50] FIG. 10 is a diagram showing the experimental environment of a second experiment conducted indoors in a modified example. [Figure 51] FIG. 10 is a first diagram showing the results of a second experiment conducted indoors in a modified example. [Figure 52]A second diagram showing the results of a second experiment conducted indoors in a modified example. [Figure 53] FIG. 3 is a third diagram showing the results of a second experiment conducted indoors in a modified example. [Figure 54] FIG. 10 is a diagram showing an experimental environment for a third experiment in a modified example. [Figure 55] FIG. 10 is a first diagram showing the results of a third experiment in the modified example. [Figure 56] FIG. 2 is a second diagram showing the results of the third experiment in the modified example. [Figure 57] FIG. 1 is a first diagram showing the accuracy of the measurement results of the slope and road width obtained based on the experimental results of the first to third experiments. [Figure 58] 2 is a second diagram showing the accuracy of the measurement results of the slope and road width obtained based on the experimental results of the first to third experiments. [Figure 59] A diagram showing the experimental environment for the control experiment. [Figure 60] FIG. 10 shows experimental results of a control experiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] (Embodiment) FIG. 1 is an explanatory diagram illustrating an overview of an autonomous mobile body control device 1 according to an embodiment. The autonomous mobile body control device 1 controls the movement of an autonomous mobile body 9 that is a control target. The autonomous mobile body 9 is an autonomously moving body such as an autonomously moving robot or automobile. Specifically, the autonomous mobile body control device 1 acquires area boundary distance information for each position of the autonomous mobile body 9, and acquires information indicating the relationship between the autonomous mobile body 9 and the target area based on the reference area information and the acquired area boundary distance information (hereinafter referred to as "destination information").

[0017] The target area means an area in space where the autonomous mobile body 9 is located. The area means an area in space. The area is, for example, a road along which the autonomous mobile body 9 can travel. In FIG. 1, a road 900 is an example of the target area. In FIG. 1, an arrow 901 indicates the direction in which the autonomous mobile body 9 will travel. In FIG. 1, the image capturing device 902 is an image capturing device that runs parallel to the autonomous mobile body 9 and faces in the same direction as the autonomous mobile body 9. The image capturing device 902 is, for example, a 3DLiDAR (3 dimensional light detection and ranging).

[0018] The image capturing device 902 may be a monocular camera. The image capturing device 902 may be provided by the autonomous mobile body 9, or may be provided by another mobile body such as a drone that travels parallel to the autonomous mobile body 9. FIG. 1 shows, as an example, a case where the image capturing device 902 is provided by the autonomous mobile body 9. The image capturing device 902 travels parallel to the autonomous mobile body 9 and faces the direction of the autonomous mobile body 9, so the direction seen from the autonomous mobile body 9 is the direction seen from the image capturing device 902. Furthermore, the image capturing device 902 travels parallel to the autonomous mobile body 9 and is at the same position as or a certain distance from the autonomous mobile body 9, so the distance to an object seen from the autonomous mobile body 9 is the distance to the object seen from the image capturing device 902.

[0019] The destination information indicates, for example, where the autonomous moving body 9 is located within the region width of the target region. The destination information indicates, for example, the relationship between the direction of the target region and the orientation of the autonomous moving body 9. The destination information indicates, for example, whether the target region is an intersection at the position of the autonomous moving body 9. If the target region is an intersection at the position of the autonomous moving body 9, for example, the destination information indicates the direction of each region that intersects at the intersection.

[0020] The area boundary distance information is information indicating the distance from the autonomous moving body 9 to each position on the boundary of the target area (hereinafter referred to as "area boundary distance"). The area boundary distance information is information indicating, for example, the distance from the autonomous moving body 9 to a subject located on the boundary of a road in each line of sight direction (hereinafter referred to as "line of sight direction") as the area boundary distance, for each direction seen from the autonomous moving body 9 with the autonomous moving body 9 at the center. The area boundary distance information is information displayed, for example, as a graph indicating the area boundary distance for each line of sight direction. Note that the subject is, for example, an obstruction.

[0021] For example, if the image capturing device 902 is a 3DLiDAR (3 dimensional light detection and ranging), the area boundary distance information is the measurement result by the 3DLiDAR. That is, if the image capturing device 902 is a 3DLiDAR, the area boundary distance is the distance of the signal. If the image capturing device 902 is a 3DLiDAR, the line of sight direction is the direction as seen from the 3DLiDAR.

[0022] The area boundary distance information may be obtained by calculation using a previously obtained distance image and the results of previously learned machine learning, based on an image captured by a monocular camera attached to the autonomous mobile body 9, for example. The distance image is, for example, the result of capturing an image of a horizontal plane. The distance image may be not only data actually observed, but also a result calculated based on the internal parameters of the camera.

[0023] If the image capturing device 902 is a monocular camera, the area boundary distance information is based on the image capturing results by the image capturing device 902, and is the result obtained by calculation, for example, using a distance image previously obtained by the autonomous mobile control device 1 and the results of previously learned machine learning.

[0024] Specifically, the result of pre-learned machine learning is a process of determining pixels indicating an area in which the autonomous moving body 9 can move from the image capture results of a monocular camera, such as semantic segmentation. When the image capture device 902 is a monocular camera, the line of sight direction is the direction seen from the monocular camera.

[0025] The reference area information is information that represents candidates for the position, orientation, and shape of the target area. Specifically, the reference area information is a function that represents the position, orientation, and shape of the target area and has one or more parameters (hereinafter referred to as the "reference area representation function"). In other words, the reference area information is specifically a mathematical model that represents candidates for the position, orientation, and shape of the target area. The parameters determine the form of the function that represents the shape of the target area, and are, for example, parameters related to the shape of the target area, such as the width of the target area. The reference area representation function is, for example, a function that indicates the correspondence between the area boundary distance and the gaze angle and includes one or more parameters. The gaze angle is an angle that indicates each gaze direction within a specified plane.

[0026] The parameters include at least parameters that represent the state of the target area as seen from the autonomous mobile body 9. When the parameters are used when the autonomous mobile body control device 1 acquires destination information, the autonomous mobile body control device 1 changes the reference area expression function based on at least the parameters that represent the state of the target area as seen from the autonomous mobile body 9. In other words, the reference area information changes based on at least the parameters that represent the state of the target area as seen from the autonomous mobile body 9. The state of the target area as seen from the autonomous mobile body 9 is, for example, the inclination, width, or length of the target area as seen from the autonomous mobile body 9.

[0027] It should be noted that the reference area representation function does not need to represent only one target area without branches, but may represent an area with branches. For example, if the area is a road, the reference area representation function may represent a single road, or may represent a branched road including the branches. For simplicity of explanation, the autonomous mobile object control device 1 will be described below using an example in which the reference area representation function represents a single road. It should be noted that in FIG. 1, image 903 is a figure that represents an example of a shape represented by reference area information on a bird's-eye view. It should be noted that in the figure shown in image 903, a notch corresponding to the field of view of the image capture device 902 exists near the lower vertex of the parallelogram.

[0028] The process (hereinafter referred to as "extraction process") in which the autonomous mobile body control device 1 acquires target information based on reference area information and area boundary distance information preferably includes an error minimization process and a target information acquisition process. In the extraction process, the target information acquisition process is executed after the error minimization process is executed.

[0029] The error minimization process is an optimization process that determines the parameter value that minimizes the difference (hereinafter referred to as "error") between the mapping graph representing the reference region information and the mapping graph representing the acquired region boundary distance information. In other words, the error minimization process is a process that determines the conditions for minimizing the error. Hereinafter, the parameter value determined by the error minimization process that minimizes the error will be referred to as the determined value. Hereinafter, the reference region representation function specified by the determined value will be referred to as the determined function. Note that the mapping graph may be defined as, for example, a set consisting of an ordered pair (a, b) of the set A × B such that b = f(a) is true when a mapping f:A → B is given. For example, the mapping graph in this specification may use the definition of a so-called general mapping graph.

[0030] The destination information acquisition process is a process for acquiring destination information based on a determined function, such as acquiring two peak positions indicated by the determined function as road edges in the target area.

[0031] In this way, the extraction process is a process for acquiring target information by performing an optimization process using area boundary distance information and reference area information. Note that image 904 in FIG. 1 is a diagram showing an example of the result of the error minimization process. Details of image 904 will be explained using FIG. 6 after explaining the Virtual Lidar process (hereinafter referred to as "VLS process") and one specific example of the error minimization process.

[0032] (Details of VLS processing) Details of the VLS processing will be described using an example in which the image capturing device 902 is a monocular camera attached to the autonomous mobile body 9. The VLS processing is an example of a technology for obtaining area boundary distance information by performing calculations using a previously obtained distance image and the results of previously learned machine learning based on an image captured by the image capturing device 902 (hereinafter referred to as the "image to be processed"). The VLS processing is a technology used by the autonomous mobile body control device 1. The VLS processing includes area segmentation processing, distance association processing, boundary pixel information acquisition processing, and virtual space distance measurement processing. In the VLS processing, the area segmentation processing, distance association processing, and boundary surface pixel information acquisition processing are performed before the virtual space distance determination processing, and the area segmentation processing is performed before the boundary surface pixel information acquisition processing. Therefore, in the VLS processing, for example, the boundary pixel information acquisition processing is performed after the area segmentation processing and distance association processing, and then the virtual space distance measurement processing is performed. The area segmentation processing and the distance association processing may be performed in any order, or they may be performed simultaneously. The boundary association process and the boundary pixel information acquisition process may be performed in any order, or may be performed simultaneously.

[0033] The region segmentation process is a process in which the autonomous mobile body control device 1 determines what kind of region is depicted in each pixel of the image to be processed, using the results of pre-trained machine learning such as segmentation that classifies each region in the image. By performing the region segmentation process, the autonomous mobile body control device 1 obtains information (hereinafter referred to as "distinction information") that distinguishes each region depicted in the image to be processed from other regions. For example, by performing the region segmentation process by the autonomous mobile body control device 1, information that distinguishes the target region depicted in the image to be processed from other regions is obtained.

[0034] Fig. 2 is a diagram showing an example of an image to be processed in an embodiment. The image in Fig. 2 shows a road running from the lower right to the upper left of the image as a target area. In the image in Fig. 2, one of the road edges of the road, which is the target area, is a boundary with a lawn.

[0035] Fig. 3 is a diagram showing an example of the result of region division processing in the embodiment. More specifically, Fig. 3 is a diagram showing an example of the result of segmentation of the processing target image. In Fig. 3, the target region appearing in the processing target image is expressed separately from other regions appearing in the processing target image.

[0036] The distance correspondence process is a process in which the autonomous mobile object control device 1 acquires distance information indicated by pixels of the distance image, which are pre-associated with each pixel of the image to be processed, as information indicating the attributes of each pixel of the image to be processed to which each pixel of the distance image corresponds. Hereinafter, information indicating the attributes of each pixel of the image to be processed to which each pixel of the distance image corresponds is referred to as planar pixel distance information. Note that the distance correspondence process acquires planar pixel distance information under the assumption that all images captured in the image to be processed are on a horizontal plane. Because the distance image is an image of the scenery seen by the image capturing device 902, the distance indicated by each pixel of the distance image is information indicating the distance from the image capturing device 902 to the image captured by each pixel of the distance image. In other words, because the distance image is an image of the scenery seen by the image capturing device 902, the distance indicated by each pixel of the distance image is information indicating the distance from the autonomous mobile object 9 to the image captured by each pixel of the distance image.

[0037] Fig. 4 is a diagram showing an example of a distance image in an embodiment. The distance image in Fig. 4 is a distance image of a horizontal plane captured by an image capture device 902 whose line of sight is parallel to the horizontal plane. In the distance image in Fig. 4, the lighter the color, the shorter the distance to the image capture device 902.

[0038] The boundary pixel information acquisition process is a process in which the autonomous mobile object controlling body 1 acquires information indicating pixels that represent boundaries between regions among the pixels of the processing target image (hereinafter referred to as "boundary pixel information") based on the distinction information.

[0039] The virtual space distance measurement process is a process in which the autonomous mobile body control device 1 acquires the distance from the image capture device 902 to the image captured by each pixel indicated by the area pixel information based on boundary pixel information and planar pixel distance information. Therefore, the virtual space distance measurement process is a process in which the autonomous mobile body control device 1 acquires area boundary distance information based on boundary pixel information and planar pixel distance information.

[0040] The virtual space distance measurement process is a process in which the autonomous mobile object control device 1 acquires, for example, the distance from the origin to the boundary of an area captured in an image to be processed on the VLS plane through calculation. The VLS plane is a virtual space having a two-dimensional coordinate system centered on the position of the autonomous mobile object 9 (i.e., the position of the image capture device 902), in which the image captured in each pixel indicated by the boundary pixel information is installed at a position away from the autonomous mobile object 9 by the distance indicated by the plane pixel distance information. The origin on the VLS plane is the position in the virtual space where the autonomous mobile object 9 is located. Measurement on the VLS plane is a process in which the autonomous mobile object control device 1 transmits a LiDAR signal from the origin on the VLS plane, calculates the time it takes for the LiDAR signal to scatter or reflect back to the origin, and converts the calculated time into distance through calculation. Therefore, the result of the measurement on the VLS plane is an example of area boundary distance information.

[0041] 5 is a diagram showing an example of the result of projecting the segmentation result shown in FIG. 3 onto the VLS plane in the embodiment. The horizontal and vertical axes of the result in FIG. 5 represent the axes of the Galilean coordinate system. The position where the value of the horizontal axis and the value of the vertical axis in FIG. 5 are 0 is the position of the image capture device 902 (i.e., the position of the autonomous moving body 9). The boundary of the trapezoidal area A1 in FIG. 5 represents the boundary of the field of view angle of the image capture device 902.

[0042] (Specific examples of error minimization processing and target information acquisition processing) A specific example of the error minimization process and the target information acquisition process will be described using the case where the image capturing device 902 is a monocular camera as an example.

[0043]

number

[0044] VLSroad is the distance from the origin on the VLS plane to the boundary of the reference area information. VLSparallelogram is the distance from the origin on the VLS plane to the boundary of an approximate shape when the shape of a target area such as a road is approximated by a predetermined shape such as a parallelogram (hereinafter referred to as an "approximate shape") without considering the boundary of the field of view of the image capture device 902. More specifically, the approximate shape is a shape that approximates the shape of the target area with a predetermined shape without considering the boundary of the field of view of the image capture device 902, and is a figure that approximates the shape of the target area when it is represented on a bird's-eye view. The approximate shape is, for example, a parallelogram.

[0045] The VLSmap represents the distance from the origin on the VLS plane to the boundary of the field of view of the image capture device 902. The VLSmap is calculated based on the monocular camera's extrinsic parameters ae.param, the monocular camera's internal parameters ai.param, and the map's measurement range arange. The monocular camera's extrinsic parameters ae.param specifically refer to the position and orientation of the monocular camera. The monocular camera's internal parameters ai.param specifically refer to the focal length and image center of the monocular camera. The distance from the origin on the VLS plane to the boundary of the field of view of the image capture device 902 depends on the monocular camera's position acenter on the VLS plane.

[0046] VLSroad depends on the inclination θangle of the target area, the length θlength of the target area, the left width θl.width of the area, and the right width θr.width of the area. For ease of explanation, the internal parameter ae.param of the monocular camera will be denoted as Θ, the external parameter of the monocular camera will be denoted as A, and the line of sight angle will be denoted as χ. In such a case, VLSroad is formulated using the following equations (2) to (4).

[0047]

number

[0048]

number

[0049]

number

[0050] VLSroad formulated by equations (2) to (4) is an example of reference region information. In the error minimization process, a process is performed to estimate the values ​​of the parameters included in VLSroad. For example, the estimation is performed by optimizing the least squares method to minimize the error. In such a case, the result of optimization by the least squares method is the determined function.

[0051] An example of a formula for calculating the square error is the following formula (5), and an example of a formula for calculating the parameters is the following formula (6).

[0052]

number

[0053]

number

[0054] xi represents the line of sight angle on the VLS plane, and yi represents the distance from the origin on the VLS plane at line of sight angle xi.

[0055] FIG. 6 shows an example of the results of error minimization processing performed using equations (1) to (6). FIG. 6 is a diagram showing an example of the results of error minimization processing in an embodiment. The horizontal axis of FIG. 6 represents gaze angle. The vertical axis of FIG. 6 represents distance. The units are distance units. FIG. 6 shows an example of area boundary distance information and an example of the results of the error minimization processing. FIG. 6 shows that the results of the error minimization processing match with a high degree of accuracy with the graph indicated by the area boundary distance information. The results of FIG. 6 show that there are peaks at gaze angles of 140° and 170°. Gaze angles of 140° and 170° respectively indicate the edges of areas such as road shoulders. Therefore, the gaze angle indicating the center of the two peaks is the angle indicating the center of the target area.

[0056] Generally, areas such as roads have both ends, so the results of the error minimization process show two peaks for each area, as shown in the results of Figure 6. Therefore, in the target information acquisition process, the gaze angle indicating the center of the two peaks is acquired from the results of the error minimization process as the gaze angle indicating the center of the target area. In this way, information indicating the direction of the target area is acquired by executing the target information acquisition process.

[0057] FIG. 7 is a diagram showing an example of destination information in an embodiment. The vertical axis of FIG. 7 represents the distance from the autonomous moving body 9. The units are distance units. The horizontal axis of FIG. 7 represents each line of sight direction in a horizontal plane. More specifically, the horizontal axis of FIG. 7 represents the angle indicating each line of sight direction in the horizontal plane (i.e., the line of sight angle in the horizontal plane). Therefore, when the image capturing device 902 is a 3DLiDAR, the horizontal axis of FIG. 7 represents the measurement angle. In FIG. 7, the traveling direction of the autonomous moving body 9 is a direction of 180°.

[0058] In FIG. 7, the angle 200° on the horizontal axis represents the direction in the horizontal plane at an angle of 20° from the traveling direction of the autonomous mobile body 9. FIG. 7 shows the road boundary and the boundary of the field of view of the camera device 902 under ideal conditions. More specifically, in FIG. 7, the "distance to the field of view boundary" is information indicating the boundary of the field of view, and indicates the distance from the camera device 902 to the boundary of the field of view of the camera device 902 (hereinafter referred to as the "field of view boundary distance"). Note that the "ideal condition" refers to the condition in which FIG. 7 is the result of converting the information on each boundary (specifically, the "distance to the field of view boundary," "shape represented by the reference area representation function," and "outside field of view boundary" in FIG. 8) described below into a representation on the VLS plane. In other words, FIG. 7 is a diagram showing an example of target information in an embodiment, and corresponds to the plan view shown in FIG. 8.

[0059] In Figure 7, "without considering the boundary of the visual field" is an example of the result of displaying the boundary of an approximate shape such as a parallelogram on a graph with the horizontal axis being the gaze angle and the vertical axis being the distance. In Figure 7, "Example of target information" is an example of the result of the target information acquisition process, and is an example of information indicating the direction of the target area. Figure 7 is also a diagram showing the center of the target area within the gaze angle range of 120° to 260°.

[0060] In Figure 7, "taking into account the boundary of the visual field" is an example of the result of displaying a reference area representation function calculated using an approximate shape such as a parallelogram and information about the boundary of the visual field on a graph with the horizontal axis being the gaze angle and the vertical axis being the distance. In other words, "taking into account the boundary of the visual field" is an example of the result of coordinate conversion of the reference area representation function from an expression in the Cartesian coordinate system of the VLS plane to an expression in a polar coordinate system with the gaze angle and distance as the coordinate axes. Specifically, using information about the boundary of the visual field means using information indicating the boundary of the target area indicated by the approximate shape that is not captured by the imaging device 902 (hereinafter referred to as "outside the field of view boundary").

[0061] Therefore, in Figure 7, "considering the boundary of the field of view" indicates the result of error optimization processing using, as the reference region representation function, a function indicating the boundary of a shape in which a notch corresponding to the field of view of the image capture device 902 exists near the lower vertex of an approximate shape such as a parallelogram. A notch is an example of a boundary outside the field of view. A shape in which a notch corresponding to the field of view of the image capture device 902 exists near the lower vertex of a parallelogram is, for example, the shape shown in image 903.

[0062] FIG. 8 is a diagram showing an example of an out-of-field boundary in an embodiment. In FIG. 8, the position where the horizontal axis value is 0 and the vertical axis value is 0 is the position of the image capture device 902 on the VLS plane. FIG. 8 shows an example of the boundary of the field of view of the image capture device 902 on the VLS plane. FIG. 8 shows an example of a shape represented by a reference region representation function on the VLS plane. FIG. 8 shows a parallelogram with a cutout as an example of a shape represented by a reference region representation function. The boundary represented by a dashed line in FIG. 8 is an example of an out-of-field boundary. Note that FIG. 8 shows the VLS plane. The horizontal axis of FIG. 8 represents the distance from the position of the image capture device 902 on the VLS plane. The vertical axis of FIG. 8 represents the distance from the position of the image capture device 902 on the VLS plane, that is, the distance in a direction perpendicular to the horizontal axis of FIG. 8.

[0063] Although a parallelogram is mentioned as an example of an approximate shape, approximate shapes are not limited to parallelograms. In other words, the shape represented by the reference area representation function is not limited to a parallelogram or a parallelogram with a notch. Other specific examples of shapes represented by the reference area representation function will be described later for ease of understanding.

[0064] In the explanation of the extraction process so far, the case where the area is a single road with no branches has been explained as an example. However, even if the road branches, information indicating the direction of the road can be obtained by the extraction process. In other words, the extraction process described so far can also be applied to cases where the road branches, and information indicating the direction of each branching road can be obtained by the extraction process.

[0065] The process of obtaining the number of branches by the extraction process is, for example, the following process. First, an error minimization process using one function expressed using K reference region representation functions (K is an integer equal to or greater than 1) is performed for each K. The value of K with the smallest error among the results of the error minimization process is obtained as the number of reference regions (i.e., the number of branches). M in equation (7) indicates the number of parameters used for estimation, and is the product of the number of reference regions (K) and the number of parameters of the reference region representation function. The process of obtaining the value of K is an example of a target information acquisition process. If the result obtained by the extraction process is that K is 2 or greater, the position of the autonomous moving body 9 is the position of an intersection. In this way, information indicating whether the position of the autonomous moving body 9 is an intersection is obtained by the extraction process.

[0066] A specific example of the error in the process of obtaining the number of branches by extraction processing is the Bayesian Information Criterion (BIC) expressed by the following equation (7).

[0067]

number

[0068] In equation (7), L represents likelihood, and N represents the number of samples observed for the region boundary distance. Therefore, N is, for example, the number of points indicating the region boundary distance information in Figure 6. The likelihood is, for example, SSE, which is the sum of squared errors between the region boundary distance and the optimization result.

[0069] In addition, when a function expressed using multiple reference region representation functions (hereinafter referred to as a "composite function") is used in the extraction process, each of the reference region representation functions contained in the composite function does not necessarily have to be the same reference region representation function, and at least one may be different from the other reference region representation functions.

[0070] FIG. 9 is a diagram showing an example of the results of error minimization processing when an intersection is present in an embodiment. The horizontal axis of FIG. 9 represents the line of sight angle. The vertical axis of FIG. 9 represents distance, in units of distance. FIG. 9 shows the results of error minimization processing executed under the condition K is 1 and the results of error minimization processing executed under the condition K is 2. FIG. 9 also shows the results of actually surveying the edges of the road as data points of the survey results. These results in FIG. 9 show that the results of error minimization processing under the condition K=2 are closer to the actual situation than the results of error minimization processing under the condition K=1.

[0071] Although the segmentation results in Figure 3 are for a road without an intersection, segmentation can also be applied to roads with intersections. 10 is a diagram showing an example of the result of segmenting an intersection in an embodiment, showing that the intersection is split into two roads.

[0072] Fig. 11 is a diagram showing an example of the results of projecting the segmentation results at an intersection onto a bird's-eye view and the field of view boundary distance in an embodiment. The upper diagram of Fig. 11 shows an example of the results of expressing the segmentation results shown in Fig. 9 on a bird's-eye view. The lower diagram of Fig. 11 shows area boundary distance information obtained from the upper diagram of Fig. 9.

[0073] 12 is a diagram illustrating an example of the functional configuration of an autonomous mobile body control device 1 according to an embodiment. The autonomous mobile body control device 1 includes a control unit 10 including a processor 91, such as a CPU (Central Processing Unit), and a memory 92 connected by a bus, and executes a program. By executing the program, the autonomous mobile body control device 1 functions as a device including the control unit 10, an input unit 11, a communication unit 12, a storage unit 13, and an output unit 14. More specifically, the processor 91 reads a program stored in the storage unit 13 and stores the read program in the memory 92. By the processor 91 executing the program stored in the memory 92, the autonomous mobile body control device 1 functions as a device including the control unit 10, the input unit 11, the communication unit 12, the storage unit 13, and the output unit 14.

[0074] The control unit 10 executes, for example, extraction processing. The control unit 10 controls, for example, the operation of various functional units provided in the autonomous mobile body control device 1 and the operation of the autonomous mobile body 9. The control unit 10 controls, for example, the operation of the communication unit 12 and acquires a processing target image via the communication unit 12. The control unit 10 acquires, for example, area boundary distance information based on the acquired processing target image. If the imaging device 902 is a device capable of acquiring area boundary distance information such as 3DLiDAR, the control unit 10 may acquire area boundary distance information instead of the processing target image.

[0075] The control unit 10 executes, for example, an extraction process. The control unit 10 controls the operation of the autonomous moving body 9, for example, via the communication unit 12. The control unit 10 may acquire information indicating the position and orientation of the autonomous moving body 9 (hereinafter referred to as "progress state information"), for example, via the communication unit 12. The control unit 10 may estimate the position and orientation of the autonomous moving body 9, for example, based on a history of control of the operation of the autonomous moving body 9.

[0076] The input unit 11 includes input devices such as a mouse, keyboard, and touch panel. The input unit 11 may be configured as an interface that connects these input devices to the device itself. The input unit 11 accepts input of various information to the device itself.

[0077] The communication unit 12 includes a communication interface for connecting the device itself to an external device. The communication unit 12 communicates with the autonomous moving body 9 via a wired or wireless connection. The communication unit 12 receives, for example, information about the progress of the autonomous moving body 9 through communication with the autonomous moving body 9. The communication unit 12 transmits, to the autonomous moving body 9, a control signal for controlling the autonomous moving body 9 through communication with the autonomous moving body 9.

[0078] The communication unit 12 communicates with the sender of the processing target image via wired or wireless communication. The communication unit 12 acquires the processing target image by communicating with the sender of the processing target image. The sender of the processing target image may be the autonomous moving body 9 itself, or may be another device such as a drone that moves together with the autonomous moving body 9.

[0079] The storage unit 13 is configured using a non-transitory computer-readable storage medium device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 13 stores various information related to the autonomous mobile body control device 1. The storage unit 13 stores, for example, the history of control of the autonomous mobile body 9 by the control unit 10. The storage unit 13 stores, for example, the history of progress status information. The storage unit 13 stores reference area information in advance. The storage unit 13 stores a distance image in advance.

[0080] The output unit 14 outputs various types of information. The output unit 14 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The output unit 14 may be configured as an interface that connects these display devices to the output unit 14 itself. The output unit 14 outputs information input to the input unit 11 or the communication unit 12, for example. The output unit 14 outputs the execution result of the extraction process by the control unit 10, for example.

[0081] 13 is a diagram showing an example of the functional configuration of the control unit 10 in the embodiment. The control unit 10 includes a progress status information acquisition unit 101, a region boundary distance information acquisition unit 102, a reference region information acquisition unit 103, a target information acquisition unit 104, and a control signal generation unit 105.

[0082] The progress status information acquisition unit 101 acquires progress status information of the autonomous moving body 9. The progress status information acquisition unit 101 may acquire the progress status information by calculating it from the history of control of the operation of the autonomous moving body 9, or may acquire it from the autonomous moving body 9 via the communication unit 12.

[0083] The area boundary distance information acquisition unit 102 acquires area boundary distance information. When the image capturing device 902 is a device capable of acquiring area boundary distance information, such as a 3DLiDAR, the area boundary distance information is acquired from a sender of the area boundary distance information, such as the image capturing device 902, via the communication unit 12. When the image capturing device 902 is a device that acquires an image to be processed, such as a monocular camera, the area boundary distance information acquisition unit 102 acquires the image to be processed via the communication unit 12 and performs VLS processing on the acquired image to be processed to acquire the area boundary distance information.

[0084] The reference region information acquisition unit 103 acquires reference region information stored in the storage unit 13. More specifically, the reference region information acquisition unit 103 reads out one or more reference region expression functions stored in the storage unit 13. The target information acquisition unit 104 executes an extraction process to acquire target information.

[0085] The control signal generation unit 105 generates a control signal that controls the operation of the autonomous moving body 9 based on the destination information. The control signal generation unit 105 transmits the generated control signal to the autonomous moving body 9 via the communication unit 12.

[0086] 14 is a diagram showing an example of the flow of processing executed by the autonomous mobile body controlling device 1 of the embodiment. The processing of FIG. 14 is repeatedly executed at a predetermined timing.

[0087] The progress status information acquisition unit 101 acquires progress status information (step S101). Next, the area boundary distance information acquisition unit 102 acquires area boundary distance information (step S102). The area boundary distance information acquisition unit 102 may acquire the area boundary distance information from a sender of the area boundary distance information via the communication unit 12, or may acquire the processing target image via the communication unit 12 and acquire the area boundary distance information by executing VLS processing on the acquired processing target image.

[0088] Next, the reference area information acquisition unit 103 acquires the reference area information stored in the storage unit 13. More specifically, the reference area information acquisition unit 103 reads out one or more reference area expression functions stored in the storage unit 13 (step S103). Next, the destination information acquisition unit 104 executes extraction processing using the reference area information and the area boundary distance information to acquire destination information (step S104). Next, the control signal generation unit 105 generates a control signal for controlling the operation of the autonomous moving body 9 based on the destination information, and controls the operation of the autonomous moving body 9 using the generated control signal (step S105).

[0089] The process of step S101 may be executed at any timing before the process of step S105 is executed. Furthermore, steps S102 and S103 do not necessarily have to be executed in this order, and may be executed in any order as long as they are executed before the execution of step S104.

[0090] In this way, the autonomous mobile body control device 1 executes a step of acquiring area boundary distance information. The autonomous mobile body control device 1 also executes a step of acquiring destination information, which is information indicating the relationship between the autonomous mobile body 9 and the target area, based on the reference area information and the area boundary distance information.

[0091] 15 is a flowchart showing an example of the processing flow in which the area boundary distance information acquisition unit 102 acquires area boundary distance information in an embodiment. More specifically, FIG. 15 is a flowchart showing an example of the processing flow in which the area boundary distance information acquisition unit 102 acquires area boundary distance information when the image capturing device 902 is a monocular camera.

[0092] The region boundary distance information acquisition unit 102 acquires a processing target image via the communication unit 12 (step S201). Next, the region boundary distance information acquisition unit 102 executes region segmentation processing (step S202). Next, the region boundary distance information acquisition unit 102 executes boundary pixel information acquisition processing (step S203). Next, the region boundary distance information acquisition unit 102 executes distance association processing (step S204). Next, the region boundary distance information acquisition unit 102 executes distance measurement processing within virtual space (step S205). Note that the processing of steps S203 and S204 may be executed at any timing as long as they are executed after execution of step S202 and before execution of step S205. Therefore, for example, step S204 may be executed after step S202, and then step S203 may be executed after that. Furthermore, for example, steps S203 and S204 may be executed at the same timing.

[0093] In addition, when the area boundary distance information acquisition unit 102 acquires the area boundary distance information from an external device via the communication unit 12, the area boundary distance information acquisition unit 102 acquires the area boundary distance information input to the communication unit 12.

[0094] (Explanation of an example of the relationship between the imaging device 902 and the horizontal plane) Here, an example of the relationship between the image capturing device 902 and the horizontal plane will be described. FIG. 16 is an explanatory diagram illustrating an example of the relationship between the moving body main body 905, the image capturing device 902, and a horizontal plane of the autonomous moving body 9 in an embodiment. The moving body main body 905 includes wheels for moving the autonomous moving body 9, a movable unit, and a control unit for controlling the movement. The autonomous moving body 9 includes the moving body main body 905 and the image capturing device 902. The image capturing device 902 is located at the top of the moving body main body 905, at a height h from the horizontal plane on which the moving body main body 905 is located. In FIG. 16, the tilt angle refers to the angle between the vertical downward direction and the optical axis of the camera (the angle with the horizontal plane). In FIG. 16, the dashed line indicates the edge of the field of view.

[0095] (Example of reference area expression function) Below, a specific example of an equation (i.e., reference area representation function) that represents the distance from the origin to the boundary on the VLS plane will be explained using an example in which the image capture device 902 is a monocular camera. Hereinafter, the term "camera" refers to the image capture device 902. Below, a specific example of the reference area representation function will be explained using an example in which the target area is a road and the image capture device 902 is a monocular camera.

[0096] First, the parameters on the VLS plane generated based on the image to be processed are shown. These parameters are examples of parameters. Also, the parameters that appear in the following explanation of specific examples of the reference region expression function are examples of parameters.

[0097] FIG. 17 is a diagram showing an example of parameters used in an equation expressing distance in an embodiment. In FIG. 17, the y-axis represents the front direction of the camera. The x-axis is a direction perpendicular to the y-axis in the VLS plane. The shape of the boundary of the camera's field of view on the VLS plane is a trapezoid with the camera position as the base and the distance m (map_height) reflected from the image to be processed on the VLS plane as the height. The slope of the left and right sides is set by the camera's internal parameters. On this VLS plane, the center of the virtual lidar is located at a position y0 from the origin in the y direction and at a position x0 from the origin in the x direction. Note that the virtual lidar is the source of the virtual lidar signal on the VLS plane.

[0098] Below, we will explain a specific example of an equation that represents the distance from the center of the virtual lidar to the boundary on the VLS plane when the center of the virtual lidar is on the y-axis (i.e., when x0 = 0).

[0099] (First specific example) The first specific example is a formula (hereinafter referred to as the "first distance formula") that represents the distance from the center of the virtual lidar to the boundary on the VLS plane when the road shape is straight. The first distance formula when the center of the virtual lidar is located at the origin of the VLS plane is an example of a reference area expression function.

[0100] The road shape is formulated using the parameters of road inclination θangle, road length θlength, road width on the left θl.width, and road width on the right θr.width. In other words, the road shape is expressed by the above-mentioned equation (3).

[0101] 18 is an explanatory diagram illustrating parameters used to formulate the shape of a straight road in an embodiment. The length of the road is the distance from the center position (cameraposition) of the camera to the edge of the road in the y-axis direction. Parameters are set for the road width separately for the road width on the right side and the road width on the left side of the virtual lidar, and the parameters are set so that the position of the autonomous moving body 9 on the road can be estimated.

[0102] The equation representing the shape of a straight road is expressed, for example, by the following equation (8).

[0103]

number

[0104] The first distance equation is formulated using equation (8). The idea behind its derivation is as follows: imagine a scene in which a signal is emitted from the center of the virtual lidar toward the edge of the road, and then imagine the process by which the emitted signal reaches the edge of the road, thereby formulating the distance from the center of the virtual lidar to the intersection of the signal and the edge of the road.

[0105] Fig. 19 is a diagram showing an example of the propagation of a signal emitted from the center of a virtual lidar when the road shape is a straight line in an embodiment. Fig. 19 shows an example of the order in which measurements using a virtual lidar signal are performed. Specifically, Fig. 19 shows measurements being performed at equal intervals over 360 degrees clockwise, with the -y-axis direction (i.e., the negative y-axis direction) being 0 degrees. The intervals are arbitrary. Points P1, P2, P3, and P4 in Fig. 19 each represent the vertices of the approximate shape.

[0106] The set of the following equations (9) to (20) is an example of the first distance equation. Angle th1, angle th2, angle th3, and angle th4 are the angles formed by the -y axis and lines connecting points P1, P2, P3, and P4 in Fig. 19 with the center of the virtual lidar. The angles th and angle are in radians.

[0107]

number

[0108]

number

[0109]

number

[0110]

number

[0111]

number

[0112]

number

[0113]

number

[0114]

number

[0115]

number

[0116]

number

[0117]

number

[0118]

number

[0119] (Second specific example) The second specific example is a formula (hereinafter referred to as the "second distance formula") that represents the distance from the center of the virtual lidar to the boundary on the VLS plane when the road shape is curved. The second distance formula when the center of the virtual lidar is located at the origin of the VLS plane is an example of a reference area expression function.

[0120] The shape of the road is formulated using the following parameters: the slope of the straight line up to the curve θangle, the length of the road up to the entrance of the curve θD1, the distance to the left road θl.width, the distance to the right road θr.width, and the road width at the end of the curve θwidth2. The values ​​of these parameters are estimated in the extraction process. The curve is formulated as an ellipse. The horizontal width of the ellipse is formulated using the road width at the front θwidth, and the vertical width of the ellipse is formulated using the road width at the end of the curve θwidth2. In other words, the shape of the road is expressed by the following equation (21).

[0121]

number

[0122] 20 is an explanatory diagram illustrating parameters used to formulate the shape of a curved road in an embodiment. In the extraction process, if there is an angle of the model representing the curve, a process of rotating the entire shape with the center of Virtual Lidar as the origin may be performed.

[0123] Examples of equations that represent curves are the following equations (22) and (23): Equation (22) represents a right curve, and equation (23) represents a left curve.

[0124]

number

[0125]

number

[0126] These formulas are used to formulate the distance from the center of the virtual lidar to the edge of the road.

[0127] Fig. 21 is a first diagram showing an example of auxiliary points used to formulate the second distance equation in an embodiment. Fig. 21 shows the shape of a left curve. Points P1, P2, and P3 in Fig. 21 are auxiliary points used to formulate the second distance equation. Angle th1, angle th2, and angle th3 each represent the angle between the line connecting points P1 to P3 and the center of the virtual lidar and the -y axis.

[0128] Fig. 22 is a second diagram showing an example of auxiliary points used to formulate the second distance equation in the embodiment. Fig. 22 shows the shape of a right curve. Points P1, P2, and P3 in Fig. 22 are auxiliary points used to formulate the second distance equation. Angle th1, angle th2, and angle th3 represent the angles formed by the -y axis and a line connecting points P1 to P3, respectively, with the center of the virtual lidar.

[0129] In formulating the second distance formula, the model (a set of formulas that represent shapes) is switched before and after angles th1, th2, and th3, and the curve is expressed using formulas that represent three shapes: a straight line, the first straight line of an ellipse, and a straight line that intersects with it.

[0130] The set of equations (24) to (40) below is an example of the second distance equation. The units of angles th and angle are radians.

[0131]

number

[0132]

number

[0133]

number

[0134]

number

[0135]

number

[0136]

number

[0137]

number

[0138]

number

[0139]

number

[0140]

number

[0141]

number

[0142]

number

[0143]

number

[0144]

number

[0145]

number

[0146]

number

[0147]

number

[0148] Equation (36) is an equation that holds when the road beyond the curve is perpendicular to the first road. Furthermore, if the center of the ellipse is defined as xc, yc, and the intersections of the Virtual Lidar signal and the ellipse are defined as x_e and y_e, then x_eth and y_eth can be obtained from the simultaneous equations in equation (37). As a result, the value of the left side of equation (38) is obtained. Equations (39) and (40) represent the calculations performed in the extraction process when angle≠0. More specifically, they represent the calculations performed after all VLScurve.th and VLSsecond_road.th are calculated.

[0149] (Third specific example) The third specific example is a formula (hereinafter referred to as the "third distance formula") that represents the distance from the center of the virtual lidar to the boundary on the VLS plane when the road is an intersection. The third distance formula when the center of the virtual lidar is located at the origin of the VLS plane is an example of a reference area expression function.

[0150] Here, we will explain the formulation of the shapes of three roads: right-hand T-shaped intersections, left-hand T-shaped intersections, and T-junctions. The shape of a T-shaped road is expressed by an equation that uses the distance θD1 to the intersection as a parameter in addition to the parameters used to express the curve. Therefore, the shape of a T-shaped road is expressed by the following equation (41).

[0151]

number

[0152] Fig. 23 is a diagram showing an example of the shape of a T-shaped road in an embodiment. Fig. 23 shows a road going from the bottom to the top of the screen (i.e., in the positive direction of the y-axis), a road with an intersection, and a road branching into a road going left and a road going up.

[0153] The following equation (42) represents the shape of a right-hand T-shaped road.

[0154]

number

[0155] The following equation (43) represents the shape of a left-hand T-shaped road.

[0156]

number

[0157] The following equation (44) represents the shape of a T-junction road.

[0158]

number

[0159] Equation (44) for the T-junction is derived using θD1 as the distance to the intersection, just like the equation for the T-junction.

[0160] These formulas are used to formulate the distance from the center of the virtual lidar to the edge of the road.

[0161] Fig. 24 is a diagram showing an example of auxiliary points used to formulate the third distance equation in an embodiment. Fig. 24 shows the shape of a road with a left-hand T-shaped curvature. Points P1, P2, and P3 in Fig. 24 are auxiliary points used to formulate the third distance equation. Angle th1, angle th2, and angle th3 each represent the angle between the line connecting points P1 to P3 and the center of the virtual lidar and the -y axis.

[0162] The formula switches before and after angles th1, th2, th3, and th4, and the shape of the intersection is expressed using two formulas: one that represents a straight line and one that represents a line perpendicular to the first straight line.

[0163] The following equation (45) is an example of the third distance equation at an intersection.

[0164]

number

[0165] The following equation (46) is an example of the third distance equation for a right-hand intersection.

[0166]

number

[0167] The following equation (47) is an example of the third distance equation for a left-hand intersection.

[0168]

number

[0169] The following equation (48) is an example of the third distance equation for a T-junction.

[0170]

number

[0171] The equation representing the line perpendicular to the first line is given by the following equation (49).

[0172]

number

[0173] As in the case where the road shape is curved, if angle≠0, the calculation of the following equation (50) is performed in the extraction process. More specifically, if angle≠0, the calculation of all VLSinsec,thVLSsecond_road,th is performed in the extraction process.

[0174]

number

[0175] Here, we will explain the results of straight line and curve classification. Specifically, straight line and curve classification is a process of classifying roads based on the observation results of Virtual Lidar using the equations for straight line, right curve, and left curve. More specifically, straight line and curve classification is a process of determining whether the target road is straight or curved. Figures 25 and 27 show an example of an overhead image and the distance from the center of Virtual Lidar to the boundary on the VLS plane. Figures 26 and 28 show the results estimated using the equations for straight line, right curve, and left curve.

[0176] 25 is a first explanatory diagram illustrating an example of a classification result in the embodiment, which shows that the road is a straight road.

[0177] FIG. 26 is a second explanatory diagram illustrating an example of classification results in an embodiment. More specifically, FIG. 26 shows classification results for the road shown in FIG. 25. In FIG. 26, "straight" indicates the result of estimation using a straight-line equation, "Right curve" indicates the result of estimation using a right-curve equation, and "Left curve" indicates the result of estimation using a left-curve equation. FIG. 26 shows that a straight-line equation was selected, and that when a straight-line equation is used, the degree of agreement between the estimation results and the observation results is also high. Therefore, FIG. 26, together with the results of FIG. 25, shows that the shape of the road was estimated with high accuracy.

[0178] 27 is a third explanatory diagram illustrating an example of a classification result in the embodiment, showing that the road curves to the right.

[0179] FIG. 28 is a fourth explanatory diagram illustrating an example of a classification result in an embodiment. More specifically, FIG. 28 shows the classification result for the road shown in FIG. 27. In FIG. 28, "straight" indicates the result of estimation using a straight line equation, "Right curve" indicates the result of estimation using a right curve equation, and "Left curve" indicates the result of estimation using a left curve equation. FIG. 28 shows that the right curve equation was selected, and that when the right curve equation was used, the degree of agreement between the estimation result and the observation result was also high. Therefore, FIG. 28, together with the result of FIG. 27, shows that the road shape was estimated with high accuracy.

[0180] Next, the results of intersection classification will be explained. Intersection classification is, specifically, a process of classifying roads based on observation results from Virtual Lidar using the formulas: straight line, right-hand T-shaped intersection, left-hand T-shaped intersection, and T-junction. More specifically, intersection classification is a process of determining whether the target road is a straight line, right-hand T-shaped intersection, left-hand T-shaped intersection, or T-junction. Figures 29, 31, and 33 show examples of overhead images and the distance from the center of Virtual Lidar to the boundary on the VLS plane. Figures 30, 32, and 34 show the results of estimation using the formulas: straight line, right-hand T-shaped intersection, left-hand T-shaped intersection, and T-junction.

[0181] Fig. 29 is a fifth explanatory diagram illustrating an example of a classification result in the embodiment, showing that the shape of the road is a right-hand T-shape.

[0182] FIG. 30 is a seventh explanatory diagram illustrating an example of classification results in an embodiment. More specifically, FIG. 30 shows classification results for the road shown in FIG. 29. In FIG. 30, "straight" indicates the estimation result using the straight line equation, "Left insec" indicates the estimation result using the left-hand T-junction equation, "T insec" indicates the estimation result using the T-junction equation, and "Right insec" indicates the estimation result using the right-hand T-junction equation. FIG. 30 shows that the right-hand T-junction equation was selected, and that when the right-hand T-junction equation was used, the degree of agreement between the estimation result and the observation result was also high. Therefore, FIG. 30, together with the results of FIG. 29, shows that the road shape was estimated with high accuracy.

[0183] Fig. 31 is an eighth explanatory diagram illustrating an example of a classification result in the embodiment. Fig. 31 shows that the shape of the road is a left-hand T-shape.

[0184] FIG. 32 is a ninth explanatory diagram illustrating an example of classification results in an embodiment. More specifically, FIG. 32 shows classification results for the road shown in FIG. 31. In FIG. 32, "straight" indicates the estimation result using the straight line equation, "Left insec" indicates the estimation result using the left T-shaped intersection equation, "T insec" indicates the estimation result using the T-junction equation, and "Right insec" indicates the estimation result using the right T-shaped intersection equation. FIG. 32 shows that the left T-shaped intersection equation was selected, and that when the left T-shaped intersection equation was used, the degree of agreement between the estimation result and the observation result was also high. Therefore, FIG. 32, together with the results of FIG. 31, shows that the shape of the road was estimated with high accuracy.

[0185] Fig. 33 is a tenth explanatory diagram illustrating an example of a classification result in the embodiment. Fig. 33 shows that the shape of the road is a T-junction.

[0186] FIG. 34 is an eleventh explanatory diagram illustrating an example of classification results in an embodiment. More specifically, FIG. 34 shows classification results for the road shown in FIG. 33. In FIG. 34, "straight" indicates the estimation result using the equation for a straight line, "Left insec" indicates the estimation result using the equation for a left-hand T-junction, "T insec" indicates the estimation result using the equation for a T-junction, and "Right insec" indicates the estimation result using the equation for a right-hand T-junction. FIG. 34 shows that the equation for a T-junction was selected, and that when the equation for a T-junction was used, the degree of agreement between the estimation result and the observation result was also high. Therefore, FIG. 34, together with the results of FIG. 33, shows that the shape of the road was estimated with high accuracy.

[0187] Note that the horizontal axis represents the line of sight angle and the vertical axis represents the distance in graph G1 in Fig. 26, graph G3 in Fig. 28, graph G5 in Fig. 30, graph G7 in Fig. 32, and graph G9 in Fig. 34. The horizontal axis represents the x-axis coordinate value on the VLS plane and the vertical axis represents the y-axis coordinate value on the VLS plane in graph G2 in Fig. 26, graph G4 in Fig. 28, graph G6 in Fig. 30, graph G8 in Fig. 32, and graph G10 in Fig. 34.

[0188] The autonomous mobile body control device 1 of the embodiment configured in this manner determines conditions that minimize errors using reference area information based on area boundary distance information, and acquires target information from the determined conditions. Therefore, the autonomous mobile body control device 1 configured in this manner can improve the accuracy of movement of the autonomous mobile body 9.

[0189] (Variation) It should be noted that the road area may not be correctly recognized if a moving obstacle is part of the subject of the image capturing device 902. In such a case, the area may be estimated by performing fitting (i.e., error minimization processing) that ignores the moving obstacle.

[0190] FIG. 35 is a diagram showing an example of the results of fitting (specifically, error minimization processing) performed by the autonomous mobile body control device 1 in a modified example, ignoring a moving obstacle when part of the subject is a moving obstacle. The horizontal axis of FIG. 35 represents the line of sight angle, and the vertical axis of FIG. 35 represents the distance. The "deleted data" in FIG. 35 is an example of the measurement results of a virtual lidar for a moving obstacle. The "true data" in FIG. 35 is data of a subject that is not a moving obstacle. In other words, unlike the "deleted data," this data is not ignored in the error minimization processing. The "estimated curve" in FIG. 35 is an example of the results of fitting (i.e., error minimization processing) performed using only the results of the "true data," ignoring the moving obstacle. FIG. 35 shows that the autonomous mobile body control device 1 appropriately estimates the area even when the moving obstacle is ignored.

[0191] As explained in the third specific example, when the road is an intersection, reference area information using a plurality of reference area expression functions is used.

[0192] An example of the flow of processing executed by the autonomous mobile body control device 1 when part of the subject is a moving obstacle will be described with reference to FIG.

[0193] FIG. 36 is a flowchart showing an example of the flow of processing executed by the autonomous mobile body control device 1 when part of the subject is a moving obstacle in the modified example.

[0194] The progress status information acquisition unit acquires progress status information (step S301). Next, the area boundary distance information acquisition unit 102 acquires the image to be processed (step S302). Next, the area boundary distance information acquisition unit 102 reads out from the storage unit 13 a segmentation model, which is a trained model previously recorded in the storage unit 13 and determines which of predetermined categories a pixel belongs to (step S303). The predetermined categories include at least dynamic obstacles. An example of generating a segmentation model will be described with reference to FIG. 37.

[0195] The area boundary distance information acquisition unit 102 acquires pixel values ​​of pixels in the image to be processed that are centered around a target pixel, which is a pixel selected according to a predetermined rule (step S304). Next, the area boundary distance information acquisition unit 102 determines a category to which the target pixel belongs using a segmentation model (step S305). Next, the category to which the target pixel belongs is recorded in the storage unit 13 (step S306). If it is determined in step S305 that the category to which the target pixel belongs is a moving obstacle, the storage unit 13 records the moving obstacle as the category to which the target pixel belongs. If it is determined in step S305 that a category other than a moving obstacle (hereinafter referred to as "category A") is the category to which the target pixel belongs, the storage unit 13 records category A as the category to which the target pixel belongs.

[0196] After step S306, the area boundary distance information acquisition unit 102 determines whether category determination has been performed for all pixels (step S307). If there are pixels for which category determination has not yet been performed (step S307: NO), the area boundary distance information acquisition unit 102 selects the next target pixel according to a predetermined rule (step S308). The next target pixel is, for example, the pixel adjacent to the current target pixel. After step S308, the process returns to step S304.

[0197] On the other hand, if the category determination has been performed for all pixels (step S307: YES), the area boundary distance information acquisition unit 102 executes boundary pixel information acquisition processing (step S309). Next, the area boundary distance information acquisition unit 102 acquires values ​​of pixels in the processing target image other than the pixels whose category to which they belong has been determined to be a moving obstacle by the processing of step S305 (step S310).

[0198] Next, the area boundary distance information acquisition unit 102 executes distance association processing using the values ​​acquired in step S310 (step S311). Therefore, in the processing of step S311, the values ​​of pixels that have been determined to belong to the category of a moving obstacle in the processing of step S305 are not used.

[0199] Next, the area boundary distance information acquisition unit 102 executes a virtual space distance measurement process using the result of step S311 (step S312). By executing the process of step S312, area boundary distance information is obtained.

[0200] In this way, by executing the processes of steps S302 to S312, the area boundary distance information acquisition unit 102 acquires area boundary distance information after deleting information about the moving obstacle. Deleting information about the moving obstacle means not using the values ​​of pixels determined to belong to the moving obstacle, and specifically means the process of step S310.

[0201] After step S312, the reference area information acquisition unit 103 acquires reference area information stored in the storage unit 13 (step S313). More specifically, the reference area information acquisition unit 103 reads out one or more reference area expression functions stored in the storage unit 13. Next, the target information acquisition unit 104 executes an error minimization process using the reference area information and the area boundary distance information acquired in step S312 (step S314). The error minimization process is a process for determining conditions that minimize the error, which is the difference between the reference area information acquired in step S313 and the area boundary distance information acquired in step S312.

[0202] Next, the destination information acquisition unit 104 executes a destination information acquisition process (step S315). Next, the control signal generation unit 105 generates a control signal for controlling the operation of the autonomous moving object 9 based on the destination information, and controls the operation of the autonomous moving object 9 using the generated control signal (step S316).

[0203] The error minimization process executed in step S314 is fitting that ignores moving obstacles, as described in Fig. 35 and its description. Fitting that ignores moving obstacles means fitting that does not use data indicating the distance to a moving obstacle from the area boundary distance information.

[0204] As shown in Fig. 36, the target information acquisition unit 104 acquires target information by executing an error minimization process that determines conditions for minimizing the error, which is the difference between the reference area information and the area boundary distance information. The error minimization process uses the reference area information. As described above, the reference area information is information that uses one or more reference area expression functions, which are functions that represent the position, orientation, and shape of the target area and have one or more parameters. As described above, the area boundary distance information is information that indicates the distance from the autonomous moving body 9 to each position on the boundary of the target area.

[0205] As shown in Figure 36, the area boundary distance information acquisition unit 102 acquires area boundary distance information after deleting information about moving obstacles, and the target information acquisition unit 104 acquires target information by performing an error minimization process that determines the conditions for minimizing the error, which is the difference between the reference area information and the area boundary distance information.

[0206] 37 is a flowchart showing an example of the flow of generating a segmentation model in a modified example. Before describing the flowchart, an overview of the generation of a segmentation model will be described.

[0207] A segmentation model is a trained mathematical model obtained by updating a pre-prepared mathematical model (hereinafter referred to as the "training stage model") that estimates the category to which each pixel of an image belongs based on an input image using machine learning methods.

[0208] A mathematical model is a set of one or more processes whose execution conditions and order (hereinafter referred to as "execution rules") are predetermined. For simplicity of the following explanation, updating a mathematical model using machine learning methods is referred to as "learning." Furthermore, updating a mathematical model means appropriately adjusting the values ​​of parameters included in the mathematical model. Furthermore, executing a mathematical model means executing each process included in the mathematical model in accordance with the execution rules.

[0209] The training model may be configured in any manner as long as it is a mathematical model that is updated by a machine learning method. The training model may be configured, for example, by a neural network. The training model may be configured, for example, by a neural network including a convolutional neural network. The training model may be configured, for example, by a neural network including an autoencoder.

[0210] The training samples used to train the learning model are pairs of data consisting of an image and an annotation indicating the category to which each pixel in the image belongs. The loss function used to update the learning model is a function whose value indicates the difference between the category of each pixel estimated based on the input image and the annotation. The annotation is data expressed as a tensor, for example. Updating the learning model means updating the values ​​of the parameters included in the learning model according to a predetermined rule so as to reduce the value of the loss function.

[0211] Now, the flowchart in Figure 37 will be explained. A training sample is input to the learning model (step S401). Next, the learning model is executed to estimate a category for each pixel in the image containing the input training sample (step S402).

[0212] Based on the estimation result obtained by the processing of step S402, the values ​​of the parameters included in the learning-stage model are updated so as to reduce the value of the loss function (step S403). Updating the values ​​of the parameters included in the learning-stage model means updating the learning-stage model. After step S403, it is determined whether a predetermined termination condition (hereinafter referred to as "learning termination condition") is satisfied (step S404). The learning termination condition is, for example, a condition that a predetermined number of updates have been performed.

[0213] If the learning termination condition is met (step S404: YES), the learning-stage model is recorded as a segmentation model in the storage unit 13 (step S405). On the other hand, if the learning termination condition is not met (step S404: NO), the process returns to step S402. Depending on the learning algorithm, the process returns to step S401, and a new training sample is input to the learning-stage model.

[0214] <Experimental Results> An example of the results of an experiment using the autonomous mobile object control device 1 will be described. The experiment was aimed at estimating parameters for a straight road. Specifically, the parameters were the slope of the road, the width of the right road, and the width of the left road. Three experiments, Experiment 1 to Experiment 3, were conducted under different conditions. Experiment 1 was an outdoor experiment using a monocular camera as the imaging device 902.

[0215] The first experiment was conducted in two locations, the first outdoor location and the second outdoor location. The second experiment was conducted indoors using a monocular camera as the imaging device 902. The first experiment was conducted in two locations, the first indoor location and the second indoor location. The third experiment was conducted indoors using 2DLiDAR (Two dimensional Light Detection And Ranging) as the imaging device 902.

[0216] In the experiment, the inclinations were -30 degrees, -20 degrees, -10 degrees, 0 degrees, 10 degrees, 20 degrees, and 30 degrees. In the experiment, the road width was the width of the road from left to right. The road width was measured during the experiment.

[0217] Fig. 38 is a diagram showing the experimental environment of the first experiment conducted outdoors in a modified example, and shows a photograph of the first outdoors.

[0218] Fig. 39 is a first diagram showing the results of a first experiment conducted outdoors in a modified example. Fig. 39 shows the results of projecting the segmentation results for the image in Fig. 38 onto an overhead view and the field of view boundary distance.

[0219] Fig. 40 is a second diagram showing the results of the first experiment conducted outdoors in the first modified example. Fig. 40 shows that the road shape can be appropriately estimated using the observed values ​​on the VLS plane.

[0220] Fig. 41 is a third diagram showing the results of the first experiment conducted outdoors in the first modified example. Fig. 41 shows, in a bird's-eye view, that the road shape can be appropriately estimated using the observed values.

[0221] Fig. 42 is a diagram showing the experimental environment of the first experiment conducted in a second outdoor location in a modified example, and shows a photograph of the second outdoor location.

[0222] Fig. 43 is a first diagram showing the results of a first experiment conducted outdoors in a second modified example. Fig. 43 shows the results of projecting the segmentation results for the image in Fig. 42 onto an overhead view and the field of view boundary distance.

[0223] Fig. 44 is a second diagram showing the results of the first experiment conducted outdoors in the second modified example. Fig. 44 shows that the road shape can be appropriately estimated using the observed values ​​on the VLS plane.

[0224] Fig. 45 is a third diagram showing the results of the first experiment conducted outdoors in the second modified example. Fig. 45 shows, in a bird's-eye view, that the road shape can be appropriately estimated using the observed values.

[0225] Fig. 46 is a diagram showing the experimental environment of the second experiment conducted in the first indoor space in the modified example, and shows a photograph of the first indoor space.

[0226] Fig. 47 is a first diagram showing the results of a second experiment conducted indoors in the first modified example. Fig. 47 shows the results of projecting the segmentation results for the image in Fig. 46 onto an overhead view and the field of view boundary distance.

[0227] Fig. 48 is a second diagram showing the results of a second experiment conducted indoors in the first modified example. Fig. 48 shows that the road shape can be appropriately estimated using the observed values ​​on the VLS plane.

[0228] Fig. 49 is a third diagram showing the results of the second experiment conducted indoors in the first modified example. Fig. 49 shows, in a bird's-eye view, that the shape of the road can be appropriately estimated using the observed values.

[0229] Fig. 50 is a diagram showing the experimental environment of the second experiment conducted in the second indoor space in the modified example, and shows a photograph of the second indoor space.

[0230] Fig. 51 is a first diagram showing the results of a second experiment conducted indoors in a second modified example. Fig. 51 shows the results of projecting the segmentation results for the image in Fig. 50 onto an overhead view and the field of view boundary distance.

[0231] Fig. 52 is a second diagram showing the results of a second experiment conducted indoors in a second modified example. Fig. 52 shows that the road shape can be appropriately estimated using observations on the VLS plane.

[0232] Fig. 53 is a third diagram showing the results of a second experiment conducted indoors in a second modified example. Fig. 53 shows, in a bird's-eye view, that the shape of the road can be appropriately estimated using the observed values.

[0233] Fig. 54 is a diagram showing the experimental environment of the third experiment in the modified example. Fig. 54 shows a photograph of the location where the third experiment was conducted. In the third experiment, a 2DLiDAR was used as the image capturing device 902.

[0234] Fig. 55 is the first diagram showing the results of the third experiment in the modified example. Fig. 55 shows that the road shape can be properly estimated using the observation values ​​on the VLS plane. This is because there are walls at the boundaries of the road area.

[0235] Fig. 56 is the second diagram showing the results of the third experiment in the modified example. Fig. 56 shows in a bird's-eye view that the road shape can be properly estimated using the observed values. This is because there are walls at the boundaries of the road area.

[0236] Figure 57 is a first diagram showing the accuracy of the tilt and road width measurement results obtained based on the experimental results from Experiments 1 and 2. Figure 57 shows that for the outdoor experiment, there was an average error of 7.08 degrees for tilt, an average error of 0.670 meters for left road width, and an average error of 0.634 meters for right road width. Figure 57 shows that for the indoor experiment, there was an average error of 6.41 degrees for tilt, an average error of 0.363 meters for left road width, and an average error of 0.356 meters for right road width.

[0237] Figure 58 is a second diagram showing the accuracy of the measurement results of the slope and road width obtained based on the experimental results of Experiments 1 and 2. Figure 58 shows the results of normalizing the results of Figure 57. The standard slope used for normalization was -60 degrees to 60 degrees, and the standard road width was 4.0 meters for the outdoor experiment and 1.92 meters for the indoor experiment.

[0238] Figure 58 shows that for the outdoor experiment, there was a 5.9 percent error rate for tilt, a 17.6 percent error rate for left path width, and a 16.8 percent error rate for right path width. Figure 58 shows that for the indoor experiment, there was a 5.34 percent error rate for tilt, a 9.57 percent error rate for left path width, and a 9.47 percent error rate for right path width.

[0239] Figure 59 is a diagram showing the experimental environment of the control experiment. In the control experiment, a 2DLiDAR was used as the imaging device 902. Figure 59 is a photographic image of the experimental environment of the control experiment. As Figure 59 shows, the control experiment was conducted in an outdoor environment similar to that of the first outdoor experiment.

[0240] Figure 60 shows an example of the experimental results of a control experiment. The horizontal axis of the figure indicates the field of view angle [°], and the vertical axis of the figure indicates the distance. The graph in the figure shows the measurement results of the 2DLiDAR in the control experiment. The measurement result in an area where no reflecting object exists was recorded as 0 meters.

[0241] Comparing the results of Figure 60 with the results of the first experiment conducted outdoors, it can be seen that when the image capturing device 902 is a 2D LiDAR, it may not be possible to estimate the road area. Therefore, if the image capturing device 902 is the monocular camera or a 3D LiDAR described above, it is possible to estimate the road area, but when it is a 2D LiDAR, it may not be possible to estimate the road area. This is an essential problem with 2D LiDAR.

[0242] It should be noted that the area boundary distance information does not necessarily have to be acquired from the image capturing device 902. The area boundary distance information may be acquired from an information processing device connected to be able to communicate via a network, such as a management device on a network, such as a server on the network. It should be noted that the image to be processed does not necessarily have to be acquired from the image capturing device 902. The image to be processed may be acquired from an information processing device connected to be able to communicate via a network, such as a management device on a network, such as a server on the network.

[0243] The autonomous mobile body control device 1 may be implemented using a plurality of information processing devices communicably connected via a network. In this case, the functional units of the autonomous mobile body control device 1 may be distributed and implemented in the plurality of information processing devices.

[0244] All or part of the functions of the autonomous mobile object control device 1 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

[0245] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0246] 1...Autonomous mobile body control device, 10...Control unit, 11...Input unit, 12...Communication unit, 13...Memory unit, 14...Output unit, 101...Progress state information acquisition unit, 102...Area boundary distance information acquisition unit, 103...Reference area information acquisition unit, 104...Destination information acquisition unit, 105...Control signal generation unit, 9...Autonomous mobile body, 900...Road, 902...Photographing device, 905...Mobile body

Claims

1. an area boundary distance information acquisition unit that acquires area boundary distance information, which is information indicating distances to each position on the boundary of a target area, which is an area in which the autonomous moving body is located, based on a processing target image captured by an imaging device attached to the autonomous moving body to be controlled, parameters of the imaging device, and a result of distinguishing a target area shown in the processing target image from other areas by executing area segmentation processing on the processing target image; a destination information acquisition unit that acquires destination information that is information indicating a relationship between the autonomous moving body and the target area based on reference area information that is a mathematical model that represents candidates for the shape of the target area and the area boundary distance information; An autonomous mobile object control device comprising:

2. Further comprising a reference region information acquisition unit that acquires the reference region information, the reference area information includes the mathematical model corresponding to any one of the types of boundary shapes of the target area; a plurality of pieces of reference region information each containing a different type of mathematical model is stored in advance; the objective information is calculated for each type of mathematical model based on the reference area information and the area boundary distance information for each type of mathematical model, the reference area information acquisition unit acquires the plurality of pieces of reference area information, The objective information acquisition unit acquires any one of the objective information selected from the objective information calculated for each type of the mathematical model. The autonomous mobile object control device according to claim 1 .

3. the target information acquisition unit executes a process to determine a condition for minimizing an error, which is a difference between a mapping graph representing the reference area information and a mapping graph representing the area boundary distance information, and acquires the target information based on the condition of the execution result. The autonomous mobile object control device according to claim 1 or 2.

4. the reference area information changes based on a parameter that represents at least a state of the target area as seen from the autonomous moving body; The autonomous mobile object control device according to claim 1 .

5. the reference area information includes information indicating the position of a boundary of the target area that is not photographed by an image capturing device that runs parallel to the autonomous moving body and faces the direction of the autonomous moving body, The autonomous mobile object control device according to claim 1 .

6. the target information acquisition unit acquires the target information by executing an error minimization process that determines a condition for minimizing an error, which is a difference between the reference area information and the area boundary distance information; the error minimization process uses the reference region information using one or more reference region expression functions, which are functions that represent the position, orientation, and shape of the target region and have one or more parameters; The autonomous mobile object control device according to claim 1 .

7. the area boundary distance information acquisition unit acquires the area boundary distance information after deleting information about moving obstacles; the target information acquisition unit acquires the target information by executing an error minimization process that determines a condition for minimizing an error, which is a difference between the reference area information and the area boundary distance information. The autonomous mobile object control device according to claim 1 .

8. a target information acquisition step of acquiring target information, which is information indicating a relationship between the autonomous moving body and the target area, based on a processing target image captured by an imaging device attached to the autonomous moving body to be controlled, parameters of the imaging device, and a result of performing area segmentation processing on the processing target image to distinguish a target area appearing in the processing target image from other areas, based on reference area information, which is a mathematical model representing candidates for the shape of a target area, which is an area in which the autonomous moving body is located, and area boundary distance information, which is information indicating distances to each position on the boundary of the target area; An autonomous mobile object control method comprising:

9. The autonomous mobile object control device according to any one of claims 1 to 7, wherein a computer is operated as the autonomous mobile object control device. A program to make it work.

Images