Control device, control method, and program

Abstract

To appropriately evaluate the risk of a travel orbit while reducing processing loads.SOLUTION: A control device includes: a surrounding image acquisition unit that acquires a surrounding image photographed with a fish eye camera loaded on a mobile body; a base orbit calculation unit that calculates an instruction related to future travel of the mobile body as a base orbit in an orthogonal coordinate system; a coordinate conversion unit that converts the base orbit in the orthogonal coordinate system acquired into a base orbit in a fish eye camera coordinate system; a risk calculation unit that calculates a risk of the base orbit in the fish eye camera coordinate system based on the surrounding image and the base orbit in the fish eye camera coordinate system; and a travel orbit calculation unit that calculates a travel orbit by correcting the base orbit in the orthogonal coordinate system based on the risk of the base orbit in the fish eye camera coordinate system. The travel orbit calculation unit provides different amounts of correction of an orbit point on the base orbit in the orthogonal coordinate system in accordance with the degree of the risk at an orbit point corresponding to the base orbit in the fish eye camera coordinate system.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a control device, a control method, and a program. [Background technology]

[0002] There is known a technology for mounting a fisheye camera on an autonomously mobile robot. For example, Patent Document 1 discloses a technology for mounting a fisheye camera on an autonomously mobile robot and calculating the robot's travel trajectory based on images captured by the fisheye camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-303137 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 involves setting up markers to indicate specific locations and running the robot using the relative distance and relative direction between the marker and the robot. However, setting up markers for running the robot is time-consuming.

[0005] Furthermore, the conventional technology involves converting the coordinates of all points in an image captured by a fisheye camera from the fisheye camera coordinate system to a Cartesian coordinate system, evaluating the risk of the driving trajectory based on the converted coordinates, and correcting the driving trajectory. Therefore, it is expected that the processing load required for converting the coordinates from the fisheye camera coordinate system to the Cartesian coordinate system is large.

[0006] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a control device, a control method, and a program that can appropriately evaluate the risks of a driving trajectory while reducing the processing load. [Means for solving the problem]

[0007] The control device, control method, and program according to the present invention employ the following configuration. (1): A control device according to one embodiment of the present invention includes a peripheral image acquisition unit that acquires a peripheral image of a moving body, which is an image captured by a fisheye camera mounted on the moving body; a base trajectory calculation unit that calculates instructions regarding the future driving of the moving body as a base trajectory in a Cartesian coordinate system; a coordinate conversion unit that coordinate converts the base trajectory in the acquired Cartesian coordinate system into a base trajectory in the fisheye camera coordinate system; a risk calculation unit that calculates a risk of the base trajectory in the fisheye camera coordinate system based on the peripheral image and the base trajectory in the fisheye camera coordinate system; and a driving trajectory calculation unit that calculates a driving trajectory by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system, wherein the driving trajectory calculation unit varies the amount of correction of a trajectory point on the base trajectory in the Cartesian coordinate system depending on the magnitude of risk at a corresponding trajectory point of the base trajectory in the fisheye camera coordinate system.

[0008] (2): In the above aspect (1), the coordinate conversion unit converts the driving trajectory in the orthogonal coordinate system into a driving trajectory in the fisheye camera coordinate system, the risk calculation unit calculates the risk of the driving trajectory in the fisheye camera coordinate system based on the surrounding image and the driving trajectory in the fisheye camera coordinate system, and the driving trajectory calculation unit recalculates the driving trajectory by correcting the driving trajectory in the orthogonal coordinate system based on the risk of the driving trajectory in the fisheye camera coordinate system.

[0009] (3): In the above aspect (1) or (2), a driving control unit is further provided which drives the moving body along the base track or the running track, and the driving control unit drives the moving body along the base track or the running track when the risk calculation unit determines that the risk of the base track or the running track is equal to or lower than a threshold value.

[0010] (4): In any of the above aspects (1) to (3), a gesture detection unit is further provided that detects a body movement by a user indicating an instruction regarding the future travel of the moving body, and the base trajectory calculation unit calculates a base trajectory in the Cartesian coordinate system based on the body movement detected by the gesture detection unit.

[0011] (5) A control method according to another aspect of the present invention involves a computer mounted on a moving body acquiring a peripheral image of the moving body, which is an image captured by a fisheye camera mounted on the moving body, acquiring instructions regarding the future driving of the moving body as a base trajectory in a Cartesian coordinate system, coordinate-transforming the acquired base trajectory in the Cartesian coordinate system into a base trajectory in a fisheye camera coordinate system, calculating a risk of the base trajectory in the fisheye camera coordinate system based on the peripheral image and the base trajectory in the fisheye camera coordinate system, and calculating a driving trajectory by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system.

[0012] (6): A program according to another aspect of the present invention causes a computer mounted on a moving body to acquire peripheral images of the moving body, which are images captured by a fisheye camera mounted on the moving body, acquire instructions regarding the future driving of the moving body as a base trajectory in a Cartesian coordinate system, coordinate-transform the acquired base trajectory in the Cartesian coordinate system into a base trajectory in a fisheye camera coordinate system, calculate a risk of the base trajectory in the fisheye camera coordinate system based on the peripheral images and the base trajectory in the fisheye camera coordinate system, and calculate a driving trajectory by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system. Effect of the Invention

[0013] According to aspects (1) to (6), the processing load can be reduced while appropriately evaluating the risk of a traveling trajectory. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing an example of a situation in which a moving object 10 equipped with a control device according to an embodiment is used. [Diagram 2] FIG. 2 is a diagram for explaining an example of the overall configuration of a moving object 10. [Diagram 3] 1A and 1B are diagrams illustrating examples of gestures performed by a user U. [Figure 4] 13 is a diagram showing an example of coordinate transformation of a base trajectory BL executed by a coordinate transformation unit 140. FIG. [Diagram 5] FIG. 13 is a diagram showing an example of a scene in which the risk calculation unit 150 calculates the risk of a base trajectory BL-2 in the fish-eye camera coordinate system. [Figure 6] 11 is a diagram showing an example of a detection process of a travelable space FS by a travelable space detection unit 160. FIG. [Figure 7] FIG. 11 is a diagram showing an example of a scene in which a running trajectory calculation unit 170 calculates a running trajectory TL-1 in a Cartesian coordinate system. [Figure 8] 13 is a diagram showing an example of a calculation process of the travel trajectory TL by the travel trajectory calculation unit 170 and a coordinate conversion of the travel trajectory TL by the coordinate conversion unit 140. FIG. [Figure 9] 4 is a flowchart showing an example of the flow of processing executed by the control device 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] <Embodiment> DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A control device, a control method, and a program according to an embodiment of the present invention will be described below with reference to the drawings.

[0016] [Overall configuration] FIG. 1 is a diagram showing an example of a situation in which a moving body 10 equipped with a control device according to an embodiment is used. The moving body 10 is an autonomous mobile robot, and includes a fisheye camera 20, a container 30, and wheels 40. The moving body 10 can be used in the following manner. A user U holds a package B, and the moving body 10 moves toward the user U in response to a gesture by the user to store the package B. Since an obstacle OB exists between the moving body 10 and the user U, the moving body 10 autonomously avoids the obstacle OB and moves to the vicinity of the user U. After the user U places the package B in the container 30, the moving body 10 moves accompanying the user U.

[0017] The fisheye camera 20 is, for example, a camera that includes a fisheye lens and can capture images of the periphery of the moving body 10 at a wide angle (for example, 360 degrees). The fisheye camera 20 is attached, for example, to the upper part of the moving body 10, and captures images of the periphery of the moving body 10 at a wide angle in the horizontal direction. The fisheye camera 20 may be realized by combining multiple 120-degree cameras or 60-degree cameras.

[0018] The container 30 is a vessel for storing any item or baggage, and is fixed to the body of the moving body 10.

[0019] The wheels 40 are driven by a plurality of motors 50 mounted inside the mobile body 10, and realize the movement of the mobile body 10. The wheels 40 include, for example, drive wheels driven in a rotational direction by the motors 50, and steering wheels which are non-driven wheels driven in a yaw direction. By adjusting the angle of the steering wheels, the mobile body 10 can change its course.

[0020] In the present invention, it is not essential that the moving body 10 includes the container 30. Furthermore, in this embodiment, the moving body 10 includes the wheels 40 as a mechanism for realizing movement, but the present invention is not limited to this configuration, and for example, the moving body 10 may be a multi-legged robot.

[0021] 2 is a diagram for explaining an example of the overall configuration of the moving body 10. The moving body 10 includes a control device 100 in addition to the above-mentioned fisheye camera 20, container 30, wheels 40, and motor 50. The control device 100 includes, for example, a surrounding image acquisition unit 110, a gesture detection unit 120, a base trajectory calculation unit 130, a coordinate conversion unit 140, a risk calculation unit 150, a travelable space detection unit 160, a travel trajectory calculation unit 170, and a travel control unit 180. Each unit of the control device 100 is realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of the components of the control device 100 may be realized by hardware (including circuitry) such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a graphics processing unit (GPU), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transient storage medium) such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (non-transient storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device.

[0022] The surrounding image acquisition unit 110 acquires an image (hereinafter referred to as a "surrounding image") captured by the fisheye camera 20. Since the acquired surrounding image is a 360-degree image, the surrounding image acquisition unit 110 holds the acquired surrounding image as pixel data in the fisheye camera coordinate system.

[0023] The gesture detection unit 120 detects a body movement (hereinafter referred to as a "gesture") by the user U based on one or more peripheral images. In this embodiment, the gesture is set in advance as an instruction regarding the future travel of the moving body 10. The gesture detection unit 120 stores data (template data) regarding a plurality of gestures in advance, and detects gestures in the peripheral image by matching feature data acquired from the peripheral image with the template data. Each of the feature data and template data is an abstract data representation of skeletal features such as human fingertips, finger joints, wrists, and elbows, and the links connecting them.

[0024] Fig. 3 is a diagram showing an example of a gesture shown by a user U. In Fig. 3, the user U stretches his right hand forward. In this embodiment, the gesture of stretching the right hand forward means "please move forward toward the user U". In addition, the gesture detection unit 120 detects various gestures having the meanings of, for example, rotating, stopping, retreating, and the like, but the detailed description thereof will be omitted.

[0025] In this embodiment, the moving body 10 has a gesture detection unit 120 that detects the gesture of the user U, but other configurations may be used as long as they can recognize instructions regarding the future travel of the moving body 10. For example, the moving body 10 may be provided with a voice recognition unit that recognizes the voice uttered by the user U, and may recognize the instruction by recognizing that the user U has given an instruction regarding the future travel of the moving body 10 by voice (for example, "come"). Furthermore, as another example, the user U may use a dedicated application to transmit information indicating instructions regarding the future travel of the moving body 10 to the moving body 10 from an information processing terminal such as a smartphone owned by the user U, and the moving body 10 may recognize the instruction by receiving the information.

[0026] The base trajectory calculation unit 130 calculates a base trajectory BL, which is initial information on the trajectory on which the moving body 10 should travel in the future, based on the gesture detected by the gesture detection unit 120. At this time, the base trajectory calculation unit 130 calculates the base trajectory BL in a Cartesian coordinate system. The Cartesian coordinate system is a coordinate system whose origin is the position of the moving body 10, which is fixedly set on the body of the moving body 10, and whose axes are in any directions perpendicular to each other. For example, when the user U makes a gesture of extending his / her right hand forward, the base trajectory calculation unit 130 calculates a base trajectory BL-1 whose starting point is the position of the moving body 10, whose initial movement vector coincides with the direction of the moving body 10, and whose ending point is the position immediately before the user U. The base trajectory BL-1 may be calculated so that the front of the moving body 10 (the surface on the side where the container 30 is located) faces the user U at the position immediately before the user U. The base trajectory BL-1 is calculated by fitting the state to a geometric model such as a Bezier curve. The base trajectory is calculated without considering the presence of an obstacle OB, for example. The base trajectory BL is actually generated as a collection of a finite number of trajectory points.

[0027] The coordinate conversion unit 140 performs coordinate conversion between the Cartesian coordinate system and the fisheye camera coordinate system. A one-to-one relationship is established between the coordinates of the Cartesian coordinate system and the fisheye camera coordinate system, and the relationship is held in the coordinate conversion unit 140 as correspondence information. Hereinafter, information in the Cartesian coordinate system is represented by the code "-1", and information in the fisheye camera coordinate system is represented by the code "-2". When the two are not distinguished, "-1" and "-2" are omitted. The coordinate conversion unit 140 converts the base trajectory BL-1 in the Cartesian coordinate system calculated by the base trajectory calculation unit 130 into the base trajectory BL-2 in the fisheye camera coordinate system. The coordinate conversion unit 140 further converts the travel trajectory TL-1 in the Cartesian coordinate system calculated by the travel trajectory calculation unit 170 into the travel trajectory TL-2 in the fisheye camera coordinate system, as described later.

[0028] Here, the running trajectory TL means a trajectory obtained by modifying the base trajectory BL, and as described below, is calculated when, for example, it is determined that there is a high risk of running on the base trajectory BL due to the presence of an obstacle OB.

[0029] Fig. 4 is a diagram showing an example of coordinate conversion of the base trajectory BL executed by the coordinate conversion unit 140. The upper left part of Fig. 4 shows a base trajectory BL-1 calculated by the base trajectory calculation unit 130, and the lower left part of Fig. 4 shows a peripheral image of the moving body 10 acquired by the peripheral image acquisition unit 110. The coordinate conversion unit 140 converts the base trajectory BL-1 into the fisheye camera coordinate system to acquire a base trajectory BL-2 in the fisheye camera coordinate system, and makes it possible to calculate the risk by superimposing the base trajectory BL-2 on the peripheral image.

[0030] The risk calculation unit 150 calculates the risk of each coordinate in the fisheye camera coordinate system, and obtains the distribution of risk in the target area of ​​the fisheye camera coordinate system. The risk is an index value indicating the likelihood that the moving body 10 will collide with another obstacle. The risk calculation unit 150 uses the distribution of risk (risk function) to calculate the risk of the base trajectory BL-2 or the running trajectory TL-2. For example, the risk calculation unit 150 calculates the risk Ji at each trajectory point on the base trajectory BL or the running trajectory TL of the moving body 10, and calculates the risk J of the base trajectory BL-2 or the running trajectory TL-2 by calculating the sum ΣJi of the risks Ji. That is, J=ΣJi is established.

[0031] The risk at a certain trajectory point on the base trajectory BL-2 or the traveling trajectory TL-2 can be calculated based on the distance between the trajectory point and an obstacle and the speed of the obstacle. For example, the risk calculation unit 150 calculates the risk at a certain trajectory point to be higher the smaller the distance between each trajectory point on the base trajectory BL-2 or the traveling trajectory TL-2 and an obstacle, and calculates the risk at the certain trajectory point to be lower the larger the distance between a certain trajectory point on the base trajectory BL-2 or the traveling trajectory TL-2 and an obstacle. More specifically, the risk calculation unit 150 sets the risk within the area of ​​the minimum circumscribing circle that contains the obstacle OB to a predetermined value (for example, 1), and calculates the risk Ji so that it becomes smaller as it moves away from the area and becomes zero at a certain point. Furthermore, since the risk of the moving obstacle OB differs at each future point, risk distribution data is prepared for each time the moving body 10 arrives on the traveling trajectory TL.

[0032] FIG. 5 is a diagram showing an example of a scene in which the risk calculation unit 150 calculates the risk of the base trajectory BL-2 in the fisheye camera coordinate system. FIG. 5 shows the risk calculation unit 150 calculating the risk of the base trajectory BL-2 in the fisheye camera coordinate system acquired by the coordinate conversion unit 140 in FIG. 4. In FIG. 5, the hatched area RR represents a risk area where the risk is positive. In the figure, P1 to P4 are trajectory points that constitute the base trajectory BL-2. The risk calculation unit 150 substitutes the positions of these trajectory points P1 to P4 into a risk function for calculating the risk of each trajectory point, and calculates a risk value J at each trajectory point. P1 , J P2 , J P3 , and J P4 Next, the risk calculation unit 150 calculates the sum of the risk values ​​at each trajectory point, and calculates the risk ΣJ=J of the base trajectory BL. P1 +J P2 +J P3 +J P4 In this way, the risk calculation unit 150 can calculate the risk of the base trajectory BL-2. The method in which the risk calculation unit 150 calculates the risk of the traveling trajectory TL-2 is similar.

[0033] After calculating the risk ΣJ of the base trajectory BL-2 in the fisheye camera coordinate system, the risk calculation unit 150 then compares the calculated risk ΣJ with a threshold Th (e.g., 1) and determines whether or not to make the moving body 10 run along the base trajectory BL. If the risk calculation unit 150 determines that the risk ΣJ is within the threshold Th, it determines to make the moving body 10 run along the base trajectory BL. On the other hand, if the risk calculation unit 150 determines that the risk ΣJ is greater than the threshold Th, it determines not to make the moving body 10 run along the base trajectory BL.

[0034] In this embodiment, the risk calculation unit 150 compares the risk ΣJ with the threshold value Th. However, the method of comparing the risk ΣJ with the threshold value Th is not limited to this configuration. For example, the risk calculation unit 150 may compare each risk value J on the base trajectory BL-2 in FIG. P1 , J P2 , J P3 , and J P4 The maximum value of the P3 ) and compare the maximum value with the threshold value Th. The method of comparing the risk ΣJ with the threshold value Th is advantageous in that it allows the moving body 10 to travel more safely, while the method of comparing the maximum value with the threshold value Th is advantageous in that it allows the moving body 10 to travel more efficiently.

[0035] When the risk calculation unit 150 determines that the moving body 10 will not travel along the base trajectory BL, the travelable space detection unit 160 detects a travelable space FS-2, which is a space in which the moving body 10 can travel, in the fisheye camera coordinate system based on the surrounding image.

[0036] Fig. 6 is a diagram showing an example of detection processing of the travelable space FS by the travelable space detection unit 160. As shown in Fig. 6, since the risk calculation unit 150 has determined not to make the moving object 10 travel along the base trajectory BL, the travelable space detection unit 160 detects the space indicated by the diagonal line, that is, the space extending from the start point (the position of the moving object 10) of the base trajectory BL-2 in the direction of the end point (the position immediately before the user U) and excluding obstacles (obstacle OB and user U), as the travelable space FS-2.

[0037] The travel trajectory calculation unit 170 calculates the correction amount of each trajectory point that is within the range of the travelable space FS-2 detected by the travelable space detection unit 160 as the correction amount in the Cartesian coordinate system based on the risk of each trajectory point calculated by the risk calculation unit 150. Here, the correction amount of each trajectory point in the Cartesian coordinate system is performed by increasing the correction width Δ in the direction perpendicular to the tangent of the trajectory point as the risk value Ji of the corresponding trajectory point in the fisheye camera coordinate system increases. That is, the travel trajectory calculation unit 170 obtains the travel trajectory TL-1 by changing the trajectory points of the base trajectory BL-1 by the correction width Δ according to each risk value Ji. In this way, the travel trajectory calculation unit 170 calculates the correction amount of the corresponding trajectory point in the Cartesian coordinate system based on the risk of each trajectory point in the fisheye camera coordinate system without coordinate conversion of the image captured by the fisheye camera 20 to the Cartesian coordinate system, so that the amount of calculation required for trajectory correction can be reduced.

[0038] Fig. 7 is a diagram showing an example of a scene in which the running trajectory calculation unit 170 calculates the running trajectory TL-1 in the Cartesian coordinate system. In Fig. 7, the trajectory points Q1, Q2, Q3, and Q4 are trajectory points obtained by changing the trajectory points P1, P2, P3, and P4 of the base trajectory BL-1 by the trajectory correction parameter Δ corresponding to each risk value Ji. In Fig. 7, the risk values ​​Ji of the trajectory points P1, P2, P3, and P4 are calculated by P3 , J P2 , J P4 , J P1Since the correction width Δ in the direction perpendicular to the tangent of each orbit point is larger in the order of P3, P2, P4, and P1, the correction width Δ in the direction perpendicular to the tangent of each orbit point is also larger in the order of P3, P2, P4, and P1.

[0039] Next, the coordinate conversion unit 140 converts the travel trajectory TL-1 in the Cartesian coordinate system into a travel trajectory TL-2 in the fisheye camera coordinate system, and the risk calculation unit 150 substitutes the positions of the trajectory points Q1 to Q4 on the travel trajectory TL-2 into a risk function to calculate a risk value J at each trajectory point. Q1 , J Q2 , J Q3 , and J Q4 The risk distribution (risk function) used in this calculation is the same as that used when calculating the risk of the base trajectory BL-2. Next, the risk calculation unit 150 calculates the sum of the risk values ​​at each trajectory point, and calculates the risk of the traveling trajectory TL-1, ΣJ=J Q1 +J Q2 +J Q3 +J Q4 In the case of FIG. 7, since the trajectory points Q1 to Q4 are outside the risk region RR of the obstacle OB, the risk calculation unit 150 calculates ΣJ=J Q1 +J Q2 +J Q3 +J Q4 =0 is obtained.

[0040] After calculating the risk ΣJ of the travel trajectory TL-1 in the fisheye camera coordinate system, the risk calculation unit 150 next compares the calculated risk ΣJ with a threshold Th to determine whether or not to make the moving body 10 travel along the travel trajectory TL. When the risk calculation unit 150 determines that the risk ΣJ is within the threshold Th, it determines that the moving body 10 travels along the travel trajectory TL. On the other hand, when the risk calculation unit 150 determines that the risk ΣJ is greater than the threshold Th, it determines that the moving body 10 is not to travel along the travel trajectory TL, and repeats the same correction until the risk ΣJ becomes equal to or less than the threshold Th, and recalculates the travel trajectory TL. In the case of FIG. 7, the risk ΣJ=0, which is smaller than the threshold Th, so the risk calculation unit 150 determines that the moving body 10 travels along the travel trajectory TL.

[0041] In the above description, the correction width Δ of the trajectory point is assumed to be a value corresponding to each risk value Ji. In this case, the traveling trajectory calculation unit 170 may multiply the trajectory correction parameter Δ by a random number to avoid the traveling trajectory TL-1 being taken as a local solution of the total risk value ΣJ.

[0042] FIG. 8 is a diagram showing an example of the calculation process of the travel trajectory TL by the travel trajectory calculation unit 170 and the coordinate conversion of the travel trajectory TL by the coordinate conversion unit 140. As described above, the travel trajectory calculation unit 170 changes the corresponding trajectory point of the base trajectory BL-1 by the trajectory correction parameter Δ using the correction amount in the orthogonal coordinate system calculated based on the risk of each trajectory point in the fisheye camera coordinate system, and calculates the travel trajectory TL-1. Next, the coordinate conversion unit 140 coordinate-converts the travel trajectory TL-1 in the orthogonal coordinate system to the travel trajectory TL-2 in the fisheye camera coordinate system, and the risk calculation unit 150 calculates the risk of the travel trajectory TL-2 based on the surrounding image and the travel trajectory TL-2 in the fisheye camera coordinate system. In this way, by calculating the correction amount of the corresponding trajectory point in the orthogonal coordinate system based on the risk of each trajectory point in the fisheye camera coordinate system, the risk of the travel trajectory can be appropriately evaluated while reducing the processing load required for the correction.

[0043] When the risk calculation unit 150 determines that the risk value of the base trajectory BL or the running trajectory TL is equal to or less than the threshold Th, the running control unit 180 causes the moving body 10 to run along the base trajectory BL or the running trajectory TL. Specifically, the running control unit 180 outputs a command value to the motor 50 such that the moving body 10 runs along the base trajectory BL or the running trajectory TL, and the motor 50 rotates the wheels 40 according to the command value.

[0044] [Processing flow] Next, a flow of processing by the control device 100 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the flow of processing executed by the control device 100. The processing of this flowchart is executed for each predetermined control cycle.

[0045] First, the peripheral image acquisition unit 110 acquires a peripheral image of the moving body 10 captured by the fisheye camera 20 (step S100). Next, the gesture detection unit 120 detects a gesture indicating an instruction regarding the future travel of the moving body 10 based on the peripheral image acquired by the peripheral image acquisition unit 110 (step S101). Next, the base trajectory calculation unit 130 calculates a base trajectory BL-1 in an orthogonal coordinate system based on the gesture detected by the gesture detection unit 120 (step S102). Next, the coordinate conversion unit 140 converts the base trajectory BL-1 in the orthogonal coordinate system calculated by the base trajectory calculation unit 130 into a base trajectory BL-2 in the fisheye camera coordinate system (step S103). Next, the risk calculation unit 150 calculates the risk of the base trajectory BL-2 in the fisheye camera coordinate system based on the peripheral image and the base trajectory BL-2 in the fisheye camera coordinate system (step S104).

[0046] Next, the risk calculation unit 150 judges whether the risk of the calculated base trajectory BL-2 is equal to or less than the threshold Th (step S105). When the risk calculation unit 150 judges that the risk of the calculated base trajectory BL-2 is equal to or less than the threshold Th, the traveling control unit 180 makes the moving body 10 travel along the base trajectory BL (step S106). On the other hand, when the risk calculation unit 150 judges that the risk of the calculated base trajectory BL-2 is greater than the threshold Th, the travelable space detection unit 160 detects the travelable space FS-2 in the fisheye camera coordinate system based on the surrounding image (step S107). Next, the traveling trajectory calculation unit 170 calculates the correction amount of each trajectory point within the range of the travelable space FS-2 detected by the travelable space detection unit 160 as the correction amount in the Cartesian coordinate system based on the risk of each trajectory point of the base trajectory BL-2 calculated by the risk calculation unit 150 (step S108). Next, the travel trajectory calculation unit 170 calculates the travel trajectory TL-1 by correcting the base trajectory BL-1 by the correction amount in the Cartesian coordinate system (step S109). Next, the coordinate conversion unit 140 converts the travel trajectory TL-1 in the Cartesian coordinate system into a travel trajectory TL-2 in the fisheye camera coordinate system (step S110). Next, the risk calculation unit 150 calculates the risk of the travel trajectory TL-2 based on the surrounding image and the travel trajectory TL-2 in the fisheye camera coordinate system (step S111).

[0047] Next, the risk calculation unit 150 judges whether the risk of the calculated travel trajectory TL-2 is equal to or less than the threshold Th (step S112). When the risk calculation unit 150 judges that the risk of the calculated travel trajectory TL-2 is equal to or less than the threshold Th, the travel control unit 180 makes the moving body 10 travel along the travel trajectory TL (step S113). On the other hand, when the risk calculation unit 150 judges that the risk of the calculated travel trajectory TL-2 is greater than the threshold Th, the process returns to step S109 again, and the travel trajectory TL-1 is recalculated.

[0048] As described above, according to the embodiment of the present invention, the control device calculates the amount of correction of the base trajectory in the orthogonal coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system without coordinate transforming the image captured by the fisheye camera into an orthogonal coordinate system, then again transforms the traveling trajectory obtained by correcting the base trajectory in the orthogonal coordinate system by the amount of correction into the fisheye camera coordinate system, and evaluates the risk of the traveling trajectory in the fisheye camera coordinate system. This makes it possible to appropriately evaluate the risk of the traveling trajectory while reducing the processing load.

[0049] The above-described embodiment can be expressed as follows. A storage device storing a program; a hardware processor; The hardware processor executes the program stored in the storage device to obtain an image of the surroundings of the moving object, the image being captured by a fisheye camera mounted on the moving object; calculating instructions for future travel of the moving body as a base trajectory in a Cartesian coordinate system; The base trajectory in the acquired orthogonal coordinate system is coordinate-transformed into a base trajectory in a fish-eye camera coordinate system; Calculating a risk of a base trajectory in the fish-eye camera coordinate system based on the peripheral image and the base trajectory in the fish-eye camera coordinate system; Calculating a travel trajectory by correcting a base trajectory in the orthogonal coordinate system based on a risk of a base trajectory in the fisheye camera coordinate system; The control device is configured as follows.

[0050] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0051] 10 Mobile 20 Fisheye Camera 30 Container 40 wheels 50 Motor 100 Control device 110 Peripheral image acquisition unit 120 Gesture detector 130 Base trajectory calculation unit 140 Coordinate conversion section 150 Risk Calculation Department 160 Travelable space detection unit 170 Trajectory Calculation Unit 180 Driving control unit

Claims

1. a surrounding image acquisition unit that acquires a surrounding image of the moving object, the surrounding image being an image captured by a fisheye camera mounted on the moving object; a base trajectory calculation unit that calculates an instruction regarding a future travel of the moving object as a base trajectory in a Cartesian coordinate system; a coordinate conversion unit that converts the acquired base trajectory in the orthogonal coordinate system into a base trajectory in a fisheye camera coordinate system; a risk calculation unit that calculates a risk of a base trajectory in the fish-eye camera coordinate system based on the peripheral image and a base trajectory in the fish-eye camera coordinate system; a travel trajectory calculation unit that calculates a travel trajectory by correcting a base trajectory in the orthogonal coordinate system based on a risk of a base trajectory in the fisheye camera coordinate system; Equipped with the travel trajectory calculation unit varies a correction amount of a trajectory point on a base trajectory in the orthogonal coordinate system depending on a magnitude of a risk at a corresponding trajectory point of the base trajectory in the fisheye camera coordinate system; Control device.

2. The coordinate conversion unit converts a traveling trajectory in the orthogonal coordinate system into a traveling trajectory in the fisheye camera coordinate system, the risk calculation unit calculates a risk of a driving trajectory in the fisheye camera coordinate system based on the peripheral image and the driving trajectory in the fisheye camera coordinate system; The travel trajectory calculation unit recalculates the travel trajectory by correcting the travel trajectory in the orthogonal coordinate system based on a risk of the travel trajectory in the fisheye camera coordinate system. The control device according to claim 1 .

3. A travel control unit that causes the moving body to travel along the base track or the travel track, When the risk calculation unit determines that the risk of the base trajectory or the traveling trajectory is equal to or less than a threshold, the traveling control unit causes the moving body to travel along the base trajectory or the traveling trajectory. The control device according to claim 1 or 2.

4. A gesture detection unit detects a body movement indicating an instruction by a user regarding a future travel of the moving object, the base trajectory calculation unit calculates a base trajectory in the orthogonal coordinate system based on the body movement detected by the gesture detection unit. The control device according to any one of claims 1 to 3.

5. A computer installed in a vehicle Acquire an image of the surroundings of the moving object, the image being an image captured by a fisheye camera mounted on the moving object; Obtaining an instruction regarding future travel of the moving object as a base trajectory in a Cartesian coordinate system; The base trajectory in the acquired orthogonal coordinate system is coordinate-transformed into a base trajectory in a fish-eye camera coordinate system; Calculating a risk of a base trajectory in the fish-eye camera coordinate system based on the peripheral image and the base trajectory in the fish-eye camera coordinate system; Calculating a travel trajectory by correcting a base trajectory in the orthogonal coordinate system based on a risk of a base trajectory in the fisheye camera coordinate system; Control methods.

6. A computer installed in a moving vehicle Acquiring an image of the surroundings of the moving object, the image being an image captured by a fisheye camera mounted on the moving object; obtaining an instruction for a future travel of the moving object as a base trajectory in a Cartesian coordinate system; The base trajectory in the acquired orthogonal coordinate system is coordinate-transformed into a base trajectory in a fish-eye camera coordinate system; Calculating a risk of a base trajectory in the fish-eye camera coordinate system based on the peripheral image and the base trajectory in the fish-eye camera coordinate system; Calculating a travel trajectory by correcting a base trajectory in the orthogonal coordinate system based on a risk of a base trajectory in the fisheye camera coordinate system; program.

Images