Control device, control method, and program

The control device for autonomous robots with fisheye cameras optimizes navigation by reducing processing load through coordinated coordinate transformations and risk-based trajectory adjustments, enhancing navigation efficiency and safety.

JP7593840B2Active Publication Date: 2024-12-03HONDA MOTOR CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
JP2021042627
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-03-16
Publication Date
2024-12-03
Estimated Expiration
2041-03-16

AI Technical Summary

Technical Problem

Existing technologies for autonomous robots with fisheye cameras face inefficiencies in processing load due to coordinate transformations from the fisheye camera system to a Cartesian system, and the installation of markers for navigation is time-consuming.

Method used

A control device and method that utilizes a fisheye camera to acquire peripheral images, calculates base trajectories in a Cartesian system, transforms and evaluates risks in the fisheye camera system, and adjusts trajectories based on risk assessment to reduce processing load and improve navigation efficiency.

Benefits of technology

Enables effective risk evaluation of robot trajectories while minimizing processing load by coordinating transformations between coordinate systems, allowing safer and more efficient navigation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007593840000001
    Figure 0007593840000001
  • Figure 0007593840000002
    Figure 0007593840000002
  • Figure 0007593840000003
    Figure 0007593840000003
Patent Text Reader

Abstract

To appropriately evaluate the risk of a travelling trajectory while reducing a processing load.SOLUTION: A control device includes: a peripheral image acquisition unit that acquires a peripheral image of a moving body, which is an image captured by a fisheye camera mounted on the moving body; a base trajectory calculation unit that calculates an instruction regarding future travel of the moving body as a base trajectory in an orthogonal coordinate system; a coordinate transformation unit that transforms the acquired base trajectory in the orthogonal coordinate system into a base trajectory in the fisheye camera coordinate system; a risk calculation unit that calculates the risk of the base trajectory in the fisheye camera coordinate system based on the peripheral image and the base trajectory in the fisheye camera coordinate system; and a traveling trajectory calculation unit that calculates a traveling trajectory by correcting the base trajectory in the orthogonal coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to a control device, a control method, and a program. [Background technology]

[0002] There is known a technology for mounting a fisheye camera on an autonomously mobile robot. For example, Patent Document 1 discloses a technology for mounting a fisheye camera on an autonomously mobile robot and calculating the robot's travel trajectory based on images captured by the fisheye camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-303137 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 involves installing markers to indicate specific locations and running the robot using the relative distance and direction between the marker and the robot. However, installing markers to run the robot is time-consuming.

[0005] Furthermore, conventional technology involves converting the coordinates of all points in an image captured by a fisheye camera from the fisheye camera coordinate system to a Cartesian coordinate system, evaluating the risk of the driving trajectory based on the converted coordinates, and correcting the driving trajectory, so it is expected that the processing load required for converting coordinates from the fisheye camera coordinate system to the Cartesian coordinate system is large.

[0006] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a control device, control method, and program that can appropriately evaluate the risks of a driving trajectory while reducing the processing load. [Means for solving the problem]

[0007] The control device, control method, and program according to the present invention employ the following configuration. (1): A control device according to one embodiment of the present invention includes a peripheral image acquisition unit that acquires peripheral images of the moving body, which are images captured by a fisheye camera mounted on the moving body; a base trajectory calculation unit that calculates instructions regarding the future driving of the moving body as a base trajectory in a Cartesian coordinate system; a coordinate conversion unit that converts the acquired base trajectory in the Cartesian coordinate system into a base trajectory in the fisheye camera coordinate system; a risk calculation unit that calculates the risk of the base trajectory in the fisheye camera coordinate system based on the peripheral image and the base trajectory in the fisheye camera coordinate system; and a driving trajectory calculation unit that calculates a driving trajectory by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system.

[0008] (2): In the above aspect (1), the coordinate conversion unit converts the driving trajectory in the Cartesian coordinate system into a driving trajectory in the fisheye camera coordinate system, the risk calculation unit calculates the risk of the driving trajectory in the fisheye camera coordinate system based on the surrounding image and the driving trajectory in the fisheye camera coordinate system, and the driving trajectory calculation unit recalculates the driving trajectory by correcting the driving trajectory in the Cartesian coordinate system based on the risk of the driving trajectory in the fisheye camera coordinate system.

[0009] (3): In the above aspect (1) or (2), a driving control unit is further provided that causes the moving body to run along the base track or running track, and the driving control unit causes the moving body to run along the base track or running track when the risk calculation unit determines that the risk of the base track or running track is below a threshold.

[0010] (4): In any of the above aspects (1) to (3), a gesture detection unit is further provided that detects a body movement by a user indicating an instruction regarding the future travel of the moving body, and the base trajectory calculation unit calculates a base trajectory in the Cartesian coordinate system based on the body movement detected by the gesture detection unit.

[0011] (5): Another aspect of the present invention relates to a control method in which a computer mounted on a moving body acquires a peripheral image of the moving body, which is an image captured by a fisheye camera mounted on the moving body, acquires instructions regarding the future driving of the moving body as a base trajectory in a Cartesian coordinate system, coordinate-transforms the acquired base trajectory in the Cartesian coordinate system into a base trajectory in a fisheye camera coordinate system, calculates the risk of the base trajectory in the fisheye camera coordinate system based on the peripheral image and the base trajectory in the fisheye camera coordinate system, and calculates a driving trajectory by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system.

[0012] (6): Another aspect of the present invention provides a program that causes a computer mounted on a moving body to acquire peripheral images of the moving body, which are images captured by a fisheye camera mounted on the moving body, acquire instructions regarding the future driving of the moving body as a base trajectory in a Cartesian coordinate system, coordinate-transform the acquired base trajectory in the Cartesian coordinate system into a base trajectory in a fisheye camera coordinate system, calculate a risk of the base trajectory in the fisheye camera coordinate system based on the peripheral images and the base trajectory in the fisheye camera coordinate system, and calculate a driving trajectory by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system. [Effects of the Invention]

[0013] According to aspects (1) to (6), the risk of a travel path can be appropriately evaluated while reducing the processing load. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram showing an example of a situation in which a moving body 10 equipped with a control device according to an embodiment is used. [Figure 2] 1 is a diagram for explaining an example of the overall configuration of a moving body 10. FIG. [Figure 3] 10A and 10B are diagrams illustrating examples of gestures performed by a user U. FIG. [Figure 4] 10 is a diagram showing an example of coordinate transformation of a base trajectory BL executed by a coordinate transformation unit 140. FIG. [Figure 5] FIG. 10 is a diagram showing an example of a scene in which the risk calculation unit 150 calculates the risk of the base trajectory BL-2 in the fish-eye camera coordinate system. [Figure 6] 10 is a diagram showing an example of a process of detecting a drivable space FS by a drivable space detection unit 160. FIG. [Figure 7] FIG. 10 is a diagram showing an example of a scene in which a running trajectory calculation unit 170 calculates a running trajectory TL-1 in a Cartesian coordinate system. [Figure 8] 10 is a diagram showing an example of a calculation process of a travel trajectory TL by a travel trajectory calculation unit 170 and a coordinate conversion of the travel trajectory TL by a coordinate conversion unit 140. FIG. [Figure 9] 4 is a flowchart showing an example of the flow of processing executed by the control device 100. DETAILED DESCRIPTION OF THE INVENTION

[0015] <Embodiment> Hereinafter, a control device, a control method, and a program according to an embodiment of the present invention will be described with reference to the drawings.

[0016] [Overall configuration] FIG. 1 is a diagram showing an example of a situation in which a moving body 10 equipped with a control device according to an embodiment is used. The moving body 10 is an autonomous mobile robot and includes a fisheye camera 20, a container 30, and wheels 40. The moving body 10 can be used in the following manner. A user U is holding luggage B, and the moving body 10 moves toward the user U in response to a gesture by the user to store the luggage B. Because an obstacle OB exists between the moving body 10 and the user U, the moving body 10 autonomously avoids the obstacle OB and moves to the vicinity of the user U. After the user U places the luggage B in the container 30, the moving body 10 moves along with the user U.

[0017] The fisheye camera 20 is, for example, a camera that includes a fisheye lens and is capable of capturing images of the periphery of the moving object 10 at a wide angle (for example, 360 degrees). The fisheye camera 20 is attached, for example, to the top of the moving object 10, and captures images of the periphery of the moving object 10 at a wide angle in the horizontal direction. The fisheye camera 20 may be realized by combining multiple 120-degree cameras or 60-degree cameras.

[0018] The container 30 is a container for storing any item or luggage, and is fixed to the body of the moving body 10.

[0019] The wheels 40 are driven by a plurality of motors 50 mounted inside the mobile body 10, enabling the mobile body 10 to move. The wheels 40 include, for example, drive wheels that are driven in a rotational direction by the motors 50, and steering wheels that are non-drive wheels that are driven in a yaw direction. By adjusting the angle of the steering wheels, the mobile body 10 can change its course.

[0020] In the present invention, it is not essential that the moving body 10 includes the container 30. Furthermore, in this embodiment, the moving body 10 includes wheels 40 as a mechanism for realizing movement, but the present invention is not limited to this configuration, and the moving body 10 may be, for example, a multi-legged robot.

[0021] 2 is a diagram illustrating an example of the overall configuration of the moving body 10. The moving body 10 includes a control device 100 in addition to the fisheye camera 20, container 30, wheels 40, and motor 50 described above. The control device 100 includes, for example, a surrounding image acquisition unit 110, a gesture detection unit 120, a base trajectory calculation unit 130, a coordinate conversion unit 140, a risk calculation unit 150, a travelable space detection unit 160, a travel trajectory calculation unit 170, and a travel control unit 180. Each unit of the control device 100 is realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of the components of the control device 100 may be implemented by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device.

[0022] The peripheral image acquisition unit 110 acquires an image (hereinafter referred to as a "peripheral image") captured by the fisheye camera 20. Since the peripheral image acquired at this time is a 360-degree image, the peripheral image acquisition unit 110 stores the acquired peripheral image as pixel data in the fisheye camera coordinate system.

[0023] The gesture detection unit 120 detects a body movement (hereinafter referred to as a "gesture") by the user U based on one or more peripheral images. In this embodiment, the gesture is set in advance as an instruction regarding the future travel of the moving object 10. The gesture detection unit 120 stores data (template data) regarding a plurality of gestures in advance, and detects gestures in the peripheral image by matching feature data acquired from the peripheral image with the template data. Each of the feature data and template data is abstract data representing, for example, characteristic parts of a person's skeleton, such as fingertips, finger joints, wrists, and elbows, and the links connecting them.

[0024] 3 is a diagram showing an example of a gesture performed by a user U. In FIG. 3, the user U extends his right hand forward. In this embodiment, the gesture of extending the right hand forward means "please move forward toward the user U." In addition, the gesture detection unit 120 detects various gestures that mean, for example, to rotate, stop, or move backward, but detailed description thereof will be omitted.

[0025] In this embodiment, the moving object 10 includes a gesture detection unit 120 that detects gestures of the user U. However, other configurations may be used as long as they are capable of recognizing instructions regarding the future travel of the moving object 10. For example, the moving object 10 may include a voice recognition unit that recognizes voice uttered by the user U, and recognize an instruction regarding the future travel of the moving object 10 by recognizing that the user U has spoken an instruction (e.g., "come") to the moving object 10, thereby recognizing the instruction. As another example, the user U may use a dedicated application from an information processing terminal such as a smartphone owned by the user U to transmit information indicating instructions regarding the future travel of the moving object 10 to the moving object 10, and the moving object 10 may receive the information and thereby recognize the instruction.

[0026] The base trajectory calculation unit 130 calculates a base trajectory BL, which is initial information about a trajectory on which the moving body 10 will travel in the future, based on the gesture detected by the gesture detection unit 120. At this time, the base trajectory calculation unit 130 calculates the base trajectory BL in a Cartesian coordinate system. The Cartesian coordinate system is a coordinate system that has the position of the moving body 10 as its origin, is fixedly set on the body of the moving body 10, and has axes in arbitrary directions that are perpendicular to each other. For example, when the user U makes a gesture of extending his / her right hand forward, the base trajectory calculation unit 130 calculates a base trajectory BL-1 that starts from the position of the moving body 10, has an initial movement vector that matches the direction of the moving body 10, and ends at a position immediately before the user U. The base trajectory BL-1 may be calculated so that the front of the moving body 10 (the side on which the container 30 is located) faces the user U at the position immediately before the user U. The base trajectory BL-1 is calculated by fitting the state to a geometric model such as a Bezier curve. The base trajectory is calculated without taking into account the presence of an obstacle OB, for example. The base trajectory BL is actually generated as a collection of a finite number of trajectory points.

[0027] The coordinate conversion unit 140 performs coordinate conversion between the Cartesian coordinate system and the fisheye camera coordinate system. A one-to-one relationship exists between the coordinates of the Cartesian coordinate system and the fisheye camera coordinate system, and this relationship is stored in the coordinate conversion unit 140 as correspondence information. Hereinafter, information in the Cartesian coordinate system is represented by the symbol "-1," and information in the fisheye camera coordinate system is represented by the symbol "-2." When there is no need to distinguish between the two, the "-1" and "-2" are omitted. The coordinate conversion unit 140 converts the base trajectory BL-1 in the Cartesian coordinate system calculated by the base trajectory calculation unit 130 into the base trajectory BL-2 in the fisheye camera coordinate system. The coordinate conversion unit 140 further converts the traveling trajectory TL-1 in the Cartesian coordinate system calculated by the traveling trajectory calculation unit 170 into the traveling trajectory TL-2 in the fisheye camera coordinate system, as described below.

[0028] Here, the running trajectory TL means a trajectory obtained by modifying the base trajectory BL, and as will be described later, is calculated when, for example, it is determined that there is a high risk in running on the base trajectory BL due to the presence of an obstacle OB.

[0029] Fig. 4 is a diagram showing an example of coordinate transformation of the base trajectory BL executed by the coordinate transformation unit 140. The upper left part of Fig. 4 shows the base trajectory BL-1 calculated by the base trajectory calculation unit 130, and the lower left part of Fig. 4 shows the peripheral image of the moving object 10 acquired by the peripheral image acquisition unit 110. The coordinate transformation unit 140 acquires the base trajectory BL-2 in the fisheye camera coordinate system by coordinate transformation of the base trajectory BL-1 into the fisheye camera coordinate system, and makes it possible to calculate the risk by superimposing the base trajectory BL-2 on the peripheral image.

[0030] The risk calculation unit 150 calculates the risk of each coordinate in the fisheye camera coordinate system and determines the distribution of risk in the target area of ​​the fisheye camera coordinate system. The risk is an index value that indicates the likelihood that the moving body 10 will collide with another obstacle. The risk calculation unit 150 uses the risk distribution (risk function) to calculate the risk of the base trajectory BL-2 or the traveling trajectory TL-2. For example, the risk calculation unit 150 calculates the risk Ji at each trajectory point on the base trajectory BL or the traveling trajectory TL of the moving body 10, and calculates the risk J of the base trajectory BL-2 or the traveling trajectory TL-2 by calculating the sum ΣJi of the risks Ji. In other words, J = ΣJi holds.

[0031] The risk at a certain trajectory point on the base trajectory BL-2 or the traveling trajectory TL-2 can be calculated based on the distance between the trajectory point and the obstacle and the speed of the obstacle. For example, the risk calculation unit 150 calculates the risk at a certain trajectory point to be higher the closer the distance between the obstacle and a certain trajectory point on the base trajectory BL-2 or the traveling trajectory TL-2 is, and lower the risk at that trajectory point the closer the distance between the obstacle and a certain trajectory point on the base trajectory BL-2 or the traveling trajectory TL-2 is. More specifically, the risk calculation unit 150 sets the risk within the area of ​​the smallest circumscribing circle that contains the obstacle OB to a predetermined value (e.g., 1), and calculates the risk Ji so that it decreases as the distance from that area increases, eventually reaching zero at a certain point. Furthermore, because the risk of a moving obstacle OB varies at each future point, risk distribution data is prepared for each time the moving object 10 arrives on the traveling trajectory TL.

[0032] FIG. 5 is a diagram showing an example of a scene in which the risk calculation unit 150 calculates the risk of the base trajectory BL-2 in the fisheye camera coordinate system. FIG. 5 shows the risk calculation unit 150 calculating the risk of the base trajectory BL-2 in the fisheye camera coordinate system acquired by the coordinate conversion unit 140 in FIG. 4. In FIG. 5, the hatched area RR represents a risk area where the risk is positive. In the figure, P1 to P4 are trajectory points that make up the base trajectory BL-2. The risk calculation unit 150 substitutes the positions of these trajectory points P1 to P4 into a risk function for calculating the risk of each trajectory point, and calculates a risk value J at each trajectory point. P1 , J P2 , J P3 , and J P4 Next, the risk calculation unit 150 calculates the sum of the risk values ​​at each trajectory point, and calculates the risk ΣJ=J of the base trajectory BL. P1 +J P2 +J P3 +J P4 In this way, the risk calculation unit 150 can calculate the risk of the base trajectory BL-2. The risk calculation unit 150 calculates the risk of the running trajectory TL-2 in the same manner.

[0033] After calculating the risk ΣJ of the base trajectory BL-2 in the fisheye camera coordinate system, the risk calculation unit 150 then compares the calculated risk ΣJ with a threshold value Th (e.g., 1) and determines whether or not to cause the moving body 10 to travel along the base trajectory BL. If the risk calculation unit 150 determines that the risk ΣJ is within the threshold value Th, it determines that the moving body 10 will travel along the base trajectory BL. On the other hand, if the risk calculation unit 150 determines that the risk ΣJ is greater than the threshold value Th, it determines that the moving body 10 will not travel along the base trajectory BL.

[0034] In this embodiment, the risk calculation unit 150 compares the risk ΣJ with the threshold value Th. However, the method of comparing the risk ΣJ with the threshold value Th is not limited to this configuration. For example, the risk calculation unit 150 may compare each risk value J on the base trajectory BL-2 in FIG. P1 , J P2 , J P3 , and J P4 The maximum value of (in the case of Figure 5, J P3 ) and compare the maximum value with the threshold value Th. The method of comparing the risk ΣJ with the threshold value Th is advantageous in that it allows the moving object 10 to travel more safely, while the method of comparing the maximum value with the threshold value Th is advantageous in that it allows the moving object 10 to travel more efficiently.

[0035] When the risk calculation unit 150 determines that the moving body 10 will not travel along the base trajectory BL, the travelable space detection unit 160 detects the travelable space FS-2, which is the space in which the moving body 10 can travel, in the fisheye camera coordinate system based on the surrounding image.

[0036] Fig. 6 is a diagram showing an example of the detection process of the travelable space FS by the travelable space detection unit 160. As shown in Fig. 6, the risk calculation unit 150 has determined that the moving object 10 will not travel along the base trajectory BL, and therefore the travelable space detection unit 160 detects the space indicated by the diagonally shaded area, that is, the space that extends from the start point of the base trajectory BL-2 (the position of the moving object 10) in the direction of the end point (the position immediately preceding the user U) and that excludes obstacles (obstacle OB and user U), as the travelable space FS-2.

[0037] Based on the risk of each trajectory point calculated by the risk calculation unit 150, the traveling trajectory calculation unit 170 calculates, as a correction amount in the Cartesian coordinate system, a correction amount for each trajectory point that is within the range of the drivable space FS-2 detected by the drivable space detection unit 160. Here, the correction amount for each trajectory point in the Cartesian coordinate system is calculated by increasing the correction width Δ in the direction perpendicular to the tangent of the trajectory point as the risk value Ji at the corresponding trajectory point in the fisheye camera coordinate system increases. That is, the traveling trajectory calculation unit 170 calculates the traveling trajectory TL-1 by changing the trajectory points on the base trajectory BL-1 by the correction width Δ corresponding to each risk value Ji. In this way, the traveling trajectory calculation unit 170 calculates the correction amount for the corresponding trajectory point in the Cartesian coordinate system based on the risk of each trajectory point in the fisheye camera coordinate system without converting the image captured by the fisheye camera 20 to the Cartesian coordinate system, thereby reducing the amount of calculation required for trajectory correction.

[0038] 7 is a diagram showing an example of a scene in which the running trajectory calculation unit 170 calculates the running trajectory TL-1 in a Cartesian coordinate system. In FIG. 7, the trajectory points Q1, Q2, Q3, and Q4 are trajectory points obtained by changing the trajectory points P1, P2, P3, and P4 of the base trajectory BL-1 by the trajectory correction parameter Δ corresponding to each risk value Ji. In FIG. 7, the risk values ​​Ji of the trajectory points P1, P2, P3, and P4 are calculated by P3 , J P2 , J P4 , J P1Therefore, the correction width Δ in the direction perpendicular to the tangent of each orbit point also increases in the order of P3, P2, P4, and P1.

[0039] Next, the coordinate conversion unit 140 converts the travel trajectory TL-1 in the Cartesian coordinate system into a travel trajectory TL-2 in the fisheye camera coordinate system, and the risk calculation unit 150 substitutes the positions of the trajectory points Q1 to Q4 on the travel trajectory TL-2 into a risk function to calculate a risk value J at each trajectory point. Q1 , J Q2 , J Q3 , and J Q4 The risk distribution (risk function) used in this calculation is the same as that used when calculating the risk of the base trajectory BL-2. Next, the risk calculation unit 150 calculates the sum of the risk values ​​at each trajectory point, and calculates the risk of the running trajectory TL-1, ΣJ=J Q1 +J Q2 +J Q3 +J Q4 In the case of FIG. 7, since the trajectory points Q1 to Q4 are outside the risk region RR of the obstacle OB, the risk calculation unit 150 calculates ΣJ=J Q1 +J Q2 +J Q3 +J Q4 =0 is obtained.

[0040] After calculating the risk ΣJ of the traveling trajectory TL-1 in the fisheye camera coordinate system, the risk calculation unit 150 then compares the calculated risk ΣJ with a threshold Th to determine whether or not to cause the moving object 10 to travel along the traveling trajectory TL. If the risk calculation unit 150 determines that the risk ΣJ is within the threshold Th, it determines that the moving object 10 should travel along the traveling trajectory TL. On the other hand, if the risk calculation unit 150 determines that the risk ΣJ is greater than the threshold Th, it determines that the moving object 10 should not travel along the traveling trajectory TL, and repeats the same correction until the risk ΣJ becomes equal to or less than the threshold Th, and recalculates the traveling trajectory TL. In the case of FIG. 7, the risk ΣJ = 0, which is less than the threshold Th, so the risk calculation unit 150 determines that the moving object 10 should travel along the traveling trajectory TL.

[0041] In the above explanation, the correction width Δ of the trajectory point is assumed to be a value corresponding to each risk value Ji. In this case, the running trajectory calculation unit 170 may multiply the trajectory correction parameter Δ by a random number to avoid the running trajectory TL-1 being taken as a local solution of the total risk value ΣJ.

[0042] FIG. 8 illustrates an example of the calculation process of the travel trajectory TL by the travel trajectory calculation unit 170 and the coordinate transformation of the travel trajectory TL by the coordinate transformation unit 140. As described above, the travel trajectory calculation unit 170 calculates the travel trajectory TL-1 by changing the corresponding trajectory point of the base trajectory BL-1 by the trajectory correction parameter Δ using the correction amount in the Cartesian coordinate system calculated based on the risk of each trajectory point in the fisheye camera coordinate system. Next, the coordinate transformation unit 140 transforms the travel trajectory TL-1 in the Cartesian coordinate system into the travel trajectory TL-2 in the fisheye camera coordinate system, and the risk calculation unit 150 calculates the risk of the travel trajectory TL-2 based on the peripheral image and the travel trajectory TL-2 in the fisheye camera coordinate system. In this way, by calculating the correction amount of the corresponding trajectory point in the Cartesian coordinate system based on the risk of each trajectory point in the fisheye camera coordinate system, the risk of the travel trajectory can be appropriately evaluated while reducing the processing load required for the correction.

[0043] When the risk calculation unit 150 determines that the risk value of the base trajectory BL or the traveling trajectory TL is equal to or less than the threshold value Th, the traveling control unit 180 causes the moving object 10 to travel along the base trajectory BL or the traveling trajectory TL. Specifically, the traveling control unit 180 outputs a command value to the motor 50 so that the moving object 10 travels along the base trajectory BL or the traveling trajectory TL, and the motor 50 rotates the wheels 40 in accordance with the command value.

[0044] [Processing flow] Next, the flow of processing by the control device 100 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the flow of processing executed by the control device 100. The processing of this flowchart is executed at each predetermined control cycle.

[0045] First, the peripheral image acquisition unit 110 acquires a peripheral image of the moving object 10 captured by the fisheye camera 20 (step S100). Next, the gesture detection unit 120 detects a gesture indicating an instruction regarding future travel of the moving object 10 based on the peripheral image acquired by the peripheral image acquisition unit 110 (step S101). Next, the base trajectory calculation unit 130 calculates a base trajectory BL-1 in a Cartesian coordinate system based on the gesture detected by the gesture detection unit 120 (step S102). Next, the coordinate conversion unit 140 converts the base trajectory BL-1 in the Cartesian coordinate system calculated by the base trajectory calculation unit 130 into a base trajectory BL-2 in the fisheye camera coordinate system (step S103). Next, the risk calculation unit 150 calculates a risk of the base trajectory BL-2 in the fisheye camera coordinate system based on the peripheral image and the base trajectory BL-2 in the fisheye camera coordinate system (step S104).

[0046] Next, the risk calculation unit 150 determines whether the risk of the calculated base trajectory BL-2 is equal to or less than the threshold Th (step S105). If the risk calculation unit 150 determines that the risk of the calculated base trajectory BL-2 is equal to or less than the threshold Th, the traveling control unit 180 causes the moving object 10 to travel along the base trajectory BL (step S106). On the other hand, if the risk calculation unit 150 determines that the risk of the calculated base trajectory BL-2 is greater than the threshold Th, the drivable space detection unit 160 detects the drivable space FS-2 in the fisheye camera coordinate system based on the peripheral image (step S107). Next, the traveling trajectory calculation unit 170 calculates, based on the risk of each trajectory point of the base trajectory BL-2 calculated by the risk calculation unit 150, a correction amount for each trajectory point that is within the range of the drivable space FS-2 detected by the drivable space detection unit 160 as a correction amount in the Cartesian coordinate system (step S108). Next, the travel trajectory calculation unit 170 calculates the travel trajectory TL-1 by correcting the base trajectory BL-1 by the correction amount in the Cartesian coordinate system (step S109). Next, the coordinate conversion unit 140 converts the travel trajectory TL-1 in the Cartesian coordinate system into a travel trajectory TL-2 in the fisheye camera coordinate system (step S110). Next, the risk calculation unit 150 calculates the risk of the travel trajectory TL-2 based on the peripheral image and the travel trajectory TL-2 in the fisheye camera coordinate system (step S111).

[0047] Next, the risk calculation unit 150 determines whether the risk of the calculated travel trajectory TL-2 is equal to or less than the threshold Th (step S112). If the risk calculation unit 150 determines that the risk of the calculated travel trajectory TL-2 is equal to or less than the threshold Th, the travel control unit 180 causes the moving object 10 to travel along the travel trajectory TL (step S113). On the other hand, if the risk calculation unit 150 determines that the risk of the calculated travel trajectory TL-2 is greater than the threshold Th, the process returns to step S109 again, and the travel trajectory TL-1 is recalculated.

[0048] As described above, according to an embodiment of the present invention, the control device calculates the amount of correction to the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system without coordinate transformation of the image captured by the fisheye camera into a Cartesian coordinate system, then coordinate transforms the travel trajectory obtained by correcting the base trajectory in the Cartesian coordinate system by the amount of correction into the fisheye camera coordinate system again, and evaluates the risk of the travel trajectory in the fisheye camera coordinate system. This makes it possible to appropriately evaluate the risk of the travel trajectory while reducing the processing load.

[0049] The above-described embodiment can be expressed as follows. a storage device storing a program; a hardware processor; The hardware processor executes the program stored in the storage device to acquire an image of the surroundings of the moving object, the image being an image captured by a fisheye camera mounted on the moving object; calculating instructions for future travel of the moving body as a base trajectory in a Cartesian coordinate system; coordinate-transforming the acquired base trajectory in the Cartesian coordinate system into a base trajectory in the fisheye camera coordinate system; Calculating a risk of a base trajectory in the fisheye camera coordinate system based on the peripheral image and the base trajectory in the fisheye camera coordinate system; Calculating a driving trajectory by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system; The control device is configured as follows.

[0050] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0051] 10 Mobile 20 Fisheye Camera 30 container 40 wheels 50 motor 100 control device 110 Peripheral image acquisition unit 120 Gesture detection unit 130 Base trajectory calculation unit 140 Coordinate conversion unit 150 Risk Calculation Department 160 Drivable space detection unit 170 Trajectory calculation unit 180 Travel control unit

Claims

1. a surrounding image acquisition unit that acquires a surrounding image of the moving object, the surrounding image being an image captured by a fisheye camera mounted on the moving object; a base trajectory calculation unit that calculates an instruction regarding a future travel of the moving object as a base trajectory in a Cartesian coordinate system; a coordinate conversion unit that converts the acquired base trajectory in the orthogonal coordinate system into a base trajectory in a fisheye camera coordinate system; a risk calculation unit that calculates a risk of a base trajectory in the fish-eye camera coordinate system based on the peripheral image and a base trajectory in the fish-eye camera coordinate system; a travel trajectory calculation unit that calculates a travel trajectory by correcting a base trajectory in the orthogonal coordinate system based on a risk of a base trajectory in the fisheye camera coordinate system; Equipped with The coordinate conversion unit converts a traveling trajectory in the orthogonal coordinate system into a traveling trajectory in the fisheye camera coordinate system, Further comprising a travel control unit that causes the moving body to travel along the base track or the travel track. Control device.

2. The risk calculation unit calculates a risk of a driving trajectory in the fisheye camera coordinate system based on the peripheral image and the driving trajectory in the fisheye camera coordinate system, The travel trajectory calculation unit recalculates the travel trajectory by correcting the travel trajectory in the orthogonal coordinate system based on a risk of the travel trajectory in the fisheye camera coordinate system. The control device according to claim 1 .

3. a surrounding image acquisition unit that acquires a surrounding image of the moving object, the surrounding image being an image captured by a fisheye camera mounted on the moving object; a base trajectory calculation unit that calculates an instruction regarding a future travel of the moving object as a base trajectory in a Cartesian coordinate system; a coordinate conversion unit that converts the acquired base trajectory in the orthogonal coordinate system into a base trajectory in a fisheye camera coordinate system; a risk calculation unit that calculates a risk of a base trajectory in the fish-eye camera coordinate system based on the peripheral image and a base trajectory in the fish-eye camera coordinate system; a travel trajectory calculation unit that calculates a travel trajectory by correcting a base trajectory in the orthogonal coordinate system based on a risk of a base trajectory in the fisheye camera coordinate system; Equipped with The coordinate conversion unit converts a traveling trajectory in the orthogonal coordinate system into a traveling trajectory in the fisheye camera coordinate system, the risk calculation unit calculates a risk of a traveling trajectory in the fisheye camera coordinate system based on the peripheral image and the traveling trajectory in the fisheye camera coordinate system; the travel trajectory calculation unit recalculates the travel trajectory by correcting the travel trajectory in the orthogonal coordinate system without correcting the travel trajectory in the fisheye camera coordinate system based on a risk of the travel trajectory in the fisheye camera coordinate system; The coordinate conversion unit converts the traveling trajectory recalculated in the orthogonal coordinate system into a traveling trajectory in the fisheye camera coordinate system, Further comprising a travel control unit that causes the moving body to travel along the base track or the travel track. Control device.

4. A gesture detection unit detects a body movement indicating an instruction by a user regarding a future travel of the moving object, the base trajectory calculation unit calculates a base trajectory in the orthogonal coordinate system based on the body movement detected by the gesture detection unit. The control device according to any one of claims 1 to 3.

5. A computer installed in a vehicle Acquire an image of the surroundings of the moving object, the image being an image captured by a fisheye camera mounted on the moving object; Obtaining an instruction regarding future travel of the moving object as a base trajectory in a Cartesian coordinate system; The base trajectory in the acquired orthogonal coordinate system is coordinate-transformed into a base trajectory in a fish-eye camera coordinate system; Calculating a risk of a base trajectory in the fish-eye camera coordinate system based on the peripheral image and the base trajectory in the fish-eye camera coordinate system; Calculate a travel trajectory by correcting a base trajectory in the orthogonal coordinate system based on a risk of the base trajectory in the fisheye camera coordinate system; A travel trajectory in the orthogonal coordinate system is coordinate-transformed into a travel trajectory in the fish-eye camera coordinate system; running the moving body along the base track or the running track; Control methods.

6. A computer installed in a moving vehicle Acquiring an image of the surroundings of the moving object, the image being an image captured by a fisheye camera mounted on the moving object; obtaining an instruction for a future travel of the moving object as a base trajectory in a Cartesian coordinate system; The base trajectory in the acquired orthogonal coordinate system is coordinate-transformed into a base trajectory in a fish-eye camera coordinate system; Calculating a risk of a base trajectory in the fish-eye camera coordinate system based on the peripheral image and the base trajectory in the fish-eye camera coordinate system; Calculating a travel trajectory by correcting a base trajectory in the orthogonal coordinate system based on a risk of a base trajectory in the fisheye camera coordinate system; A travel trajectory in the orthogonal coordinate system is coordinate-transformed into a travel trajectory in the fish-eye camera coordinate system; running the moving body along the base track or the running track; program.

Citation Information

Patent Citations

  • Vehicle intelligent traveling method, computer readable storage medium and AGV

    CN112130576A

  • Specific position guiding device and specific position guide control method for autonomous traveling type robot

    JP2004303137A

  • Position teaching device and position teaching method to mover

    JP2006051550A

  • Robot control device

    JP2006185438A

  • Teleoperation device, method and program

    JP2017054475A