Control device, control method, and program

Abstract

To appropriately evaluate the risk of a travel orbit while reducing processing loads.SOLUTION: A control device includes: a surrounding image acquisition unit that acquires a surrounding image photographed with a fish eye camera loaded on a mobile body; a base orbit calculation unit that calculates an instruction related to future travel of the mobile body as a base orbit in an orthogonal coordinate system; a coordinate conversion unit that converts the base orbit in the orthogonal coordinate system acquired into a base orbit in a fish eye camera coordinate system; a risk calculation unit that calculates a risk of the base orbit in the fish eye camera coordinate system based on the surrounding image and the base orbit in the fish eye camera coordinate system; and a travel orbit calculation unit that calculates a travel orbit by correcting the base orbit in the orthogonal coordinate system based on the risk of the base orbit in the fish eye camera coordinate system. The travel orbit calculation unit provides different amounts of correction of an orbit point on the base orbit in the orthogonal coordinate system in accordance with the degree of the risk at an orbit point corresponding to the base orbit in the fish eye camera coordinate system.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control device, a control method, and a program. [Background technology]

[0002] A technology for equipping autonomously mobile robots with fisheye cameras is known. For example, Patent Document 1 discloses a technology for equipping autonomously mobile robots with fisheye cameras and calculating the robot's trajectory based on images captured by the fisheye cameras. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2004-303137 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] The technology described in Patent Document 1 involves setting up markers to indicate specific locations and using the relative distance and direction between the markers and the robot to guide it. However, setting up markers for robot movement is time-consuming.

[0005] Furthermore, conventional technology involves transforming the coordinates of all points in an image captured by a fisheye camera from the fisheye camera coordinate system to a Cartesian coordinate system, evaluating the risk of the travel trajectory based on the transformed coordinates, and then correcting the travel trajectory. Therefore, it is assumed that the processing load is large when transforming the coordinates from the fisheye camera coordinate system to a Cartesian coordinate system.

[0006] This invention has been made in consideration of these circumstances, and one of its objectives is to provide a control device, a control method, and a program that can appropriately evaluate the risks of a travel track while reducing the processing load. [Means for solving the problem]

[0007] The control device, control method, and program according to this invention employ the following configuration. (1) A control device according to one aspect of the present invention includes: a peripheral image acquisition unit that acquires a peripheral image of the moving body, which is an image captured by a fisheye camera mounted on the moving body; a base trajectory calculation unit that calculates instructions for the future movement of the moving body as a base trajectory in a Cartesian coordinate system; a coordinate transformation unit that transforms the acquired base trajectory in a Cartesian coordinate system into a base trajectory in a fisheye camera coordinate system; a risk calculation unit that calculates the risk of the base trajectory in the fisheye camera coordinate system based on the peripheral image and the base trajectory in the fisheye camera coordinate system; and a travel trajectory calculation unit that calculates a travel trajectory by correcting the base trajectory in a Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system, wherein the travel trajectory calculation unit makes the amount of correction of the trajectory points on the base trajectory in the Cartesian coordinate system different according to the magnitude of the risk at the corresponding trajectory point on the base trajectory in the fisheye camera coordinate system.

[0008] (2): In the embodiment of (1) above, the coordinate transformation unit transforms the trajectory in the Cartesian coordinate system to the trajectory in the fisheye camera coordinate system, the risk calculation unit calculates the risk of the trajectory in the fisheye camera coordinate system based on the surrounding image and the trajectory in the fisheye camera coordinate system, and the trajectory calculation unit recalculates the trajectory by correcting the trajectory in the Cartesian coordinate system based on the risk of the trajectory in the fisheye camera coordinate system.

[0009] (3) In the embodiment of (1) or (2) above, the system further comprises a running control unit that runs the mobile body along the base track or running track, wherein the running control unit runs the mobile body along the base track or running track when the risk calculation unit determines that the risk of the base track or running track is below a threshold.

[0010] (4): In any embodiment of (1) to (3) above, the system further comprises a gesture detection unit that detects a body movement indicating a user's instruction regarding the future movement of the moving body, and the base trajectory calculation unit calculates the base trajectory in the Cartesian coordinate system based on the body movement detected by the gesture detection unit.

[0011] (5) A control method according to another aspect of the present invention involves a computer mounted on a mobile body acquiring a peripheral image of the mobile body, which is an image captured by a fisheye camera mounted on the mobile body; acquiring instructions for the future movement of the mobile body as a base trajectory in a Cartesian coordinate system; transforming the acquired base trajectory in the Cartesian coordinate system into a base trajectory in a fisheye camera coordinate system; calculating the risk of the base trajectory in the fisheye camera coordinate system based on the peripheral image and the base trajectory in the fisheye camera coordinate system; and calculating the travel trajectory by modifying the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system.

[0012] (6): Another aspect of the present invention involves a program that causes a computer mounted on a mobile body to acquire an image of the surrounding area of ​​the mobile body, which is an image captured by a fisheye camera mounted on the mobile body; to acquire instructions for the future movement of the mobile body as a base trajectory in a Cartesian coordinate system; to perform a coordinate transformation of the acquired base trajectory in a Cartesian coordinate system to a base trajectory in a fisheye camera coordinate system; to calculate the risk of the base trajectory in the fisheye camera coordinate system based on the surrounding image and the base trajectory in the fisheye camera coordinate system; and to calculate the travel trajectory by modifying the base trajectory in a Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system. [Effects of the Invention]

[0013] According to the embodiments of (1) to (6), the risks of the train track can be appropriately evaluated while reducing the processing load. [Brief explanation of the drawing]

[0014] [Figure 1] This is a diagram showing an example of a scenario in which a mobile body 10 equipped with a control device according to an embodiment is used. [Figure 2] This is a diagram for explaining an example of the overall configuration of the mobile body 10. [Figure 3] This is a diagram showing an example of a gesture indicated by a user U. [Figure 4] This is a diagram showing an example of coordinate conversion of a base trajectory BL executed by a coordinate conversion unit 140. [Figure 5] This is a diagram showing an example of a scenario in which a risk calculation unit 150 calculates the risk of a base trajectory BL-2 in a fisheye camera coordinate system. [Figure 6] This is a diagram showing an example of a detection process of a drivable space FS by a drivable space detection unit 160. [Figure 7] This is a diagram showing an example of a scenario in which a travel trajectory calculation unit 170 calculates a travel trajectory TL-1 in a rectangular coordinate system. [Figure 8] This is a diagram showing an example of a calculation process of a travel trajectory TL by a travel trajectory calculation unit 170 and a coordinate conversion of the travel trajectory TL by a coordinate conversion unit 140. [Figure 9] This is a flowchart showing an example of a processing flow executed by a control device 100.

Mode for Carrying Out the Invention

[0015] <Embodiment> Hereinafter, a control device, a control method, and a program according to an embodiment of the present invention will be described with reference to the drawings.

[0016] [Overall Configuration] Figure 1 shows an example of a scenario in which a mobile body 10 equipped with a control device according to an embodiment is used. The mobile body 10 is an autonomous mobile robot and is equipped with a fisheye camera 20, a storage container 30, and wheels 40. The mobile body 10 can be used in the following ways: User U is holding luggage B, and the mobile body 10 moves toward user U to store luggage B in response to the user's gesture. Since there is an obstacle OB between the mobile body 10 and user U, the mobile body 10 autonomously avoids the obstacle OB and moves toward the vicinity of user U. After user U places luggage B in the storage container 30, the mobile body 10 moves along with user U.

[0017] The fisheye camera 20 is, for example, a camera that includes a fisheye lens and is capable of capturing a wide-angle (e.g., 360 degrees) image of the area around the moving object 10. The fisheye camera 20 is, for example, mounted on the top of the moving object 10 to capture a wide-angle image of the area around the moving object 10 in the horizontal direction. The fisheye camera 20 may also be realized by combining multiple 120-degree cameras or 60-degree cameras.

[0018] The container 30 is a container for storing any items or luggage and is fixed to the body of the mobile body 10.

[0019] The wheels 40 are driven by multiple motors 50 mounted inside the mobile body 10, enabling movement by the mobile body 10. The wheels 40 include, for example, drive wheels driven in the rotational direction by the motors 50 and steering wheels, which are non-drive wheels driven in the yaw direction. By adjusting the angle of the steering wheels, the mobile body 10 can change its course.

[0020] It should be noted that in this invention, it is not essential that the mobile body 10 is equipped with a housing 30. Furthermore, in this embodiment, the mobile body 10 is equipped with wheels 40 as a mechanism for achieving movement, but the present invention is not limited to this configuration, and for example, the mobile body 10 may be a multi-legged walking robot.

[0021] Figure 2 is a diagram illustrating an example of the overall configuration of the mobile body 10. In addition to the fisheye camera 20, housing 30, wheels 40, and motor 50 described above, the mobile body 10 includes a control device 100. The control device 100 includes, for example, a surrounding image acquisition unit 110, a gesture detection unit 120, a base trajectory calculation unit 130, a coordinate transformation unit 140, a risk calculation unit 150, a drivable space detection unit 160, a trajectory calculation unit 170, and a driving control unit 180. Each part of the control device 100 is realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of the components of the control device 100 may be implemented by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device equipped with a non-transient storage medium), or it may be stored in a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed when the storage medium is inserted into a drive device.

[0022] The peripheral image acquisition unit 110 acquires images captured by the fisheye camera 20 (hereinafter referred to as "peripheral images"). Since the acquired peripheral images are 360-degree images, the peripheral image acquisition unit 110 stores the acquired peripheral images as pixel data in the fisheye camera coordinate system.

[0023] The gesture detection unit 120 detects a physical movement (hereinafter referred to as "gesture") by the user U based on one or more surrounding images. In this embodiment, the gesture is pre-set to indicate an instruction regarding the future movement of the mobile body 10. The gesture detection unit 120 stores data (template data) related to multiple gestures in advance and detects the gesture in the surrounding image by matching the feature data acquired from the surrounding image with the template data. The feature data and template data, for example, represent the characteristic parts of a person's skeleton, such as fingertips, finger joints, wrists, and elbows, and the links connecting them, using abstract data.

[0024] Figure 3 shows an example of a gesture made by user U. In Figure 3, user U extends their right hand forward. In this embodiment, the gesture of extending the right hand forward means "move forward towards user U." In addition, the gesture detection unit 120 detects various gestures that have meanings such as rotation, stopping, and moving backward, but the details of these will not be explained.

[0025] In this embodiment, the mobile body 10 has a gesture detection unit 120 that detects the gestures of user U, but other configurations may be used as long as they can recognize instructions regarding the future movement of the mobile body 10. For example, the mobile body 10 may be equipped with a voice recognition unit that recognizes voices emitted by user U, and recognize instructions regarding the future movement of the mobile body 10 by recognizing that user U has given instructions regarding the future movement of the mobile body 10 by voice (for example, "Come"). Furthermore, as another example, user U may transmit information indicating instructions regarding the future movement of the mobile body 10 to the mobile body 10 using a dedicated application from an information processing terminal such as a smartphone owned by the user U, and the mobile body 10 may recognize the instructions by receiving said information.

[0026] The base trajectory calculation unit 130 calculates the base trajectory BL, which is initial information about the trajectory that the mobile body 10 should travel in the future, based on the gesture detected by the gesture detection unit 120. At this time, the base trajectory calculation unit 130 calculates the base trajectory BL in a Cartesian coordinate system. A Cartesian coordinate system is a coordinate system in which the position of the mobile body 10 is the origin and axes are arbitrarily set in mutually orthogonal directions, fixed to the body of the mobile body 10. For example, if user U makes a gesture of extending their right hand forward, the base trajectory calculation unit 130 calculates a base trajectory BL-1 starting from the position of the mobile body 10, with the initial movement vector matching the direction of the mobile body 10, and ending at the position immediately in front of user U. The base trajectory BL-1 may be calculated so that the front of the mobile body 10 (the side with the housing 30) faces user U at the position immediately in front of user U. The base trajectory BL-1 is calculated by fitting the state to a geometric model such as a Bézier curve. The base trajectory is calculated without considering, for example, the presence of an obstacle OB. The base orbit BL is actually generated as a finite collection of orbital points.

[0027] The coordinate transformation unit 140 performs coordinate transformations between the Cartesian coordinate system and the fisheye camera coordinate system. A one-to-one relationship exists between the coordinates in the Cartesian coordinate system and the fisheye camera coordinate system, and this relationship is stored in the coordinate transformation unit 140 as correspondence information. Hereafter, information in the Cartesian coordinate system will be represented by the symbol "-1", and information in the fisheye camera coordinate system will be represented by the symbol "-2". If there is no distinction between the two, "-1" or "-2" will be omitted. The coordinate transformation unit 140 transforms the base trajectory BL-1 in the Cartesian coordinate system, calculated by the base trajectory calculation unit 130, to the base trajectory BL-2 in the fisheye camera coordinate system. Furthermore, as will be described later, the coordinate transformation unit 140 also transforms the travel trajectory TL-1 in the Cartesian coordinate system, calculated by the travel trajectory calculation unit 170, to the travel trajectory TL-2 in the fisheye camera coordinate system.

[0028] Here, the running track TL refers to the track obtained by modifying the base track BL, and as will be explained later, it is calculated, for example, when it is determined that there is a high risk in running on the base track BL due to the presence of an obstacle OB.

[0029] Figure 4 shows an example of the coordinate transformation of the base trajectory BL performed by the coordinate transformation unit 140. The upper left of Figure 4 shows the base trajectory BL-1 calculated by the base trajectory calculation unit 130, and the lower left of Figure 4 shows the surrounding image of the moving object 10 acquired by the surrounding image acquisition unit 110. The coordinate transformation unit 140 obtains the base trajectory BL-2 in the fisheye camera coordinate system by transforming the base trajectory BL-1 into the fisheye camera coordinate system, and makes it possible to calculate the risk by superimposing the base trajectory BL-2 onto the surrounding image.

[0030] The risk calculation unit 150 calculates the risk for each coordinate in the fisheye camera coordinate system and determines the risk distribution in the target area of ​​the fisheye camera coordinate system. Risk is an index value indicating the likelihood of the moving object 10 colliding with other obstacles. The risk calculation unit 150 uses this risk distribution (risk function) to calculate the risk of the base trajectory BL-2 or the travel trajectory TL-2. For example, the risk calculation unit 150 calculates the risk Ji at each trajectory point on the base trajectory BL or travel trajectory TL of the moving object 10, and calculates the risk J of the base trajectory BL-2 or travel trajectory TL-2 by finding the sum ΣJi of the risks Ji. That is, J = ΣJi holds.

[0031] The risk at a given point on the base track BL-2 or the travel track TL-2 can be calculated based on the distance between that point and the obstacle, as well as the speed of the obstacle. For example, the risk calculation unit 150 calculates a higher risk at a given point the smaller the distance between that point and the obstacle on the base track BL-2 or the travel track TL-2, and a lower risk at that point the larger the distance between that point and the obstacle on the base track BL-2 or the travel track TL-2. More specifically, the risk calculation unit 150 sets the risk within the region of the smallest circumscribed circle encompassing the obstacle OB to a predetermined value (e.g., 1), and calculates the risk Ji such that it decreases as you move away from that region and becomes zero at a certain point. Furthermore, since the risk of a moving obstacle OB differs at each future point in time, risk distribution data is prepared for each time the moving object 10 will reach on the travel track TL.

[0032] Figure 5 shows an example of how the risk calculation unit 150 calculates the risk of the base orbit BL-2 in the fisheye camera coordinate system. Figure 5 shows the risk calculation unit 150 calculating the risk of the base orbit BL-2 in the fisheye camera coordinate system acquired by the coordinate transformation unit 140 in Figure 4. In Figure 5, the hatched region RR represents the risk region where the risk is positive. In the figure, P1 to P4 are orbital points that constitute the base orbit BL-2. The risk calculation unit 150 substitutes the positions of these orbital points P1 to P4 into the risk function for calculating the risk of each orbital point, and calculates the risk value J at each orbital point. P1 , J P2 , J P3 , and J P4 Next, the risk calculation unit 150 takes the sum of the risk values ​​at each orbit point and calculates the risk ΣJ = J for the base orbit BL. P1 +J P2 +J P3 +J P4 This is calculated. This allows the risk calculation unit 150 to calculate the risk of the base track BL-2. The method by which the risk calculation unit 150 calculates the risk of the running track TL-2 is similar.

[0033] When the risk calculation unit 150 calculates the risk ΣJ of the base trajectory BL-2 in the fisheye camera coordinate system, it then compares the calculated risk ΣJ with a threshold Th (for example, 1) to determine whether to drive the moving body 10 along the base trajectory BL. If the risk calculation unit 150 determines that the risk ΣJ is within the threshold Th, it determines to drive the moving body 10 along the base trajectory BL. On the other hand, if the risk calculation unit 150 determines that the risk ΣJ is greater than the threshold Th, it determines not to drive the moving body 10 along the base trajectory BL.

[0034] In this embodiment, the risk calculation unit 150 compares the risk ΣJ with the threshold Th. However, the method of comparing the risk ΣJ with the threshold Th is not limited to this configuration. For example, in FIG. 5, the risk calculation unit 150 may extract the maximum value of each risk value J P1 、J P2 、J P3 、and J P4 on the base trajectory BL-2 (in the case of FIG. 5, J P3 ) and compare the maximum value with the threshold Th. The method of comparing the risk ΣJ with the threshold Th is advantageous in that the moving body 10 can travel more safely, and the method of comparing the maximum value with the threshold Th is advantageous in that the moving body 10 can travel more efficiently.

[0035] When the risk calculation unit 150 determines not to drive the moving body 10 along the base trajectory BL, the drivable space detection unit 160 detects, in the fisheye camera coordinate system, a drivable space FS-2 that is a space where the moving body 10 can travel based on the surrounding image.

[0036] Figure 6 shows an example of the detection process for the drivable space FS by the drivable space detection unit 160. As shown in Figure 6, the risk calculation unit 150 determined that the mobile body 10 should not travel along the base track BL, so the drivable space detection unit 160 detects the space indicated by the shaded area, that is, the space extending from the starting point (position of the mobile body 10) to the ending point (position directly in front of the user U) of the base track BL-2, and excluding obstacles (obstacle OB and user U), as the drivable space FS-2.

[0037] The trajectory calculation unit 170 calculates the correction amount for each trajectory point in the Cartesian coordinate system, based on the risk of each trajectory point calculated by the risk calculation unit 150, so that the trajectory point is within the range of the trajectory space FS-2 detected by the trajectory space detection unit 160. Here, the correction amount for each trajectory point in the Cartesian coordinate system is made by increasing the correction width Δ in the direction perpendicular to the tangent of the trajectory point, as the risk value Ji of the corresponding trajectory point in the fisheye camera coordinate system increases. In other words, the trajectory calculation unit 170 determines the trajectory TL-1 by changing the trajectory points of the base trajectory BL-1 by a correction width Δ corresponding to each risk value Ji. In this way, the trajectory calculation unit 170 can reduce the amount of computation required for trajectory correction because it calculates the correction amount for the corresponding trajectory point in the Cartesian coordinate system based on the risk of each trajectory point in the fisheye camera coordinate system, without transforming the image captured by the fisheye camera 20 into a Cartesian coordinate system.

[0038] Figure 7 shows an example of the trajectory calculation unit 170 calculating the trajectory TL-1 in a Cartesian coordinate system. In Figure 7, trajectory points Q1, Q2, Q3, and Q4 are trajectory points obtained by changing each trajectory point P1, P2, P3, and P4 of the base trajectory BL-1 by a trajectory correction parameter Δ corresponding to their respective risk values ​​Ji. In Figure 7, the risk values ​​Ji of trajectory points P1, P2, P3, and P4 are J P3 , J P2 , J P4 , J P1Because they increase in that order, the correction amount Δ in the direction perpendicular to the tangent line of each orbital point also increases in the order of P3, P2, P4, and P1.

[0039] Next, the coordinate transformation unit 140 transforms the travel trajectory TL-1 in the Cartesian coordinate system into the travel trajectory TL-2 in the fisheye camera coordinate system, and the risk calculation unit 150 substitutes the positions of the trajectory points Q1 to Q4 on the travel trajectory TL-2 into the risk function and calculates the risk value J at each trajectory point. Q1 , J Q2 , J Q3 , and J Q4 The risk distribution (risk function) used in this calculation is the same as the one used to determine the risk of the base orbit BL-2. Next, the risk calculation unit 150 takes the sum of the risk values ​​at each orbit point and calculates the risk of the running orbit TL-1 ΣJ = J Q1 +J Q2 +J Q3 +J Q4 The calculation is performed. In the case of Figure 7, since the trajectory points Q1 to Q4 are outside the risk area RR of the obstacle OB, the risk calculation unit 150 calculates ΣJ = J. Q1 +J Q2 +J Q3 +J Q4 We get =0.

[0040] The risk calculation unit 150 calculates the risk ΣJ of the travel trajectory TL-1 in the fisheye camera coordinate system. Next, it compares the calculated risk ΣJ with a threshold Th and determines whether or not to allow the mobile body 10 to travel along the travel trajectory TL. If the risk calculation unit 150 determines that the risk ΣJ is within the threshold Th, it decides to allow the mobile body 10 to travel along the travel trajectory TL. On the other hand, if the risk calculation unit 150 determines that the risk ΣJ is greater than the threshold Th, it decides not to allow the mobile body 10 to travel along the travel trajectory TL, and repeats the same correction until the risk ΣJ is less than or equal to the threshold Th, and recalculates the travel trajectory TL. In the case of Figure 7, the risk ΣJ = 0, which is less than the threshold Th, so the risk calculation unit 150 decides to allow the mobile body 10 to travel along the travel trajectory TL.

[0041] In the above explanation, the correction range Δ for the trajectory points was assumed to be a value corresponding to each risk value Ji. However, in this case, the trajectory calculation unit 170 may multiply the trajectory correction parameter Δ by a random number to avoid the trajectory TL-1 being a local solution of the total risk value ΣJ.

[0042] Figure 8 shows an example of the calculation process of the travel trajectory TL by the travel trajectory calculation unit 170 and the coordinate transformation of the travel trajectory TL by the coordinate transformation unit 140. As described above, the travel trajectory calculation unit 170 calculates the travel trajectory TL-1 by changing the corresponding trajectory point of the base trajectory BL-1 by the trajectory correction parameter Δ using the correction amount in the Cartesian coordinate system, which is calculated based on the risk of each trajectory point in the fisheye camera coordinate system. Next, the coordinate transformation unit 140 transforms the coordinates of the travel trajectory TL-1 in the Cartesian coordinate system to the travel trajectory TL-2 in the fisheye camera coordinate system, and the risk calculation unit 150 calculates the risk of the travel trajectory TL-2 based on the surrounding image and the travel trajectory TL-2 in the fisheye camera coordinate system. In this way, by calculating the correction amount of the corresponding trajectory point in the Cartesian coordinate system based on the risk of each trajectory point in the fisheye camera coordinate system, the risk of the travel trajectory can be appropriately evaluated while reducing the processing load required for correction.

[0043] If the risk calculation unit 150 determines that the risk value of the base track BL or the travel track TL is less than or equal to a threshold Th, the travel control unit 180 causes the mobile body 10 to travel along the base track BL or the travel track TL. Specifically, the travel control unit 180 outputs a command value to the motor 50 such that the mobile body 10 travels along the base track BL or the travel track TL, and the motor 50 rotates the wheels 40 according to the command value.

[0044] [Process Flow] Next, with reference to Figure 9, the processing flow by the control device 100 according to this embodiment will be described. Figure 9 is a flowchart showing an example of the processing flow executed by the control device 100. The processing in this flowchart is executed at predetermined control cycles.

[0045] First, the peripheral image acquisition unit 110 acquires a peripheral image of the moving object 10 captured by the fisheye camera 20 (step S100). Next, the gesture detection unit 120 detects a gesture indicating an instruction regarding the future movement of the moving object 10 based on the peripheral image acquired by the peripheral image acquisition unit 110 (step S101). Next, the base trajectory calculation unit 130 calculates the base trajectory BL-1 in the Cartesian coordinate system based on the gesture detected by the gesture detection unit 120 (step S102). Next, the coordinate transformation unit 140 transforms the base trajectory BL-1 in the Cartesian coordinate system, calculated by the base trajectory calculation unit 130, into a base trajectory BL-2 in the fisheye camera coordinate system (step S103). Next, the risk calculation unit 150 calculates the risk of the base trajectory BL-2 in the fisheye camera coordinate system based on the peripheral image and the base trajectory BL-2 in the fisheye camera coordinate system (step S104).

[0046] Next, the risk calculation unit 150 determines whether the calculated risk of the base trajectory BL-2 is less than or equal to the threshold Th (step S105). If the risk calculation unit 150 determines that the calculated risk of the base trajectory BL-2 is less than or equal to the threshold Th, the travel control unit 180 travels the mobile body 10 along the base trajectory BL (step S106). On the other hand, if the risk calculation unit 150 determines that the calculated risk of the base trajectory BL-2 is greater than the threshold Th, the drivable space detection unit 160 detects the drivable space FS-2 in the fisheye camera coordinate system based on the surrounding image (step S107). Next, the travel trajectory calculation unit 170 calculates the correction amount for each trajectory point that is within the range of the drivable space FS-2 detected by the drivable space detection unit 160, based on the risk of each trajectory point of the base trajectory BL-2 calculated by the risk calculation unit 150, as a correction amount in the Cartesian coordinate system (step S108). Next, the trajectory calculation unit 170 calculates the trajectory TL-1 by correcting the base trajectory BL-1 by the correction amount in the Cartesian coordinate system (step S109). Next, the coordinate transformation unit 140 transforms the trajectory TL-1 in the Cartesian coordinate system to the trajectory TL-2 in the fisheye camera coordinate system (step S110). Next, the risk calculation unit 150 calculates the risk of the trajectory TL-2 based on the surrounding image and the trajectory TL-2 in the fisheye camera coordinate system (step S111).

[0047] Next, the risk calculation unit 150 determines whether the calculated risk of the travel trajectory TL-2 is less than or equal to the threshold Th (step S112). If the risk calculation unit 150 determines that the calculated risk of the travel trajectory TL-2 is less than or equal to the threshold Th, the travel control unit 180 makes the mobile body 10 travel along the said travel trajectory TL (step S113). On the other hand, if the risk calculation unit 150 determines that the calculated risk of the travel trajectory TL-2 is greater than the threshold Th, the process returns to step S109 and recalculates the travel trajectory TL-1.

[0048] As described above, according to the embodiment of the present invention, the control device calculates the amount of correction to the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system, without converting the image captured by the fisheye camera to a Cartesian coordinate system. The control device then converts the trajectory obtained by correcting the base trajectory in the Cartesian coordinate system by that amount back to the fisheye camera coordinate system, and evaluates the risk of the trajectory in the fisheye camera coordinate system. This makes it possible to appropriately evaluate the risk of the trajectory while reducing the processing load.

[0049] The embodiments described above can be expressed as follows. A memory device that stores the program, Equipped with a hardware processor, The hardware processor executes a program stored in the memory device to acquire an image of the surrounding area of ​​the mobile body, which is an image captured by a fisheye camera mounted on the mobile body. The instructions for the future movement of the aforementioned moving body are calculated as a base trajectory in a Cartesian coordinate system. The base orthogonal coordinate system obtained above is transformed into a base orthogonal coordinate system in the fisheye camera coordinate system. Based on the surrounding image and the base trajectory in the fisheye camera coordinate system, the risk of the base trajectory in the fisheye camera coordinate system is calculated. The travel trajectory is calculated by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system. A control device configured in such a way.

[0050] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0051] 10 Mobile Units 20 Fisheye Cameras 30 containers 40 wheels 50 motors 100 Control device 110 Peripheral image acquisition unit 120 Gesture detection unit 130 Base Orbit Calculation Unit 140 Coordinate Transformation Unit 150 Risk Calculation Department 160 Driving space detection unit 170 Track Calculation Unit 180 Driving Control Unit

Claims

1. A peripheral image acquisition unit acquires images of the surrounding area of ​​the moving object, which are images captured by a fisheye camera mounted on the moving object. A base trajectory calculation unit that calculates instructions regarding the future movement of the aforementioned moving body as a base trajectory in a Cartesian coordinate system, A coordinate transformation unit that transforms the calculated base trajectory in the Cartesian coordinate system to the base trajectory in the fisheye camera coordinate system, A risk calculation unit that calculates the risk of the base trajectory in the fisheye camera coordinate system based on the surrounding image and the base trajectory in the fisheye camera coordinate system, A trajectory calculation unit that calculates the travel trajectory by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system, Equipped with, The trajectory calculation unit adjusts the amount of correction for the trajectory points on the base trajectory in the Cartesian coordinate system according to the magnitude of the risk at the corresponding trajectory points on the base trajectory in the fisheye camera coordinate system. Control device.

2. The coordinate transformation unit transforms the travel trajectory in the Cartesian coordinate system to the travel trajectory in the fisheye camera coordinate system. The risk calculation unit calculates the risk of the travel trajectory in the fisheye camera coordinate system based on the surrounding image and the travel trajectory in the fisheye camera coordinate system. The trajectory calculation unit recalculates the trajectory by correcting the trajectory in the Cartesian coordinate system based on the risk of the trajectory in the fisheye camera coordinate system. The control device according to claim 1.

3. The system further comprises a travel control unit that moves the mobile body along the base track or travel track, If the risk calculation unit determines that the risk of the base track or the travel track is below a threshold, the travel control unit will move the mobile body along the base track or the travel track. The control device according to claim 1 or 2.

4. The system further includes a gesture detection unit that detects body movements indicating instructions from the user regarding the future movement of the mobile object, The base trajectory calculation unit calculates the base trajectory in the Cartesian coordinate system based on the body movements detected by the gesture detection unit. The control device according to any one of claims 1 to 3.

5. The computer mounted on the mobile vehicle, The surrounding image of the moving object is obtained, which is an image captured by a fisheye camera mounted on the moving object. Instructions regarding the future movement of the aforementioned moving object are obtained as a base trajectory in a Cartesian coordinate system. The base orthogonal coordinate system calculated above is transformed into a base orthogonal coordinate system in the fisheye camera coordinate system. Based on the surrounding image and the base trajectory in the fisheye camera coordinate system, the risk of the base trajectory in the fisheye camera coordinate system is calculated. The travel trajectory is calculated by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system. Control method.

6. The computer mounted on the mobile vehicle, The surrounding image of the moving object is obtained by a fisheye camera mounted on the moving object. Instructions regarding the future movement of the aforementioned moving body are acquired as a base trajectory in a Cartesian coordinate system. The base orbit in the Cartesian coordinate system calculated above is transformed into the base orbit in the fisheye camera coordinate system. Based on the surrounding image and the base trajectory in the fisheye camera coordinate system, the risk of the base trajectory in the fisheye camera coordinate system is calculated. The travel trajectory is calculated by correcting the base trajectory in the Cartesian coordinate system based on the risk of the base trajectory in the fisheye camera coordinate system. program.

Images