Display control system, display control method, and program

Abstract

To accurately notify all around an autonomous mobile device of a moving state of the device at and after the present point of time.SOLUTION: A display control system comprises: an autonomous movement control unit configured to perform control for autonomously moving an autonomous mobile device; and a display control unit configured to cause a projection unit to project a video on a movement target surface in front of the autonomous mobile device, the video notifying movement of the autonomous mobile device at and after the present point of time based on the control of the autonomous movement control unit.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a display control system, a display control method, and a program. [Background technology]

[0002] There is known technology in which a rectangular area corresponding to the sensing area of ​​a laser scanner attached to an automatic guided vehicle (AGV) is displayed on a floor surface using an illuminated display device (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-053801 Summary of the Invention [Problem to be solved by the invention]

[0004] In an environment where autonomous mobile devices capable of moving autonomously coexist with humans, in order to achieve smooth cooperation between the two without interfering with each other's progress, it is preferable to be able to accurately notify those around the autonomous mobile device of how it will move from the present time onwards.

[0005] In consideration of the above-mentioned problems, an object of the present invention is to make it possible to accurately notify surrounding parties of the moving status of an autonomous mobile device from the present time onwards. [Means for solving the problem]

[0006] (1) One aspect of the present invention that solves the above-mentioned problems is a display control system that includes an autonomous movement control unit that executes control to autonomously move an autonomous mobile device, and a display control unit that projects, by a projection unit configured to project an image onto a moving target surface in front of the autonomous mobile device, an image informing about movement of the autonomous mobile device from the present time onwards based on the control of the autonomous movement control unit.

[0007] (2) One aspect of the present invention is the display control system described in (1), in which the display control unit may cause the projection unit to project an image indicating a direction in which the autonomous mobile device will move from a current time onwards.

[0008] (3) One aspect of the present invention is the display control system described in (1) or (2), wherein the display control unit may cause the projection unit to project an image indicating a current state of the autonomous mobile device.

[0009] (4) One aspect of the present invention is the display control system described in (3), wherein, when the current state of the autonomous mobile device is that it is temporarily stopped, the display control unit may cause the projection unit to project an image indicating that the autonomous mobile device is temporarily stopped.

[0010] (5) One aspect of the present invention is the display control system described in (4), wherein the display control unit may cause the projection unit to project an image indicating that the current state of the autonomous mobile device is temporarily stopped and that the time until the pause is lifted is within a predetermined period of time.

[0011] (6) One aspect of the present invention is a display control system described in any one of (1) to (5), wherein the autonomous movement control unit sets a movement route to avoid an obstacle upon detection of the obstacle ahead, and the display control unit causes the projection unit to project an image showing the movement route set by the autonomous movement control unit.

[0012] (7) One aspect of the present invention is a display control system described in any one of (1) to (6), wherein the display control unit may change the color of the image in response to detection of an obstacle ahead.

[0013] (8) One aspect of the present invention is a display control method in a display control system, the display control method including an autonomous movement control step in which an autonomous movement control unit executes control to autonomously move an autonomous mobile device, and a display control step in which a display control unit projects an image informing about movement of the autonomous mobile device from a current time onward based on the control of the autonomous movement control unit, using a projection unit configured to project the image onto a moving target surface in front of the autonomous mobile device.

[0014] (9) One aspect of the present invention is a program for causing a computer to function as a display control unit that projects, by a projection unit configured to project an image onto a moving target surface in front of the autonomous mobile device, an image informing about the movement of the autonomous mobile device from a present time onward based on control for autonomously moving the autonomous mobile device. Effect of the Invention

[0015] According to the present invention, it is possible to obtain an effect of being able to accurately notify surrounding people of the moving status of an autonomous mobile device from the present time onwards. [Brief description of the drawings]

[0016] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of an autonomous mobile device according to the present embodiment. [Diagram 2] 10A to 10C are diagrams illustrating examples of display of a notification image by an autonomous mobile device in the present embodiment. [Diagram 3] FIG. 4 is a diagram showing an example of notification video data in the present embodiment. [Figure 4] 10A to 10C are diagrams showing examples of notification images in the present embodiment. [Diagram 5] 10A to 10C are diagrams illustrating an example of how a notification image changes depending on the moving status of an autonomous mobile device in this embodiment. [Figure 6] 10 is a flowchart illustrating an example of a processing procedure executed by an autonomous mobile device in the present embodiment in response to displaying a notification video. [Figure 7] 10 is a flowchart illustrating an example of a processing procedure executed by an autonomous mobile device in the present embodiment in response to route replanning. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] 1 shows an example of a functional configuration of an autonomous mobile device 100 according to this embodiment. The autonomous mobile device 100 may perform autonomous movement (autonomous movement) in various facilities such as offices, factories, museums, hospitals, etc., and may perform, for example, guidance, transport of luggage, etc. The autonomous mobile device 100 of this embodiment includes a computer including a CPU, a GPU, a ROM, a RAM, and an auxiliary storage device such as a HDD or an SDD. The functions of the autonomous mobile device 100 shown in Fig. 1 are realized by the computer included in the autonomous mobile device 100 executing a program.

[0018] 1 includes a sensor unit 101, a movement mechanism unit 102, a projection unit 103, a control unit 104, and a storage unit 105. The autonomous mobile device 100 may be provided with a user interface function including an operation unit and a display unit that correspond to operations performed by an administrator or the like, but this is omitted in the figure.

[0019] The sensor unit 101 includes a predetermined sensor that detects information for detecting surrounding objects and obstacles and estimating the self-position. The sensor unit 101 may include, for example, a two-dimensional sensor. The two-dimensional sensor may be LiDAR (Light Detection and Ranging). The two-dimensional sensor using LiDAR includes an irradiating unit of laser light and a light receiving unit that receives reflected laser light, and measures distance based on the time from when the irradiating unit irradiates the surroundings with pulsed laser light until the light receiving unit receives the reflected light. In the autonomous mobile device 100, the autonomous mobile control unit 141 uses the measurement results to generate surrounding environment data indicating the shape of the surrounding environment, and estimates the self-position on, for example, an area map. The area map is, for example, a map indicating the inside of a facility or the like through which the autonomous mobile device 100 moves.

[0020] The movement mechanism unit 102 is a drive mechanism that enables movement of the autonomous mobile device 100. The movement mechanism unit 102 may be configured to be capable of causing the autonomous mobile device 100 to move forward or backward, as well as to change direction to the left or right, based on information input from the autonomous movement control unit 141.

[0021] The projection unit 103 is configured with a display device that projects an image and displays it on a screen. In this embodiment, the projection unit 103 projects a notification image that notifies the user of the current state of the autonomous mobile device 100 or the movement of the autonomous mobile device 100 from the current time point, in response to the control of the display control unit 142.

[0022] 2 shows an example of the projection of a notification image 200 by the projection unit 103 provided in the autonomous mobile device 100. This figure shows the autonomous mobile device 100 as viewed from the side. In the figure, the autonomous mobile device 100 moves in a traveling direction indicated by an arrow DR. At this time, the direction along the traveling direction of the autonomous mobile device 100 is the front of the autonomous mobile device 100. The projection unit 103 is provided in front of the autonomous mobile device 100 as shown in the figure, for example, and projects the notification image 200 on a portion of the floor surface FL (an example of a moving target surface) that is in front of the autonomous mobile device 100. That is, the projection unit 103 in this case projects the notification image 200 using the portion of the floor surface FL that is in front of the autonomous mobile device 100 as a screen.

[0023] Returning to the explanation in Fig. 1, the control unit 104 executes various controls in the autonomous mobile device 100. The control unit 104 includes an autonomous mobile control unit 141 and a display control unit 142.

[0024] The autonomous movement control unit 141 plans a route to the destination by referring to the area map stored in the area map storage unit 151. The autonomous movement control unit 141 stores the planned route in the route information storage unit 152. In a moving state, the autonomous movement control unit 141 refers to an area map, estimates its own position using the detection information of the sensor unit 101, and controls so that the estimated own position is along the route. By performing control in this manner by the autonomous movement control unit 141, the autonomous mobile device 100 can move autonomously so as to follow a planned route.

[0025] The storage unit 105 stores various types of information corresponding to the autonomous mobile device 100. The storage unit 105 includes an area map storage unit 151, a route information storage unit 152, and an announcement video data storage unit 153.

[0026] The area map storage unit 151 stores an area map corresponding to the area of ​​the movement range of the autonomous mobile device 100.

[0027] The route information storage unit 152 stores route information indicating the route planned by the autonomous movement control unit 141.

[0028] The notification video data storage unit 153 stores video data of various notification videos (notification video data). As shown in FIG. 3, the notification video data storage unit 153 stores therein information indicating the corresponding movement situation for each piece of notification video data in association with the data.

[0029] In a situation where people are mixed in the movement range of the autonomous mobile device 100, the autonomous mobile device 100 may approach a person while moving autonomously. For example, when the autonomous mobile device 100 encounters a person face-to-face, it is difficult for the person to grasp what kind of movement situation the autonomous mobile device 100 will be in from now on. In addition, it is difficult for the autonomous mobile device 100 to predict the future movement of the person it has encountered. For this reason, in a situation where the autonomous mobile device 100 approaches a person so as to face each other, an event such as the autonomous mobile device 100 pausing to avoid a collision is likely to occur due to the action of the autonomous mobile control unit 141 that has received the output of the sensor unit 101. If the autonomous mobile device 100 has more opportunities to pause in this way, it will lead to a decrease in the operating rate of the autonomous mobile device 100.

[0030] Therefore, when the autonomous mobile device 100 of this embodiment moves autonomously along a planned route, it displays an alarm image 200 informing of the upcoming movement status by projecting it onto the floor surface FL ahead in the direction of travel using the projection unit 103. By displaying the notification video 200 in this way, the moving status of the autonomous mobile device 100 from the current time point onward can be accurately notified to people around the autonomous mobile device 100 during autonomous movement. Therefore, people around the autonomous mobile device 100 during autonomous movement can understand how the autonomous mobile device 100 will move from its current position, and can act, for example, to avoid contact with the autonomous mobile device 100. By allowing people to act in this way so as not to impede the future movement of the autonomous mobile device 100, the autonomous mobile device 100 can continue autonomous movement without stopping temporarily even if a person approaches from the opposite direction, and the operating rate can be prevented from decreasing.

[0031] With reference to Figs. 4(A) to 4(H), examples of notification images displayed when the autonomous mobile device 100 is moving autonomously will be described. 4(A) to 4(H), the autonomous mobile device 100 is viewed from a planar direction. In the drawings, the traveling direction of the autonomous mobile device 100 is indicated by an arrow DR. As shown in the drawings, the notification image 200 is displayed at a position forward of the traveling direction of the autonomous mobile device 100.

[0032] The notification image 200 in FIG. 4(A) notifies the user that the autonomous mobile device 100 will move straight along the traveling direction from the current time onward.

[0033] The notification image 200 in FIG. 4(B) notifies the driver that the autonomous mobile device 100 will make a left turn at an angle of 90 degrees from the current time onward.

[0034] The notification image 200 in FIG. 4(C) notifies the user that the autonomous mobile device 100 will change its traveling direction diagonally left from the current time point onward. The notification image 200 in FIG. 4(D) notifies the driver that the autonomous mobile device 100 will make a right turn at an angle of 90 degrees from the current time onward. The notification image 200 in FIG. 4(E) notifies the user that the autonomous mobile device 100 will change its traveling direction diagonally to the right from the current time onward. The notification image 200 in FIG. 4(F) notifies the user that the autonomous mobile device 100 will move to the left from the current time onwards in the same traveling direction as indicated by the arrow DR. The notification image 200 in FIG. 4(G) notifies the user that the autonomous mobile device 100 will move to the right from the current time onwards in the same direction as indicated by the arrow DR. The notification image 200 in FIG. 4(H) notifies that the autonomous mobile device 100 is currently (and will be from now on) stopped.

[0035] For example, when the autonomous mobile device 100 is about to turn left at a corner of a passageway, there is a person ahead of the left turn. In this case, the autonomous mobile device 100 displays a notification image 200 notifying the person that it will turn left, as shown in Fig. 4(B). By looking at the notification image 200 indicating a left turn that is displayed on the floor in front of the autonomous mobile device 100, the person understands that the autonomous mobile device 100 is moving toward them, and can take action such as making space so that the autonomous mobile device 100 can turn left and move. Also, for example, when the autonomous mobile device 100 is in an elevator and the elevator arrives at the destination floor and the door opens, a person who is about to board the elevator from the destination floor is waiting in front of the elevator. In this case, the autonomous mobile device 100 displays, for example, a notification image 200 indicating a straight line from inside the elevator car toward the elevator hall in order to get off the elevator car. By seeing such notification image 200, the person who was about to board the elevator can understand that the autonomous mobile device 100 is about to get off the elevator car. In this case, in order to avoid contact with the autonomous mobile device 100, the person can act by waiting until the autonomous mobile device 100 gets off the elevator car before boarding the elevator.

[0036] With reference to Fig. 5, an example of the change in the notification image 200 according to the progress of the movement of the autonomous mobile device 100 will be described. In the figure, the movement status of the autonomous mobile device 100 in the passage AS according to times t1, t2, and t3, and the notification image 200 are shown. In the figure, the autonomous mobile device 100 is viewed from a planar direction. In the figure, the traveling direction of the autonomous mobile device 100 is indicated by an arrow DR.

[0037] First, the autonomous mobile device 100 detects a person M as an obstacle ahead while moving straight along the aisle AS at a central position in the width direction of the aisle AS. In response to the detection of the person M, the autonomous mobile device 141 re-plans the route so as to move the person M closer to the right of the aisle AS. In this case, the autonomous mobile device 141 changes the route along which the autonomous mobile device 100 will proceed so as to move closer to the right of the aisle AS, and therefore, for example, instead of displaying a notification image indicating going straight up until now, it displays a notification image 200 indicating that the autonomous mobile device 100 will move closer to the right of the aisle as shown in the figure.

[0038] At time t2, a certain time after time t1, the autonomous mobile device 100 changes its travel direction to the right and moves diagonally to the right to move to the right of the passage AS. After this, the autonomous mobile device 100 moves straight along the passage AS at a position that is off to the right, and therefore displays a notification image 200 indicating that the autonomous mobile device will move diagonally to the left as shown in the figure.

[0039] After time t2, the autonomous mobile device 100 moves until it approaches the right side of the passage AS, and when it is determined that it has approached the right side of the passage AS, it changes its traveling direction to a direction going straight along the passage AS, as shown at time t3. At this time, the autonomous mobile device 100 changes the display from the notification image 200 at time t2 to the notification image 200 indicating going straight. In this way, the autonomous mobile device 100 can replan a route in response to detection of the person M approaching in the opposite direction, and move along the replanned route while changing and displaying the notification image 200 so as to indicate the direction of movement from the current point in time according to the movement of the route. By looking at such notification image 200, the person M can know in advance which direction the autonomous mobile device 100 will move from now on, and can move in a way that does not impede the movement of the autonomous mobile device 100.

[0040] An example of a process procedure executed by the autonomous mobile device 100 in relation to displaying a notification video will be described with reference to the flowchart of FIG. Step S100: In the autonomous mobile device 100, the autonomous mobile control unit 141 compares the surrounding environment detected based on the output of the sensor unit 101 with the area map stored in the area map storage unit 151, and estimates the current position of the autonomous mobile device 100.

[0041] Step S102: The autonomous movement control unit 141 refers to the currently planned route from the route information stored in the route information storage unit 152. Note that the route referred to in step S102 is re-planned each time an obstacle is detected by the process in Fig. 7 described below. That is, in step S102, the autonomous movement control unit 141 can refer to the latest route that reflects the results of the previous plan (re-plan).

[0042] Step S104: The autonomous movement control unit 141 determines a movement status of the autonomous mobile body device 100 from the current time until the predetermined time based on the current position estimated in step S100 and the route referred to in step S102. Specifically, the autonomous movement control unit 141 may determine a change in the traveling direction from the current time until the predetermined time as the movement status.

[0043] Step S106: The display control unit 142 acquires, from the notification video data storage unit 153, the notification video data of the notification video corresponding to the moving state determined in step S104.

[0044] Step S108: The display control unit 142 displays the notification video by causing the projection unit 103 to project the video of the notification video data acquired in step S106. When the notification video is displayed in step S108, the process returns to step S100.

[0045] An example of a process procedure executed by the autonomous mobile device 100 in response to route replanning will be described with reference to the flowchart of FIG. Step S200: In the autonomous mobile device 100, the autonomous movement control unit 141 estimates the surrounding environment based on the detection output of the sensor unit 101, and waits for an obstacle to be detected in the estimated surrounding environment.

[0046] Step S202: In response to the detection of an obstacle, the autonomous movement control unit 141 re-plans the route.

[0047] Step S204: The autonomous movement control unit 141 updates the route information stored in the route information storage unit 152 so as to reflect the route replanned in step S202. When the route is updated by the process of Fig. 7, the updated route is referred to in step S102 of Fig. 6. As a result, as illustrated in Fig. 5, the autonomous mobile body device 100 can move according to the replanned route and display the notification image.

[0048] In addition, when a plurality of autonomous mobile devices 100 are present and operating in the same movement range, even when one autonomous mobile device 100 detects another autonomous mobile device 100 as an obstacle, it may replan a route as illustrated in FIG. 5, and move along the replanned route and display the notification image 200. In this case, the other autonomous mobile device 100 may identify the notification image 200 of the opposing autonomous mobile device 100 and perform an operation such as moving or stopping so as not to impede the movement of the autonomous mobile device 100. In this case, when the autonomous mobile devices 100 approach each other in opposition to each other, a predetermined rule may be set so that one of them replans a route and moves, and the other identifies the notification image 200 and operates. As an example, an autonomous mobile device 100 moving up the passage may be given priority to replanning its route and moving along the replanned route, while an autonomous mobile device 100 moving down the passage may be configured to identify and act on the notification image 200 of an opposing autonomous mobile device 100.

[0049] In response to the detection of an obstacle such as a person, the display control unit 142 may change the color of the notification image 200 to, for example, indicate a warning.

[0050] In addition, when the autonomous mobile device 100 is temporarily stopped and the notification image 200 indicating the temporary stop is displayed as shown in FIG. 4(H), the display control unit 142 may change the display, such as by blinking the notification image 200 indicating the temporary stop, in response to the time until the temporary stop is released and movement begins being resumed falling within a predetermined range.

[0051] Note that one or more projection devices (projectors) serving as the projection unit 103 may be provided in the facility, rather than being provided in the autonomous mobile device 100. In this case, the projection unit 103 may be configured to be able to change the position at which the image is projected so as to follow the movement of the autonomous mobile device 100 within a predetermined projection possible range on the floor surface described in the next section. Furthermore, the projection unit 103 may be connected to a display control server so as to be able to communicate with it. In this case, the autonomous mobile device 100 may transmit the determined movement status after the current time and the current position of the autonomous mobile device 100 to the display control server. The display control server may control the projection unit 103, whose projection possible range is the range of the floor surface including the position of the autonomous mobile device 100, to display a notification image according to the movement status at a position in front of the autonomous mobile device 100, following the movement of the autonomous mobile device 100.

[0052] In addition, all or part of the autonomous mobile device 100 in the above-mentioned embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. In addition, the "computer system" referred to here includes hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a storage device such as a hard disk or SSD built into a computer system. Furthermore, the "computer-readable recording medium" may include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned function, and may further be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA.

[0053] The "Sustainable Development Goals (SDGs)" are 17 international goals that were adopted at the United Nations Summit in September 2015. The display control system according to this embodiment can contribute to the achievement of one of the 17 SDGs, for example goal 9 "Build resilient infrastructure, promote industry, innovation and infrastructure." [Explanation of symbols]

[0054] 100 Autonomous mobile device, 101 Sensor unit, 102 Moving mechanism unit, 103 Projection unit, 104 Control unit, 105 Memory unit, 141 Autonomous movement control unit, 142 Display control unit, 151 Area map memory unit, 152 Route information memory unit, 153 Notification video data memory unit, 200 Notification video

Claims

1. an autonomous movement control unit that executes control to autonomously move the autonomous mobile device; a display control unit that causes a projection unit to project an image notifying the autonomous mobile body device of the movement of the autonomous mobile body device from the current time onward based on the control of the autonomous mobile control unit onto a moving target surface in front of the autonomous mobile body device; A display control system comprising:

2. The display control unit causes the projection unit to project an image indicating a moving direction of the autonomous mobile device from a current time onward. The display control system according to claim 1 .

3. The display control unit causes the projection unit to project an image indicating a current state of the autonomous mobile device. The display control system according to claim 1 .

4. The display control unit, when the autonomous mobile device is currently in a paused state, causes the projection unit to project an image indicating that the autonomous mobile device is currently in a paused state. The display control system according to claim 3 .

5. The display control unit causes the projection unit to project an image indicating that the autonomous mobile device is currently in a temporary stop and that a predetermined time will pass until the temporary stop is released. The display control system according to claim 4 .

6. the autonomous movement control unit sets a movement route so as to avoid the obstacle in response to detecting the obstacle ahead; The display control unit causes the projection unit to project an image showing a movement path set by the autonomous movement control unit. The display control system according to claim 1 .

7. The display control unit changes a color of the image in response to detection of an obstacle ahead. The display control system according to claim 1 .

8. A display control method in a display control system, comprising: an autonomous movement control step in which an autonomous movement control unit executes control to autonomously move an autonomous mobile device; a display control step in which a display control unit projects an image notifying about the movement of the autonomous mobile body device from the current time point onward based on the control of the autonomous mobile control unit, using a projection unit provided to project the image onto a moving target surface in front of the autonomous mobile body device; A display control method comprising:

9. Computer, A display control unit that causes a projection unit to project an image notifying the autonomous mobile device of the movement of the autonomous mobile device from the current time onward based on control for autonomously moving the autonomous mobile device onto a moving target surface in front of the autonomous mobile device. A program to function as a

Images