Moving object, control method, and computer program

Abstract

To provide a moving object that can accept instructions from a plurality of directors while executing one workflow.SOLUTION: A moving object capable of moving along a predetermined route to a plurality of destinations in order to execute a predetermined workflow based on a predetermined instruction, comprises: registration means for registering a person recognized in the vicinity of at least one of the plurality of destinations as a director during execution of the workflow; and reception means for receiving an instruction from the director registered by the registration means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a moving object, a control method, and a computer program. [Background technology]

[0002] Conventionally, automatic transport robots and autonomous robots that guide users to desired locations and transport luggage are known. In particular, in the case of autonomous robots, Patent Document 1 discloses a mobile robot that moves while maintaining a predetermined distance from people when providing the above-mentioned services. Patent Document 2 discloses a technology that first photographs a person and then accepts gesture operations from the person who photographed it. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5617562 [Patent Document 2] Patent Publication No. 2022-132905 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technologies disclosed in the above-mentioned Patent Documents 1 and 2, instructions are received from a single person who issues a specific command (hereinafter, "instructor"), but instructions from multiple instructors cannot be received during a series of operations. Also, in order to execute a task that involves moving between multiple instructors at different locations, it is necessary to register all instructors in advance, which makes flexible operation impossible.

[0005] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide a mobile object capable of receiving instructions from a plurality of instructors while executing one workflow. [Means for solving the problem]

[0006] In order to achieve the above object, one aspect of the present invention is a mobile object capable of moving to a plurality of destinations along a predetermined route in order to execute a predetermined workflow based on a predetermined instruction. a registration means for registering a person recognized in the vicinity of at least one of the plurality of destinations as an instructor during execution of the workflow; and a receiving means for receiving an instruction from the instructor registered by the registration means. Effect of the Invention

[0007] According to the present invention, it is possible to provide a moving object capable of receiving instructions from a plurality of instructors while executing one workflow. [Brief description of the drawings]

[0008] [Figure 1] 1 is a perspective view showing a schematic configuration of a moving object 1 in a first embodiment. [Diagram 2] 4 is a flowchart for specifically explaining an example of a control method for the moving object 1 in the first embodiment. [Diagram 3] 4 is a flowchart for explaining a generalized control method for the moving object 1 in the first embodiment. [Figure 4] 11 is a flowchart showing a method for controlling the moving object 1 in the second embodiment. [Diagram 5] 11 is a flowchart showing a control method according to the third embodiment. [Figure 6] 13 is a flowchart showing a control method according to the fourth embodiment. [Figure 7] 13 is a diagram for explaining a moving object having an imaging unit according to a fifth embodiment. FIG. [Figure 8] 13 is a diagram showing an example of an image taken by a moving object having an imaging unit according to the fifth embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In the drawings, the same members or elements are given the same reference numerals, and duplicated descriptions thereof will be omitted or simplified. <Embodiment 1>

[0010] Fig. 1 is a perspective view showing a schematic configuration of a moving object 1 in embodiment 1. In Fig. 1, 1 is a moving object, 3 is a traveling unit, 5 is an imaging unit, 7 is a map information acquisition unit, 9 is a sound receiving unit, and 11 is a loading platform unit.

[0011] The moving body 1 has a driving source such as a motor for movement, a gripping function for gripping the transported object, a function for fixing and mounting the transported object, etc. It may also have a battery for driving the motor, a solar panel, etc. It may also have an engine that uses gasoline, etc.

[0012] In this embodiment, the moving body 1 is configured to move autonomously without a person on board, but some functions may be remotely controlled. The moving body 1 according to the first embodiment uses a cylindrical aluminum housing having a height of 60 cm and a diameter of 50 cm, for example. The moving body 1 also includes, as hardware, a battery, a calculation unit such as a CPU as a computer, and a storage unit as a storage medium such as a RAM, a ROM, a flash memory, and a hard disk drive.

[0013] The running unit 3 is equipped with four rubber tires with a diameter of 10 cm and is driven by a motor, allowing the entire mobile unit 1 to move forward, backward, left and right. The running unit 3 is the means of movement of the mobile unit, and may be one that simulates a walking function, one that uses a propeller-type moving mechanism like a helicopter or drone, or one that uses a floating moving mechanism like a hovercraft.

[0014] That is, the moving body 1 of the first embodiment is a moving robot such as an AMR (Autonomous Mobile Robot) or an AGV (Automated Guided Vehicle). However, the moving body of the embodiment may be a drone or a car, and is not limited to the above-mentioned AMR or AGV. The moving body of the first embodiment may be anything that moves from specific location to specific location to achieve a specific purpose.

[0015] The imaging unit 5 of the first embodiment is composed of at least a lens and a two-dimensional sensor such as a CMOS sensor, and is capable of capturing still and moving images of the periphery of a moving object. The lens is a camera that can observe at a viewing angle of ±60 degrees using a general-purpose lens group.

[0016] The captured image is analyzed by the image recognition unit to recognize the instructor or other person in the image in order to determine the autonomous movement of the mobile unit 1. The recognition is performed by estimating the shape, etc. from the contrast of the image and determining whether there is a certain degree of possibility that the person could be the instructor.

[0017] The command of the instructor is also recognized and identified through image recognition. That is, the command is accepted by image recognition of the shape of a part of the instructor's body, gestures, or other movements, or a specific pattern such as a QR code (registered trademark). Note that voice commands or commands based on character input may also be accepted.

[0018] The map information acquisition unit 7 acquires two-dimensional or three-dimensional point cloud information of objects around the mobile object 1 by using an infrared sensor, a LiDAR (Light Detection And Ranging), a parallax image acquisition camera, etc., relative to the position of the mobile object 1. In addition, it generates map information of the surroundings based on the acquired point cloud information.

[0019] The moving body 1 can generate a travel route for moving and move autonomously to the destination along the generated route by using the map information and the position and orientation information from the imaging unit 5. The movement to the destination may be based on a single route generation, or the route may be modified while moving.

[0020] In addition, if map information and current position and orientation information of the moving body 1 within the map information are stored in advance within the moving body 1 or an external server, sensors such as the imaging unit 5 may be used to confirm the stored map information and the self-position and orientation of the moving body 1.

[0021] Note that multiple cameras arranged at positions overlooking the travel route of the moving body may be used to confirm the self-position of the moving body 1 and to confirm changes in map information. That is, for example, when an obstacle occurs or when the position of an obstacle changes, these may be detected by another camera.

[0022] The sound receiving unit 9 is, for example, a microphone. The instructor himself may be recognized by analyzing the voice acquired by the sound receiving unit 9. In other words, by receiving a character string of specific words, the instructor's unique voiceprint may be linked to the instructor and registered. Also, the instruction content of the instructor's command may be analyzed from the voice signal, and the instruction may be recognized and identified.

[0023] When recognizing the voice of the instructor itself, the recognition rate may be increased in conjunction with image recognition by the imaging unit 5. In addition, a plurality of sound receiving units may be provided, and the position and direction of the instructor may be estimated based on the difference in arrival time of the sound reaching the sound receiving units, thereby identifying the instructor.

[0024] In this way, a microphone may be used as a means for increasing the likelihood of estimating the location of the destination and tracking the location. Also, as described above, the moving body 1 may be equipped with an input means (such as a button or a touch panel) for an instructor to input commands as characters.

[0025] In this way, the moving body 1 has functional parts such as an image recognition unit that recognizes the instructor, a reception unit that receives commands from the instructor, a map information acquisition unit 7, an imaging unit 5, and a sound receiving unit 9, and uses these to generate a travel route.

[0026] Some of the above-mentioned functional parts of the moving body 1 are realized by having a CPU or the like as a computer (not shown) included in the autonomous driving system 150 or the moving body 1 execute a computer program stored in a memory (not shown) as a storage medium.

[0027] However, some or all of these may be realized by hardware. For the hardware, a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP) may be used. Furthermore, the functional blocks shown in Fig. 3 do not have to be built into the same housing, and may be configured by separate devices connected to each other via signal paths.

[0028] The platform 11 is a platform on which transported items are placed on the moving body 1, for example, for carrying containers containing coffee or the like, or for carrying food, and the moving body 1 of this embodiment uses the platform 11 for, for example, a food serving function. The platform 11 may also have a multi-stage configuration. Note that the moving body 1 may be provided with a gripping mechanism such as a mechanical arm or hand, so that the arm or hand can grip the transported item and load it onto or unload it from the platform 11.

[0029] Fig. 2 is a flowchart for specifically explaining an example of a control method for the moving object 1 in the embodiment 1, and the execution flow of the workflow of the moving object 1 according to the embodiment 1 will be explained with reference to Fig. 2. Note that the operation of each step in the flowchart in Fig. 2 is performed sequentially by the CPU serving as a computer of the moving object 1 executing a computer program stored in a memory.

[0030] In this embodiment, a workflow refers to a series of interactions between the mobile object 1 and a user of the mobile object 1 until a specific purpose is achieved using the mobile object 1. The mobile object 1 of this embodiment can move to a plurality of destinations along a predetermined route to execute a predetermined workflow based on a predetermined instruction from an instructor, using a control method as described below.

[0031] In the workflow described with reference to FIG. 2, when a moving object 1 is used in a home, there are person A and person B who appear as instructors, with person A being in place a and person B being in place b.

[0032] In Figure 2, when person A places an order for a beverage such as coffee to moving object 1, moving object 1 moves to location b, finds person B, and has person B make the coffee. When person B places a cup of coffee on moving object 1, moving object 1 carries the coffee cup to person A's location.

[0033] That is, in the workflow in Figure 2, when the flow starts, person A at location a is identified and registered, the order is taken, person B goes to location b, is identified and registered, the ordered items are placed on the order, and the order is brought to person A at location a.

[0034] 2 starts, for example, when a predetermined time period arrives. Alternatively, it starts when it is detected that a person has entered a room. Next, in step S201, the CPU of the moving object 1 sets the first destination as the location a.

[0035] Next, in step S202, the CPU of the moving object 1 starts moving the moving object 1 from a home position (initial position) during standby for charging, etc., toward a destination, place a. Place a is, for example, a place around a sofa in a living room where person A is relaxing.

[0036] In step S203, the CPU of the moving object 1 determines that the moving object 1 has arrived at the location a as the first destination, and in step S204, the CPU confirms through internal processing that the location a is the location where the instructor is to be added (registered). Furthermore, in step S205, the CPU of the moving object 1 captures the gesture operation of person A, who is the instructor.

[0037] Specifically, the person who is about to give an order is indicated in a recognizable manner by a gesture such as waving or raising a hand, by the moving body 1. When the gesture is recognized, in step S206, the CPU of the moving body 1 registers person A as an instructor (a person giving orders).

[0038] Here, step S206 functions as a registration step (registration means) for registering a person recognized in the vicinity of at least one of a plurality of destinations as an instructor during execution of the workflow.

[0039] Next, in step S207, the CPU of the moving object 1 identifies the person A as the person instructing the user and receives the command. That is, when the person A utters a voice such as "I want to drink coffee," the voice is picked up by the sound receiving unit 9 in FIG. 1 and the voice is received as a command from the person A who is the person instructing the user.

[0040] Here, step S207 functions as a receiving step (receiving means) for receiving an instruction from the instructor registered in the registration step. By receiving an instruction from the registered instructor, the receiving means makes the instruction executable in step S208 and after.

[0041] By receiving the command, the objective at the destination location a is completed. Next, in step S208, the CPU of the moving object 1 sets the next destination to the second destination location b (for example, the kitchen).

[0042] In step S209, the CPU of the moving object 1 causes the moving object 1 to head toward the location b, which is the destination, and in step S210, the CPU of the moving object 1 determines that the moving object 1 has arrived at the location b, which is the second destination.

[0043] In step S211, the CPU of the moving body 1 recognizes that the place b is the location where an instructor is to be added. Person B is present at the place b, and person B notifies the moving body 1 that he or she intends to prepare coffee, for example, by gestures, voice operation, button operation, or the like.

[0044] In step S212, the CPU of the moving body 1 detects the above gestures, etc., and in step S213, the CPU of the moving body 1 additionally registers person B as an instructor. In this way, the registration means registers a person as an instructor based on at least one of a predetermined gesture of the person recognized in the vicinity of the destination, the voice of the person, or a button operation of the person.

[0045] When the coffee is prepared by person B, person B places the cup containing the coffee on the carrier 11 of the moving object 1. Then, in step S214, the CPU of the moving object 1 recognizes that person B has spoken to the moving object 1, such as "Bring the coffee" or "Thank you," and receives a command from person B, who is the instructor.

[0046] This completes the purpose at the location b, and in step S215 the CPU of the moving object 1 sets the next destination to the third destination, location a. That is, the original location a is set as the third destination.

[0047] Next, in step S216, the CPU of the moving object 1 directs the moving object 1 to the location a, which is the destination, and in step S217, the CPU of the moving object 1 determines that the moving object 1 has arrived at the location a, which is the third destination.

[0048] Next, in step S218, the CPU of the moving body 1 determines that the location a, which is the third destination, is not an instructor addition point. That is, since the instructor at the location a is the first instructor, person A, who has already been registered, no new instructor is added, and it is determined that the location a, which is the third destination, is not an instructor addition point.

[0049] Next, in step S219, the CPU of the moving object 1 detects a user operation by person A. That is, a user operation such as receiving a cup of coffee from person A is detected. Since the purpose of this workflow is thus completed, in step S220, the CPU of the moving object 1 returns to the home position as an end process, and ends the flow of FIG.

[0050] In the above flow, the location a and the location b as the destination may be coordinates in space, may be a specific area, or may be the location of a person in a specific area.

[0051] In this embodiment, although the workflow is decided at the start, it is not decided who will be the first and second instructors. Then, after moving to location a, person A who will be the first instructor is recognized and registered as an instructor, and after moving to location b, person B who will be the second instructor is recognized and registered. That is, the process surrounded by the dotted line as step S221 in Fig. 2 is characteristic.

[0052] To complete the workflow's objective of "receive and transport coffee," the person who places the cup of coffee on the loading platform 11 does not have to be person B, but may be another person B' who is in the vicinity of location b and can perform a similar task. Therefore, any person who can perform the task at location b is recognized as a related person and registered as the second instructor.

[0053] This allows the operator to give any instruction to the mobile unit 1 if he / she wishes to add, change, overwrite, etc. an instruction, while also being able to flexibly respond to cases where the operator changes at any time without the need for pre-settings, etc.

[0054] When the moving body 1 receives a command, the moving body 1 may identify whether the command is given by a previously registered instructor. That is, the moving body 1 may be configured to identify a person previously registered as an instructor, receive commands from that person, and not receive commands from people other than the instructor.

[0055] In addition, by identifying the person giving the command, it is possible to prevent situations where commands are overwritten contrary to the intentions of the original commander, for example, when a third party unrelated to the workflow makes an action that is interpreted as a gesture or utters a sound that is interpreted as an command near the moving object 1.

[0056] For example, the first person may give instructions to pick up something from the second person and deliver it to the third person. This is a workflow used in, for example, transporting food and drink within a home, clearing the table at a restaurant, and in the delivery business.

[0057] Also, if there are multiple delivery destinations, a fourth or fifth person may be added. Therefore, if the workflow of this embodiment is generalized, a person N1 is recognized as an instructor at a location n=1, registered, receives an instruction, and heads to a location n=2. This process is repeated a number of times where location n is a finite integer.

[0058] This workflow can be expressed as a generalized flowchart as shown in Fig. 3. That is, Fig. 3 is a flowchart for generalizing and explaining the control method of the moving body 1 in embodiment 1. Note that the operation of each step in the flowchart in Fig. 3 is performed sequentially by the CPU, which serves as the computer of the moving body 1, executing a computer program stored in the memory.

[0059] The relationship between Figure 2 and Figure 3 will be explained. In the example of Figure 2, the flow goes to location a, then to location b, and finally to location a. In contrast, in Figure 3, each location is expressed as a variable called location n, and the location a determined in step S201 is generalized as location n=1.

[0060] 2 is set as location n = 2, and location a as a third location set in step S215 is set as location n = 3. Steps S202 to S208 are set as a first group of processes, steps S209 to S215 are set as a second group of processes, and steps S216 to S218 are set as a third group of processes.

[0061] These can be collectively described as steps S302 to S308 in Fig. 3. In particular, the third group of steps corresponds to steps S302 to S304 in Fig. 3. In this way, the workflow in Fig. 2 can be generalized and described as in Fig. 3.

[0062] Furthermore, the flow in FIG. 3 can also include the workflow shown in FIG. 2, where the process of starting an action from an undetermined state, moving to a destination, and then adding an instructor can be shown as the dotted line in step S312.

[0063] Next, with reference to FIG. 3, a specific flow for ordering coffee and having the coffee delivered will be described below.

[0064] At the start, in step S301, the CPU of the moving body 1 sets the destination at location n=1, and in step S302, heads to the destination location n. At this point, the instructor has not yet been determined. In step S303, the CPU of the moving body 1 confirms that it has arrived at the destination location n, and in step S304, determines whether this is a location to add an instructor. At this time, it is assumed that person N=1 who plans to order coffee raises their hand.

[0065] The imaging unit 5 of the moving body 1 performs image recognition, and in step S305 the CPU of the moving body 1 recognizes the person N=1 who raised his hand as the leader, and in step S306 classifies the facial features and voice features of person N=1 and registers him as the leader.

[0066] Thereafter, in S307, the CPU of the moving body 1 identifies the person N=1 as the person instructing, receives the person N=1's speech "I want to drink coffee" at the sound receiving unit 9, and receives the command by analyzing the words.

[0067] In S308, the CPU of the moving body 1 sets the next destination location n=n+1, proceeds to step S302, and moves to location n=2. At this point, a person to instruct the moving body to make coffee has not yet been decided. When the moving body 1 reaches location n=2 in step S303, the CPU of the moving body 1 determines in S304 that this is now a location to add a person to instruct the moving body to prepare coffee. At location n=2, person N=2 raises his hand and says, "I'll make coffee."

[0068] In S305, the CPU of the moving body 1 recognizes the person N=2 as an instructor through image recognition, and in S306 registers the facial features and voice features of person N=2 as an instructor for person N=2. After that, person N=2 loads the prepared cup of coffee into the loading platform 11 of the moving body 1, and when the CPU of the moving body 1 detects a voice instruction from person N=2 (e.g., "carry it") in S307, it sets the next destination, location n=3, in step S308.

[0069] Thereafter, again in step S302, the CPU of moving body 1 causes the moving body to head to location n=3, and in step S303 the moving body reaches location n=3, but since there is no more person to add in step S304, the process proceeds to step S309.

[0070] In step S309, the CPU of the moving object 1 determines whether location n is the final point, and if No, proceeds to step S307. However, in this case, n=3 is the final point, so proceeds to step S310, detects an operation to receive person N's coffee cup, and confirms that the coffee has been delivered to person N=1. Then, proceeds to step S311, where the CPU of the moving object 1 performs an end process and returns to the home position.

[0071] By executing such a flow, it is possible to start the workflow operation as in the first embodiment even when the instructor is not recognized at the start of the operation. Then, the instructor can be identified along the way and the final destination cup of coffee can be carried to the first instructor.

[0072] Also, for example, while the mobile object is serving coffee, if a third person who is not a registered person issues an instruction to the mobile object 1 such as "Bring me a coffee too," the instruction will not be accepted if the third person is not a registered person. In other words, since the receiving means does not accept instructions from anyone other than registered instructors during the execution of the workflow, the initial objective can be reliably accomplished.

[0073] Also, for example, while the mobile object is serving coffee, if a third person who is not a registered person raises their hand and tries to stop them in the same way as the first person, the new person will not become a registered person unless they are in a specific location.

[0074] That is, the registration means does not register instructors other than those near the destination during execution of the workflow. Therefore, in the first embodiment, the work is not interrupted until the flowchart in FIG. 3 is completed, and the first objective can be completed.

[0075] In the above, an example of the moving object 1 transporting a cup of coffee in a home has been described. However, the present invention can be applied to any workflow that involves repeated movement, detection of people in a specific area, and tasks, such as transporting or delivering other items in a store, factory, company, or community.

[0076] It is also possible to automatically delete the registered instructors when the flow of Fig. 3 is completed. In other words, when the workflow is completed, the registration means may delete at least some of the instructors registered during the execution of the workflow. This allows for appropriate handling of new orders, for example, when serving food in a restaurant.

[0077] Alternatively, when the flow of Fig. 3 is completed, a specific registrant may be used as a registrant the next time the flow of Fig. 3 is started. In other words, the registration means may be able to select not to erase at least a part of the registration of the instructor who was registered during the execution of the workflow when the workflow is completed.

[0078] This can improve efficiency, for example, when a store or the like needs to exclusively use the same mobile vehicle for a specific customer for a specified period of time, or when the same customer requests additional services.

[0079] Furthermore, when the flow in Figure 3 is completed, the user may be able to choose whether or not to automatically delete the registration of the registrant, and if automatic deletion is not selected, the user may be able to manually delete each registrant.

[0080] Before starting the flow of FIG. 3 or during the flow of FIG. 3, a user may be allowed to reserve and register the person who will instruct the next execution of the flow of FIG. 3 by using a specific gesture or the GUI of the mobile terminal.

[0081] That is, while a workflow is being executed, a person other than the instructor registered by the registration means may be made available to be registered as an instructor for the next workflow. This allows, for example, the first instructor to be determined in advance when the next flow of Fig. 3 is started. Therefore, for example, a person who wishes to order coffee or the like can register a reservation while a workflow is being executed, and orders can be accepted in the order of reservation registration.

[0082] Also, the priority of the instruction of the person who issued the instruction immediately before to the moving body 10 may be set high. That is, after the moving body 1 starts a predetermined operation based on the instruction of the person who issued the instruction immediately before, the instruction of the person who issued the instruction immediately before may be given priority until the operation is completed. Also, if the person who issued the instruction immediately before changes or cancels the content of the instruction immediately before, the instruction change or cancellation may be accepted preferentially.

[0083] This allows the instruction of the immediately previous instructor to be executed with priority, and even if multiple registered instructors issue various change instructions during the workflow of the mobile body, the action instructed immediately before can be reliably carried out. In other words, in this embodiment, it is possible to set priorities for multiple instructors, and the most recently registered instructor may be given the highest priority.

[0084] For example, if, while carrying coffee, the person who ordered the coffee issues an instruction such as "Tea, please," the robot may go back to get the tea, still holding the coffee. By not changing the behavior until the command of the person who prepared the coffee, that is, the previous instruction to deliver the coffee, is executed, a reliable workflow can be implemented.

[0085] If there is no person to be the instructor at the destination, the robot may wait for a predetermined time. If the person to be the instructor at the destination does not appear even after waiting for the predetermined time, the robot may return to the previous location or the home position to wait for instructions. In other words, if the registered instructor does not return to the predetermined location even after the predetermined time has passed, the robot may return to the previous location or the home position.

[0086] For example, if a coffee is received in the kitchen and the person who ordered it disappears, the coffee may be returned to the person who prepared it. This allows for stable response even when such an unexpected event occurs.

[0087] By using a mobile object having the functions of this embodiment and using the control method shown in the flowcharts described in Figures 2 and 3, it is possible to realize a mobile object that can efficiently carry out a workflow even if all of the instructors have not been determined at the start. <Embodiment 2>

[0088] Fig. 4 is a flowchart showing a method for controlling the moving body 1 in embodiment 2. Note that the CPU serving as a computer of the moving body 1 executes a computer program stored in a memory, thereby sequentially performing the operations of the steps in the flowchart of Fig. 4.

[0089] The second embodiment will be described with reference to Fig. 4. The second embodiment has the same configuration as the first embodiment, except that the flowchart in Fig. 3 of the first embodiment is replaced by the flowchart in Fig. 4.

[0090] The flowchart in FIG. 4 of the second embodiment differs from FIG. 3 in that step S312 (that is, steps S304 to S306) of the flowchart in FIG. 3 of the first embodiment is omitted, and other processing is the same as that in FIG.

[0091] That is, steps S401 to S403 and steps S407 to S411 in Fig. 4 are the same as steps S301 to S303 and steps S307 to S311 in Fig. 3, respectively. Therefore, a description of the processing in those steps will be omitted.

[0092] In the flowchart of Fig. 4, since there is no process of step S312 for recognizing and registering an instructor during the process, it is necessary to register an instructor who appears in the flow before starting the flow of Fig. 4. Therefore, in the second embodiment, it is not possible to deal with a case where the person who requests coffee or the person who makes coffee in the kitchen suddenly changes. <Embodiment 3>

[0093] Fig. 5 is a flowchart showing a control method of embodiment 3. Embodiment 3 has the same configuration as embodiment 1, except that the flowchart of Fig. 3 of embodiment 1 is replaced by the flowchart of Fig. 5. Note that the operation of each step of the flowchart of Fig. 5 is performed sequentially by the CPU as a computer of the moving object 1 executing a computer program stored in the memory.

[0094] The flowchart in Fig. 5 differs from Fig. 3 of the first embodiment in that a step group (steps S530 and S531) of step S532 in Fig. 5 is added compared to Fig. 3. That is, steps S501 to S511 in Fig. 5 are the same as steps S301 to S311 in Fig. 3, respectively, and therefore a description of the processing in those steps will be omitted.

[0095] 5, on the way to the destination location n, the CPU of the moving object 1 determines in step S530 whether a command has been issued from person N-1, who was the previous instructor. For example, assume that person N-1 issues a command by voice, such as "I forgot the sugar. Come back."

[0096] By recognizing this voice as a command, the CPU of the moving object 1 sets the previous destination, location n=n-1, in step S531, and moves to location n=n-1 in step S502. Therefore, the insertion of such step group 532 is a feature of the third embodiment.

[0097] In this way, in the third embodiment, since the instruction of the registered user can be accepted even during the movement, the destination can be corrected to the previous destination, and the workflow can be adjusted from step S502. Therefore, it is possible to realize a moving body 1 that can reduce human error. <Embodiment 4>

[0098] Fig. 6 is a flowchart showing a control method of embodiment 4. Note that the CPU serving as a computer of the moving body 1 executes a computer program stored in the memory, thereby sequentially performing the operations of the steps in the flowchart of Fig. 6.

[0099] Fig. 6 differs from the flowchart of Fig. 3 only in that steps S613 and S614 are added. That is, steps S601 to S612 in Fig. 6 are the same as steps S301 to S312 in Fig. 3, respectively, and therefore a description of the processing in those steps will be omitted.

[0100] In step S613, the CPU of the moving object 1 determines whether or not there is a command from the instructor to specify the next destination. If NO, the process proceeds to step S608, and the same process as in the flow of Fig. 3 is performed. On the other hand, if the result in step S613 is YES, the process proceeds to step S614.

[0101] In step S614, the CPU of the moving body 1 updates the content of the next destination, location n+1, according to the instruction of the instructor. That is, for example, when receiving an instruction from person N as an instructor specifying the next destination, such as "Go to location d" or "Deliver this to person A," the next location or area to go to is changed according to the received instruction. In this way, the instructor registered by the registration means can specify the next location to go to during the execution of the workflow.

[0102] This allows appropriate correction of the destination. It also makes it possible to increase the number of workflow moves along the way, or to add additional workflows. This allows appropriate achievement of goals that flexibly respond to real-time changes in goals. <Embodiment 5> Next, the fifth embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a diagram for explaining a moving object 1 having an imaging unit according to the fifth embodiment, and Fig. 8 is a diagram showing an example of an image captured by the moving object 1 having the imaging unit according to the fifth embodiment.

[0103] In the fifth embodiment, the imaging unit 5 as the imaging means mounted on the moving object 1 used in the first embodiment is a fisheye camera, which differs from the first embodiment in that it can continue to track the instructor. As shown in Fig. 8, the fisheye camera used as the imaging unit in the fifth embodiment has the center of the optical axis of the lens facing the ceiling C1, and can observe an angle of view of approximately 90 degrees on one side from the optical axis.

[0104] In other words, it is possible to capture images of the entire horizontal periphery from the upward optical axis of the fisheye camera. Note that the imaging device serving as the imaging means does not have to be a fisheye camera, and any imaging device capable of capturing wide-angle images for tracking the instructor may be used.

[0105] In this manner, in this embodiment, an imaging device capable of wide-angle imaging is used to track the instructors, and therefore, unless they are hidden by obstacles such as pillars or walls, the two instructors, person N1 who ordered the coffee and person N2 who prepared the coffee, can be imaged and tracked within the field of view from the start of their movement to the end of their movement.

[0106] Therefore, even if a person registered as an instructor moves during the workflow of the moving object 1, the latest location of the person can be grasped. Therefore, even if person N1 moves when receiving coffee in the kitchen and then delivering it to person N1, a route to the destination person N1 can be generated with high accuracy, and the coffee can be delivered reliably.

[0107] Although the present invention has been described in detail above based on the preferred embodiments, the present invention is not limited to the above-mentioned embodiments, and various modifications and combinations of the above-mentioned embodiments are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The present invention includes the following combinations.

[0108] (Configuration 1) A mobile body capable of moving to a plurality of destinations along a predetermined route in order to execute a predetermined workflow based on a predetermined instruction, the mobile body comprising: a registration means for registering a person recognized in the vicinity of at least one of the plurality of destinations as an instructor during execution of the workflow; and a reception means for receiving commands from the instructor registered by the registration means.

[0109] (Configuration 2) The moving body according to configuration 1, wherein the receiving means is capable of executing the instruction by receiving the instruction from the registered instructor.

[0110] (Configuration 3) The mobile body according to configuration 1 or 2, wherein the registration means does not perform the registration outside the vicinity of the destination during execution of the workflow.

[0111] (Configuration 4) The mobile body according to any one of configurations 1 to 3, wherein the accepting means does not accept commands from any other than the registered instructor during execution of the workflow.

[0112] (Configuration 5) The mobile body according to any one of configurations 1 to 4, wherein the registration means deletes at least a portion of the instructors registered during execution of the workflow when the workflow is completed.

[0113] (Configuration 6) The mobile body described in any one of configurations 1 to 5, wherein the registration means is capable of selecting not to erase at least a portion of the instructors registered during execution of the workflow when the workflow is completed.

[0114] (Configuration 7) The mobile body described in any one of configurations 1 to 6, characterized in that during execution of the workflow, a person other than the instructor registered by the registration means can be reserved and registered as the instructor in the next workflow.

[0115] (Configuration 8) The mobile body according to any one of configurations 1 to 7, wherein the instructor registered by the registration means can specify a next location to move to during execution of the workflow.

[0116] (Configuration 9) The mobile body described in any one of configurations 1 to 8, characterized in that the registration means registers the person as the instructor based on at least one of a predetermined gesture of the person, the person's voice, or a button operation recognized in the vicinity of the destination.

[0117] (Configuration 10) The moving body according to any one of configurations 1 to 9, further comprising an imaging means capable of wide-angle imaging for tracking the instructor.

[0118] (Configuration 11) The moving body according to any one of configurations 1 to 10, characterized in that priorities can be set for a plurality of the instructors.

[0119] (Configuration 12) The moving body according to configuration 11, wherein the priority of the instructor who was registered last is set to be the highest.

[0120] (Configuration 13) The moving body according to any one of configurations 1 to 12, characterized in that if the registered instructor does not return to a predetermined location even after a predetermined time has elapsed, the moving body returns to the previous position or to a home position.

[0121] (Method) A control method for a mobile object capable of moving to multiple destinations along a specified route to execute a specified workflow based on specified instructions, the control method comprising: a registration step of registering a person recognized in the vicinity of at least one of the multiple destinations during execution of the workflow as an instructor; and a reception step of accepting commands from the instructor registered by the registration step.

[0122] (Program) A computer program for controlling each means of the moving object according to any one of configurations 1 to 13 by a computer.

[0123] In addition, in order to realize some or all of the control in the above embodiment, a computer program that realizes the functions of the above-mentioned embodiment may be supplied to a moving body, autonomous driving system 150, etc. via a network or various storage media.

[0124] The computer (or CPU, MPU, etc.) in the moving object or the autonomous driving system 150 may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0125] 1: Moving object 3: Running part 5: Imaging section 7: Map information acquisition section 9: Sound receiving section 11: Cargo area

Claims

1. A mobile object capable of moving to a plurality of destinations along a predetermined route in order to execute a predetermined workflow based on a predetermined instruction, a registration means for registering a person recognized in the vicinity of at least one of the plurality of destinations as an instructor during execution of the workflow; a receiving unit for receiving an instruction from the instructor registered by the registration unit.

2. 2. The mobile body according to claim 1, wherein the accepting unit is capable of executing the instruction by accepting the instruction from the registered instructor.

3. 2. The mobile body according to claim 1, wherein the registration means does not perform the registration outside the vicinity of the destination during execution of the workflow.

4. 2. The mobile body according to claim 1, wherein said reception means does not receive commands from any person other than the registered instructor during execution of said workflow.

5. 2. The mobile body according to claim 1, wherein the registration means deletes at least a part of the instructors registered during execution of the workflow when the workflow is completed.

6. 2. The mobile body according to claim 1, wherein the registration means is capable of selecting not to erase at least a part of the instructors registered during execution of the workflow when the workflow is completed.

7. 2. The mobile body according to claim 1, wherein a person other than the instructor registered by the registration means can be reserved and registered as the instructor in the next workflow during execution of the workflow.

8. 2. The mobile body according to claim 1, wherein the instructor, who is registered by the registration means, can specify a next location to which the mobile body should move during execution of the workflow.

9. The mobile body according to claim 1, characterized in that the registration means registers the person as the instructor based on at least one of a predetermined gesture of the person, the person's voice, or a button operation recognized in the vicinity of the destination.

10. 2. The moving body according to claim 1, further comprising an image pickup means capable of taking wide-angle images for tracking the instructor.

11. 2. The moving body according to claim 1, wherein priorities can be set for a plurality of instructors.

12. 12. The mobile body according to claim 11, wherein the priority of the instructor who was registered last is set to be the highest.

13. 2. The moving body according to claim 1, wherein if the registered instructor does not return to a predetermined location within a predetermined time, the moving body returns to a previous location or a home position.

14. A method for controlling a mobile object capable of moving to a plurality of destinations along a predetermined route in order to execute a predetermined workflow based on a predetermined instruction, comprising the steps of: a registration step of registering a person recognized in the vicinity of at least one of the plurality of destinations as an instructor during execution of the workflow; a receiving step of receiving an instruction from the instructor registered in the registration step.

15. A computer program for controlling each means of a moving body according to any one of claims 1 to 13 by a computer.

Images