Autonomous mobile body operation system, autonomous mobile body operation method, and recording medium

Abstract

An autonomous mobile body operation system, an autonomous mobile body operation method, and a recording medium. A robot's response to a person is a secondary service with respect to a primary service such as carrying and cleaning. Furthermore, the robot's action area is also limited. Thus, the robot only stays in dialogue with a person and cannot widely respond to a question or inquiry raised by the person. In an autonomous mobile body operation system having a plurality of autonomous mobile bodies and an operation device capable of communicating with the plurality of autonomous mobile bodies, the operation device has a cooperative processing section that, when receiving a first inquiry of a first person from a first autonomous mobile body that is one of the plurality of autonomous mobile bodies, selects a second person capable of responding to the first inquiry in accordance with the first inquiry, and instructs a second autonomous mobile body that is one of the plurality of autonomous mobile bodies different from the first autonomous mobile body to make a second inquiry based on the first inquiry to the second person.

Description

Technical Field

[0001] This disclosure relates to an autonomous mobile entity application system, autonomous mobile entity application method, and recording medium that utilizes an autonomous mobile entity to perform its main business. Background Technology

[0002] In recent years, the working-age population has been declining, making the elimination of labor shortages a top priority. Therefore, autonomous mobile robots performing specialized tasks have been introduced into various human-occupied locations such as restaurants, offices, hospitals, nursing facilities, commercial facilities, and construction sites. Furthermore, some of these robots can even interact with humans.

[0003] In Patent Document 1, robots such as handling robots and cleaning robots guide customers to their seats in a restaurant. At this time, the robot determines a conversation topic that it estimates the customer will be highly interested in or enjoy based on the customer's appearance. Then, while guiding the customer to their seat, it engages in small talk related to that topic, thereby improving customer service.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2023-139205

[0005] However, in the technology of Patent Document 1, the robot's interaction with customers is a secondary function compared to its primary tasks such as handling and cleaning. Furthermore, the robot's operational area is limited. Therefore, the robot is only capable of conversing with customers and cannot broadly address their questions and inquiries. Summary of the Invention

[0006] This disclosure was made to address the aforementioned issues. Autonomous mobile units, whose main functions are transporting and cleaning, can respond to inquiries from people broadly beyond their operational area by collaborating with other autonomous mobile units. Therefore, the aim is to achieve more efficient utilization of autonomous mobile units.

[0007] In the autonomous mobile body application system disclosed herein, which has multiple autonomous mobile bodies and an application device capable of communicating with multiple autonomous mobile bodies, the application device has a cooperation processing unit. When the cooperation processing unit receives a first inquiry from a first autonomous mobile body, which is one of the multiple autonomous mobile bodies, it selects a second person capable of responding to the first inquiry based on the first inquiry, and instructs the second autonomous mobile body, which is one of the multiple autonomous mobile bodies different from the first autonomous mobile body, to conduct a second inquiry to the second person based on the first inquiry.

[0008] According to this disclosure, autonomous mobile entities, through cooperation with other autonomous mobile entities, can respond to inquiries from humans broadly beyond their operational area. This enables highly efficient utilization of autonomous mobile entities. Attached Figure Description

[0009] Figure 1 This is a block diagram showing the general structure of the robot application system in Embodiment 1.

[0010] Figure 2 This is an image of the transport robot.

[0011] Figure 3 This is a block diagram showing the general structure of the transport robot.

[0012] Figure 4 This is an image of the cleaning robot.

[0013] Figure 5 This is a block diagram showing the general structure of a cleaning robot.

[0014] Figure 6 It is the employee information stored in the employee database.

[0015] Figure 7 This is a block diagram showing the structure of the collaborative processing unit.

[0016] Figure 8 This is a flowchart of the robot application system in Implementation Method 1.

[0017] Figure 9 This is a schematic diagram of a knowledge graph.

[0018] Figure 10 This is a flowchart illustrating the process when similarity calculation is used.

[0019] Figure 11 This is a diagram illustrating an example of utilizing the hardware resources of a device.

[0020] Figure 12 This is another diagram illustrating the use of the device's hardware resources.

[0021] Figure 13 This is a flowchart of the robot application system in Implementation Method 2.

[0022] Figure 14 This is a block diagram showing the general structure of the robot application system in Embodiment 3.

[0023] Figure 15 This is an image of the nursing robot.

[0024] Figure 16 This is a block diagram showing the general structure of the nursing robot.

[0025] Figure 17 It is robot information stored in the robot database.

[0026] Figure 18 This is a flowchart of the robot application system in implementation method 3.

[0027] Figure 19 This is a flowchart of the robot application system in implementation method 4.

[0028] Figure 20 This is a block diagram showing the general structure of the robot application system in Embodiment 5.

[0029] Figure 21 It is the patient information stored in the patient database.

[0030] Figure 22 This is a flowchart of the patient database data collection / accumulation process.

[0031] Figure 23 This is a flowchart of the robot application system in implementation method 5.

[0032] Label Explanation

[0033] 100: Robot; 100a: Robot A; 100b: Robot B; 100c: Robot C; 100d: Robot D; 101a, 101b: Head; 102a, 102b, 102d: Main body; 102d1: Arm; 102d2: Fingers; 103a, 103b: Driving unit; 103d: Walking unit; 103a1, 103b1: Tires; 103d1: Feet; 104a, 104b, 104d: Camera unit; 105a: Shelf; 106a, 106b, 106d: Monitor; 107a, 107b, 107d: Ambient sensors; 108a, 108b, 108d: Control unit; 109a, 109b...

[0034] 109d: Position Detection Unit; 110a, 110b, 110c: Transceiver Unit; 111a, 111b, 111c: Battery; 112a, 112b, 112d: Speaker; 113a, 113b, 113d: Microphone; 114a: Transport Management Unit; 115b: Cleaning Unit; 200: Operating Device; 200a: Processor; 200b: Memory; 200c: Transceiver Circuit; 200d: Dedicated Hardware; 201: Transceiver Unit; 202: Image / Sound Analysis Unit; 203: Personnel Position Storage Unit; 204: Environmental Storage Unit; 205: Inquiry Analysis Unit; 206: Collaboration Processing Unit; 206a: Staff Selection Unit; 206b: Robot Selection Unit; 206c: Response Instruction Production Department; 206d: Collaborative Response Production Department; 207: Response Production Department; 208: Response Database; 209: Staff Database; 209a: Staff Name Data; 209b: Photo Data; 209c: Department Data; 209d: Skill Data; 209e: Business Status Data; 210: Feedback Processing Department; 211: Robot Database; 211a: Identifier Data; 211b: Category Data; 211c: Responsible Location Data; 211d: Functional Data; 212: Database Accumulation Department; 213: Patient Database; 213a: Patient Name Data; 213b: Gender Data; 213c: Photo Data; 213d: Ward Data; 213e: Skill Data; 213f: Personality Data. Detailed Implementation

[0035] The embodiments for implementing this disclosure are described with reference to the accompanying drawings. Furthermore, in the drawings, identical or equivalent parts are labeled with the same reference numerals, and repetitive descriptions are simplified or omitted where appropriate.

[0036] Implementation Method 1

[0037] Figure 1 This is a block diagram illustrating the general structure of the autonomous mobile body application system according to Embodiment 1. Figure 2 This is an image of the transport robot. Figure 3 This is a block diagram showing the general structure of the transport robot. Figure 4 This is an image of the robot vacuum cleaner. Figure 5 This is a block diagram illustrating the general structure of a cleaning robot. Additionally, it is assumed that this autonomous mobile system is being implemented in the offices of a company consisting of multiple floors.

[0038] exist Figure 1 In this system, the robot application system 1, which serves as an autonomous mobile body application system, mainly consists of an application device 200 and multiple robots 100 that function as autonomous mobile bodies. Additionally, in... Figure 1 The text records three robots: A100a, B100b, and C100c. However, there could be more than three robots.

[0039] like Figure 2 , Figure 3 As shown, robot A100a is a transport robot.

[0040] in addition, Figure 2 (a) is the front view of robot A100a. Figure 2 (b) is a side view of robot A100a.

[0041] The robot A100a is mainly composed of a head 101a, a main body 102a, and a driving unit 103a. The driving unit 103a has four tires 103a1, two on the left and two in the front and two in the back. Moreover, the robot A100a can move by rotating these tires 103a1.

[0042] Furthermore, a camera unit 104a is provided on the head 101a. The head 101a is capable of rotating in the horizontal direction. Thus, the camera unit 104a can perform 360-degree horizontal shooting.

[0043] like Figure 2 As shown in (b), the main body 102a becomes a two-tiered shelf 105a. Employees who wish to move goods to other departments or floors place documents and goods here. Furthermore, a monitor 106a is installed on the opposite side of the shelf 105a. In addition to display functions, the monitor 106a also has input functions. For example, employees wishing to move goods can input the destination using characters.

[0044] The robot A100a also includes an ambient sensor 107a, a control unit 108a, a position detection unit 109a, a transceiver unit 110a, a battery 111a, a speaker 112a, a microphone 113a, and a transport management unit 114a. The ambient sensor 107a includes contact sensors, infrared sensors, ultrasonic sensors, radar sensors, etc.

[0045] The control unit 108a receives images from the imaging unit 104a and detection information from the surrounding sensors 107a. Furthermore, the robot A100a uses the position detection unit 109a to determine its current position and controls the driving unit 103a for autonomous driving. In addition, the control unit 108a controls the generation of voice from the speaker 112a. The control unit 108a also collects sound input from the microphone 113a. Finally, it performs overall control to execute commands sent from the application device 200.

[0046] The transceiver unit 110a communicates with external devices. Additionally, the position detection unit 109a is, for example, a GPS device.

[0047] like Figure 4, Figure 5 As shown, robot B100b is a cleaning robot.

[0048] in addition, Figure 4 (a) is the front view of robot B100b. Figure 4 (b) is a side view of robot B100b.

[0049] The robot B100b is mainly composed of a main body 102b and a driving unit 103b. The driving unit 103b has four tires 103b1, two on the left and two in the front and two in the back. The robot B100b can move by rotating these tires 103b1. Furthermore, a camera unit 104b is provided on the main body 102b.

[0050] A monitor 106b is provided on the upper surface of the main body 102b. In addition to display functions, the monitor 106b also has an input function. Furthermore, a cleaning unit 115b is provided on the lower surface of the main body 102b. This cleaning unit 115b sucks up debris and other debris from the ground and stores it in the debris collection section of the main body 102b.

[0051] Robot B100b also includes a surrounding sensor 107b, a control unit 108b, a position detection unit 109b, a transceiver unit 110b, a battery 111b, a speaker 112b, and a microphone 113b. Furthermore, their structure is similar to... Figure 3 The robot described is the same as A100a.

[0052] In addition, the C100c robot is the same cleaning robot as the B100b robot.

[0053] exist Figure 1 The application device 200 includes a transceiver unit 201, an image / sound analysis unit 202, a person location storage unit 203, an environment storage unit 204, an inquiry analysis unit 205, a collaboration processing unit 206, a response production unit 207, a response database 208, a staff database 209, and a feedback processing unit 210.

[0054] The transceiver unit 201 receives image and sound information from the robot 100, and sends and receives response and reply instructions to the robot 100.

[0055] The image / sound analysis unit 202 analyzes the image and sound information sent from the robot 100. Specifically, the image / sound analysis unit 202 extracts the person from the image information. It then compares the person with the person photo data stored in the employee DB209. Finally, the image / sound analysis unit 202 identifies the person, associates it with the employee name, and stores its location in the person location storage unit 203.

[0056] Figure 6 This refers to the employee information stored in Employee DB209. Employee DB209 also stores employee name data 209a, photo data 209b, department data 209c, skill data 209d, and performance status data 209e. Skill data 209d includes past work experience, qualifications, and professional knowledge. Performance status data 209e includes information such as emails, phone calls, and meeting attendance within a recent period. Employee DB209 is a type of personnel database.

[0057] The image / sound analysis unit 202 extracts characteristic content such as objects from the images and stores it in the environment storage unit 204. For example, it stores special products that are only located in this unit.

[0058] When the image / sound analysis unit 202 extracts human speech from the sound information sent by the robot 100, it sends the speech information and image information to the inquiry analysis unit 205.

[0059] The inquiry analysis unit 205 analyzes the inquiries contained in the image and audio information. First, specifically, it extracts the content of the inquiries contained in the audio information. Furthermore, based on the image information, it uses a staff database to identify the person who made the inquiry. Then, if the inquiry can be answered immediately, it is sent to the response production unit 207. The response production unit 207 generates a response. If a response cannot be answered immediately, it is sent to the collaboration processing unit 206.

[0060] Figure 7 This is a block diagram showing the general structure of the collaboration processing unit 206. The collaboration processing unit 206 consists of a staff selection unit 206a, a robot selection unit 206b, a response instruction generation unit 206c, and a collaboration response generation unit 206d. The staff selection unit 206a is an example of a personnel selection unit. The robot selection unit 206b is an example of an autonomous mobile body selection unit. The collaboration processing unit 206 identifies the robot 100 to which it should collaborate and generates a response instruction. Furthermore, it processes the responses from the robots 100 to which it should collaborate.

[0061] The response database 208 stores models for processing in the inquiry analysis unit 205, the collaborative processing unit 206, and the response production unit 207, as well as past processing conditions. The inquiry analysis unit 205, the collaborative processing unit 206, and the response production unit 207 appropriately refer to the response database 208 for processing.

[0062] The feedback processing unit 210 processes the feedback information from the inquirer and reflects the feedback. Specifically, it corrects the staff DB209. In addition, it corrects the analysis processing in the inquiry analysis unit 205 and the collaboration processing in the collaboration processing unit 206. Furthermore, feedback content from past processing situations is stored in the response DB208.

[0063] Next, according to Figure 8 The application flowchart of the robot application system 1 Figure 1 The robot's actions using System 1 will be explained.

[0064] Robot B100b performs cleaning as its primary task (step S001). During this time, image information captured by camera 104b and audio information acquired by microphone 113b are transmitted to application device 200 via transceiver 110b. Similarly, robot A100a performs transport as its primary task (step S201). Image and audio information are also transmitted to application device 200. Although not shown, robot C100c also performs cleaning as its primary task.

[0065] The image / sound analysis unit 202 of the application device 200 acquires image and sound information from robots A100a, B100b, and C100c via the transceiver unit 201. Then, the image / sound analysis unit 202 analyzes the information, stores the person's information in the person's location storage unit 203, and stores the characteristic information in the environment storage unit 204 (step S101).

[0066] In this state, employee K, who is equivalent to the first person on the floor where robot B100b is located, verbally conveys an inquiry unrelated to cleaning, namely the first inquiry, to robot B100b. Robot B100b handles this inquiry through microphone 113b (step S002). Since it is known within the company that robots can handle employee inquiries, employees can easily make their questions.

[0067] Additionally, the question could be phrased as "Please tell me ++++". Furthermore, it could be "Where is ++++?"

[0068] The image / sound analysis unit 202 receives the inquiry from robot B100b via the transceiver unit 201 and extracts it as voice information. Then, the inquiry analysis unit 205 receives and analyzes the inquiry and identifies the content of the inquiry (step S102).

[0069] The inquiry analysis unit 205 determines whether the inquiry can be answered (step S103). For example, if the inquiry is "What is the phone number of ***'s department?", it can be answered using employee DB209. Furthermore, if the inquiry is "Where is ++++?", it can be answered using the environment storage unit 204.

[0070] However, questions that are not held as data (such as those related to work within the company) cannot be answered. For example, "the progress of Project A," etc. However, such inquiries can be answered by professionals in the relevant field.

[0071] If, in step S103, it is determined that an answer can be given, the inquiry is sent to the answer generation unit 207. In the answer generation unit 207, an answer is generated (step S104). Then, an answer instruction is sent to robot B100b via the transceiver unit 201 (step S105). The answer instruction refers to responding to the inquirer, employee K, with the generated answer content.

[0072] If the query is determined to be unanswerable in step S103, it is sent to the collaboration processing unit 206. In the collaboration processing unit 206, the staff selection unit 206a selects a staff member who can respond (hereinafter referred to as "responding staff member") based on staff member DB209 (step S106: staff member selection step).

[0073] Furthermore, the statements in the query may not match those in skill data 209d. Also, sometimes multiple people cannot be selected using only skill data 209d without proper filtering. Therefore, the employee selection department 206a uses a knowledge graph for selection. The model of this knowledge graph is stored in the application DB208 for use.

[0074] As a method, it is described below. Figure 9 This is a schematic diagram.

[0075] • Extract characteristic keywords from the query content. For example, let's call it α.

[0076] • Calculate and extract the degree of homogeneity between the department data 209c, skill data 209d, and business status data 209e possessed by employee DB209 and the keyword α of the query content. Additionally, connect each employee with respect to the query content using lines, with the strength of the association represented by the line thickness.

[0077] In this case, the lines are thickened when the projects undertaken by both parties have a high degree of similarity. For example, this can be determined based on the content of technical documents in business status data 209e, whether they attended meetings together, etc. Furthermore, in business status data 209e, lines are thickened when there are many emails, phone calls, or conversations.

[0078] Then, regarding the inquiry, those with the highest total number of lines (thickness and thinness) are estimated as central figures relevant to the expectation, and are prioritized to select candidates. Additionally, under equal conditions, proficiency is used to determine merit. Proficiency is calculated based on job experience (skill data 209d) and responses to inquiries to date. Furthermore, in Figure 9 In this context, people with darker colors tend to be more skilled.

[0079] For example, if it is Figure 9 Based on the relevant graph, the candidates with knowledge capable of responding to the keyword α in the query are in the order X (3 items) > W (2 items and more proficient than Y) > Y (2 items) > Z (1 item). Then, in step S104, employee X is selected as the responding employee. That is, employee X is equivalent to the second person.

[0080] Next, the robot selection unit 206b selects the robot corresponding to the responding employee X (step S107: autonomous moving body selection step). This is the robot closest to the current position of the responding employee X. In the personnel location storage unit 203, information about where the employee is now is stored based on the image captured by the robot 100.

[0081] For example, suppose that the image captured by the camera unit 104a of robot A100a shows the corresponding employee X, and this image is stored in the personnel location storage unit 203. In this case, robot A100a is selected as the corresponding robot.

[0082] If there is no information about employee X in the personnel location storage unit 203, the location is determined based on the surveillance cameras (not shown) located on the floor. In this case, the robot 100 closest to the employee X captured by the surveillance camera is selected as the corresponding robot. Furthermore, if employee X is not shown on the surveillance camera, the cleaning robot from the department to which employee X belongs is selected.

[0083] Next, the response instruction production unit 206c produces a response instruction for robot A100a. Then, the response instruction is sent via the transceiver unit 201 (step S108: instruction step). This response instruction refers to going to the response staff member X, issuing a second question based on the first question, and receiving a response. Alternatively, the second question here could be the question itself received from staff member K in step S002. Furthermore, the skills of the response staff member X can be considered to reconstruct the content of the first question in a way that the response staff member X can understand.

[0084] Robot A100a receives response instructions using transceiver unit 110a (step S202).

[0085] Next, robot A100a interrupts its main operations and conducts a second inquiry with employee X (step S203). Specifically, robot 100a, having received the response instruction, first confirms the presence of employee X. If employee X is in conversation with someone or in a meeting, it waits. Based on the image captured by camera 104a, control 108a determines the action.

[0086] Robot A100a, observing that employee X is available, moves to his side. Then, it issues a question via voice through speaker 112a. For example, "Employee K would like to hear about the progress of project A. What is the status?"

[0087] In response to this inquiry, employee X should verbally answer. That is, robot A100a uses microphone 113a to acquire the voice of the first answer. Then, the voice information of the answer is sent to application device 200 via transceiver 110a (step S204). Then, the interrupted main service is restarted (step S205).

[0088] In the application device 200, the collaborative response generation unit 206d of the collaborative processing unit 206 receives the response voice information. Then, a response instruction is generated (step S109). The response instruction refers to a second response to the employee K, who is the inquirer, based on the first response received from the responding employee X. Alternatively, the second response here may be the response received from the responding employee X in step S204 itself. Furthermore, the content of the first response may be reconstructed in a way that the employee K can understand. Then, the response instruction is sent to the robot B100b via the transceiver unit 201 (step S105: response instruction step).

[0089] After receiving the inquiry in step S002, robot B100b resumes its primary task of cleaning (step S003). This is because a response may not be immediately received from the application device 200.

[0090] In this state, in robot B100b, control unit 108b receives response instructions via transceiver unit 110b (step S004).

[0091] Robot B100b interrupts its main business and moves to the vicinity of employee K, who is the inquirer, and responds via voice from speaker 112b (step S005).

[0092] Then, robot B100b receives feedback from employee K, who is the inquirer (step S006). For example, sometimes employee K is satisfied with the answer and expresses gratitude. Or, sometimes employee K expresses dissatisfaction with the answer. Or, sometimes there are subtle nuances in tone, such as asking follow-up questions or saying "next time."

[0093] In the case of voice representation, the feedback is sent as audio information to the application device 200. In the application device 200, the feedback content is sent from the image / audio analysis unit 202 to the feedback processing unit 210 for processing (step S110). For example, it may be reflected in the skill data 209d of the respondent X in the employee DB209. Alternatively, it may be used to revise the model of the knowledge graph in the respondent DB208. Or, it may be reflected in the logic of the respondent selection in the collaboration processing unit 206. Thus, the response content is upgraded in cases of similar inquiries.

[0094] Thus, in Implementation Method 1, robot B100b, through cooperation with robot A100a, can respond to human inquiries beyond its own operational area. This enables more efficient utilization of robots primarily used for cleaning and transporting, further promoting their widespread adoption.

[0095] Furthermore, if only the name and contact information of a skilled employee are provided to the inquirer, for example, even if the inquirer asks via email or chat, no answer can be obtained if the inquirer does not read the message. In contrast, in Implementation 1, the robot receives answers directly from the responding employee, thus enabling a rapid response to inquiries.

[0096] Alternatively, the inquiry in step S203 may not be spoken, but displayed on monitor 106a. In this case, the answer in step S204 is also input as characters into monitor 106a.

[0097] Furthermore, when responding in step S005, the name of the responding employee X can also be given. For example, "This response was obtained with the assistance of employee X," etc. If employee X provided assistance, then giving their name becomes an opportunity to create a synergy.

[0098] In addition to cleaning robots and transport robots, robots can also be nursing robots, guidance robots, medical robots, etc., with their own main functions.

[0099] Furthermore, AI can also be used in the inquiry analysis department 205, the collaborative processing department 206, the response production department 207, and the response database 208. By learning from numerous examples and then relearning based on feedback, the inference ability to provide answers is improved, enabling more appropriate responses.

[0100] Furthermore, in step 106, multiple responders can also be selected. For example, in Figure 8 In the candidate list, the order is: Employee X, Employee W, Employee Y. Accuracy is improved by using responses from multiple employees. Furthermore, even if there are employees whose responses are unavailable, the system can still respond using the responses from the remaining employees.

[0101] Alternatively, text-based similarity calculation can be used as an option for step S106. This involves converting the query content into text and transforming it into a vector representation to calculate the similarity. Figure 10 This is a flowchart illustrating the process when similarity calculation is used. Furthermore, the similarity calculation model is stored for use with DB208.

[0102] In the employee selection unit 206a, the query analyzed in step S102 is textualized and a text judgment is performed (step S1061). Next, vector analysis of the query content is performed (step S1062). Then, the similarity between the query after vector analysis and the skill data 208d of employee DB209 is calculated using the concept of cosine similarity or Euclidean distance (step S1063). Then, the candidates are ranked according to the similarity, and the person at the top is set as the respondent employee (step S1064). For example, if the similarity is X, W, Y, Z, then X becomes the respondent employee. In addition, steps S1061 to S1064 correspond to step S106.

[0103] Furthermore, feedback is provided in step S004, therefore, the feedback is collected (step S1101) and saved (step S1102) in the feedback processing unit 210. Then, the model of the response DB208 is reconstructed or fine-tuned (step S1103). In addition, past response performance and information of the responding staff related to the feedback are retrieved (step S1104), and text judgment of the current inquiry content is performed (step S1105). Then, based on the text judgment and the model, the vector representation of the skill data 209d is corrected (step S1106), which is reflected in the similarity calculation in the subsequent step S106. In addition, steps S1101 to S1105 correspond to step S110.

[0104] Figure 11 This diagram illustrates an example of the hardware resources of the application device 200. The application device 200 is a computer that includes a processor 200a, a memory 200b, and a transceiver circuit 200c as hardware resources. Alternatively, there may be multiple processors 200a, memory 200b, and transceiver circuits 200c.

[0105] In Embodiment 1, the image / sound analysis unit 202, the inquiry analysis unit 205, the collaboration processing unit 206, the response generation unit 207, and the feedback processing unit 210 are executed by the processor 200a using software, firmware, or a combination of software and firmware stored in the memory 200b. The transceiver circuit 200c is responsible for acquiring information in the transceiver unit 201. The memory 200b is responsible for the personnel location storage unit 203, the environment storage unit 204, the response database 208, and the staff database 209.

[0106] Additionally, the processor 200a is also referred to as a CPU (Central Processing Unit), central processing unit, arithmetic unit, microprocessor, or DSP. The memory 200b can also be a semiconductor memory, disk, floppy disk, optical disk, high-density disk, mini-disk, or DVD. Suitable semiconductor memories include RAM, ROM, flash memory, EPROM, and EEPROM.

[0107] Figure 12 This diagram illustrates another example of the use of hardware resources in device 200. Figure 12 In the example, the application device 200 has processing circuitry, which includes a processor 200a, a memory 200b, a transceiver circuitry 200c, and dedicated hardware 200d. Furthermore, a portion of the functions of the application device 200 are implemented by the dedicated hardware 200d. Alternatively, all the functions of the application device 200 can be implemented by the dedicated hardware 200d. The dedicated hardware 200d can be a single circuit, a composite circuit, a programmable processor, a parallel programmable processor, an ASIC, an FPGA, or a combination thereof.

[0108] Implementation Method 2

[0109] In implementation method 2, the robot checks whether a person is experiencing distress. This is primarily intended for locations frequented by unspecified individuals, such as hospitals or amusement parks. Figure 13 This is a flowchart illustrating the operation of robot application system 1 in implementation method 2. Additionally, in Figure 13 In China, regarding and Figure 8 For identical or similar treatments, use the same labels and omit or simplify the descriptions.

[0110] While performing its primary cleaning task in step S001, robot B100b acquires images of the surrounding environment via its camera unit 104b. If robot B100b detects a person engaging in distressing behavior in the images, it approaches the person to confirm their distress (step S007). For example, it may ask, "Are you troubled?" from speaker 112b. If the person responds to this question, further interaction can proceed. Figure 6 The same procedures apply. For example, questions like "How do I get to ***?" or "What are the procedures for ***?"

[0111] In addition, behaviors that cause confusion include standing and looking around, walking back and forth in the same place, and not moving in front of the signboard.

[0112] Processing after step S007 and Figure 8 The same applies. However, after step S007, robot B100b does not restart the main operation. This is because, if the main operation were to restart, an unfamiliar inquirer might go somewhere else.

[0113] In this way, by starting with communication with robots, services can be improved in places where there are no specific people.

[0114] Furthermore, Implementation Method 2 can also be applied to a company's office. In this case, for example, in step S007, the message "There is someone who can solve your problem. Assistance or inquiry?" is delivered via voice.

[0115] Implementation Method 3

[0116] In implementation 3, other robots are made to perform actions other than questioning. Figure 14 This is a block diagram illustrating the general structure of the robot application system 1 in Embodiment 3. It is assumed here to be a hospital.

[0117] exist Figure 14 In addition, a robot D100d was added.

[0118] like Figure 15 , Figure 16 As shown, the D100d robot is a humanoid nursing robot. Additionally, Figure 15 (a) is the front view of robot D100d. Figure 15 (b) is a side view of D100d.

[0119] The robot D100d is mainly composed of a head 101d, a main body 102d, and a walking part 103d. The walking part 103d has two feet 103d1, one on the left and one on the right. Moreover, it can move by moving these feet 103d1.

[0120] Furthermore, the head 101d is equipped with two left and right camera units 104d. The head 101d can rotate horizontally. Thus, the camera units 104d can capture images 360 degrees horizontally.

[0121] The main body 102d has arms 102d1 on the left and right sides. Fingers 102d2 are mounted on these arms 102d1, enabling them to grasp objects. Furthermore, a monitor 106d is provided on the rear side.

[0122] The robot D100d also includes a surrounding sensor 107d, a control unit 108d, a position detection unit 109d, a transceiver unit 110d, a battery 111d, a speaker 112d, and a microphone 113d. Furthermore, their structure is similar to... Figure 3 The robot described is the same as A100a.

[0123] exist Figure 14 In the process, the device 200 has a robot DB (database) 211. Figure 17 This refers to the robot information stored by robot DB211. Robot DB211 stores identifier data 211a, category data 211b, responsible location data 211c, and function data 211d. Furthermore, since it is assumed to be a hospital, the staff of employee DB209 are doctors and nurses.

[0124] Next, according to Figure 18 The application flowchart of the robot application system 1 Figure 14 The robot's actions using System 1 will be explained. Additionally, in Figure 18 In China, regarding and Figure 8 For identical or similar treatments, use the same labels and omit or simplify the descriptions.

[0125] Robot B100b performs cleaning as its primary task (step S001). During this period, image information captured by camera 104b and audio information obtained by microphone 113b are transmitted to application device 200 via transceiver 110b.

[0126] Similarly, robots A100a and D100d perform their main tasks (step S301). That is, robot A100a delivers food and other necessities to the ward, while robot D100d provides patient care in the ward.

[0127] In this state, a patient in the ward that robot B100b is cleaning verbally inquires about a matter unrelated to cleaning. Robot B100b receives the inquiry via microphone 113b (step S002). For example, the inquiries might include phrases like, "My stomach hurts. I need to go to the bathroom. Call the nurse."

[0128] The inquiry analysis unit 205 receives inquiries from robot B100b via the receiving and dispatching unit 201 and the image / sound analysis unit 202, and analyzes the content of the inquiries (step S102).

[0129] Then, the query analysis unit 205 determines whether it can handle the situation (step S111). If it can handle the situation, the steps after step S105 are performed.

[0130] However, if the content is like this, it cannot be handled. If it is determined in step S111 that it cannot be handled, the staff selection unit 206a extracts a staff member who can handle the content (here, a nurse who can handle the content) based on staff member DB209 (hereinafter referred to as "handling nurse") (step S106).

[0131] Next, robot selection unit 206b selects the robot corresponding to the responding nurse and selects the robot that will directly take action (step S112). This selects the robot closest to the responding nurse's current location. Additionally, a nursing robot that assists the patient to the restroom is selected. Furthermore, the selection of the nursing robot is based on the responsible location data 211c and functional data 211d of robot DB211. In this process, robot A100a is selected as the robot that notifies the responding nurse. Additionally, robot D100d is selected as the robot that assists the patient.

[0132] Next, the response instruction generation unit 206c generates a response instruction and sends it to the selected robot (step S113). For example, because a patient tells the response nurse that they are feeling unwell, the instruction robot A100a asks whether to go to the patient's room. Furthermore, the instruction robot D100d goes to the patient's room to assist them.

[0133] In contrast, robots A100a and D100d receive response instructions (step S302) and respond accordingly (step S303). For example, robot A100a asks the responding nurse, "XX is feeling unwell. Should we go to the ward immediately?" Meanwhile, robot D100d goes to the ward.

[0134] Next, the robot D100d receives feedback from the patient (step S304), and the feedback processing unit 210 reflects the content of the feedback (step S114).

[0135] In this way, different instructions can be issued to multiple robots to respond to a single query, thereby enabling faster processing.

[0136] Additionally, in step S002, robot B100b can also generate inquiries based on the patient's condition. For example, it can identify if the patient is in pain in bed, generate an inquiry such as "Pain. Call a doctor," and send it to the application device 200, etc.

[0137] Implementation Method 4

[0138] In Implementation 4, if a staff member cannot be selected in step S106, response instructions are issued to multiple robots simultaneously. Figure 19This is a flowchart illustrating the operation of robot application system 1 in implementation method 4. Additionally, in Figure 19 In China, regarding and Figure 8 For identical or similar treatments, use the same labels and omit or simplify the descriptions. Here, we assume a construction site such as a building. Furthermore, we assume robot B100b is a material transport robot for construction.

[0139] In step S002, robot B100b receives an inquiry from the operator. For example, suppose the inquiry relates to the usage of newly introduced machinery.

[0140] In step S103, no answer can be given. Therefore, in step S106, the employee selection unit 206a selects a suitable employee. However, if there is no information related to the newly imported machine in the employee's skill data 208d, a suitable employee cannot be selected.

[0141] In this case, the robot selection unit 206b selects one or more arbitrary robots, and if possible, selects all robots on the construction site (step S115). Then, the response instruction generation unit 206c sends a response instruction such as a voice inquiry (step S116). Here, let's say robot A100a is selected.

[0142] In step S202, robot A100a receives the response instruction. Then, in step S203, it emits a voice from speaker 112a inquiring about whether there are any people nearby who know how to use the newly introduced machinery. Then, it obtains a response from nearby staff who are able to answer.

[0143] In this way, when it is impossible to select an employee who can answer the question, questions can be sent from other robots to employees in the vicinity and answers can be obtained, thereby improving the response rate.

[0144] Implementation Method 5

[0145] In enclosed spaces like hospitals, patients still yearn to communicate with each other. However, communicating with people who have different interests or ideas can be tedious and sometimes even painful. To address this, in implementation 5, data is accumulated in a patient database (DB), a type of person database, based on information obtained by the robot. Then, it becomes possible to respond to inquiries from patients.

[0146] Figure 20 This is a block diagram showing the general structure of the robot application system 1 in embodiment 5. Additionally, a DB storage unit 212 and a patient DB 213 are added here. Other than this, the structure is similar to... Figure 1 Because they are the same and identical, the same labels are used, and the descriptions are omitted or simplified.

[0147] Figure 21 This refers to the patient information stored in patient DB213. Patient DB213 also stores patient name data 213a, gender data 213b, photo data 213c, ward data 213d, skill data 213e, and personality data 213f.

[0148] Next, according to Figure 22 The data collection / accumulation flowchart illustrates the data collection / accumulation for patient DB213.

[0149] First, the patient name data 213a, gender data 213b, photo data 213c, and ward data 213d in patient DB213 are data held by the hospital. Therefore, they have been preserved from the beginning.

[0150] In this state, robot 100 performs its main tasks while acquiring image and sound information, which is received by application device 200 (step S401). In application device 200, image / sound analysis unit 202 analyzes the image and sound information. Then, DB accumulation unit 212 receives voice and image information from image / sound analysis unit 202 and analyzes it based on the patient DB 213 photo data 213c (step S402). Next, based on each patient's voice and image, the patient is identified, and words related to the patient's skills are extracted (step S403). Then, the extracted skills are accumulated as skill data 213e (step S404).

[0151] Furthermore, personality is inferred based on the tone of the image and voice (step S405). Then, the inferred personality data is accumulated as personality data 213f (step S406). As personality, words that affect communication with people, such as extroverted, irritable, and easygoing, are used to represent it. It is determined whether the robot has finished its main task (step S407). If it has not finished, the steps after step S401 are performed. By continuing this process, data is accumulated in the patient DB213.

[0152] Next, according to Figure 23 The flowchart of the robot application system 1 illustrates the actions of the robot application system 1 when the patient DB213 is used. Additionally, Figure 23 In China, regarding and Figure 8 For identical or similar treatments, use the same labels and omit or simplify the descriptions.

[0153] First, in step S002, robot B100b receives inquiries from patients. Let the content of these inquiries be related to other patients' knowledge and communication. For example, it might be, "Are there any patients interested in ***?"

[0154] In this case, the query analysis unit 205 cannot answer in step 103. The collaborative processing unit 206, which has a patient selection unit (not shown) with the same functions as the staff selection unit 206a, selects a suitable patient (step S117). The selection is primarily based on the patient DB212's skill data 212e. Additionally, gender data 212b and personality data 212f are also considered. Furthermore, the patient selection unit is also an example of a personnel selection unit.

[0155] Therefore, if a patient can be selected, the steps after step S107 are executed. In step S203, the patient is questioned. Furthermore, let's assume the question in step S002 is something like, "Are there any patients interested in ***?" The question in step S203 could be, for example, "The patient is looking for someone interested in ***, could you introduce them?"

[0156] Then, obtain a response from the responding patient regarding the inquiry, such as "OK," and proceed with the steps after S204. Additionally, if the response from the responding patient is affirmative, the response in step S005 may be something like, "The ward *** has the same interests and wishes to communicate," etc.

[0157] Therefore, even without accurate information such as skills, it is possible to respond to inquiries by frequently collecting and accumulating information from robots moving around people.

[0158] Furthermore, during the collection and storage of patient data, if permitted, a communication connection can be established when the robot 100 approaches a patient's smartphone or other portable terminal. Moreover, patient-related information can be obtained from the portable terminal, and the data can be stored in the patient's DB213.

[0159] Furthermore, if patients DB213 share common skills, in step S002, robot 100 may suggest that the patient communicate with other patients. For example, "There are people with common interests. Shall we communicate?"

[0160] The preferred embodiments have been described in detail above. However, various modifications and substitutions can be made to the above embodiments without departing from the scope of disclosure, without being limited by these embodiments.

[0161] Furthermore, when the number, quantity, amount, range, etc., of each element are mentioned in the embodiments, the apparatus of this disclosure is not limited to the mentioned quantities, unless specifically stated or clearly determined in principle. Moreover, the structures described in these embodiments are not essential, unless specifically stated or clearly determined in principle.

[0162] The various methods disclosed herein will be uniformly referred to as appendices below.

[0163] (Postscript 1)

[0164] An autonomous mobile entity application system, comprising multiple autonomous mobile entities and an application device capable of communicating with the multiple autonomous mobile entities, characterized in that...

[0165] The application device has a cooperation processing unit that, when receiving a first inquiry from a first autonomous mobile body, which is one of the plurality of autonomous mobile bodies, selects a second person capable of responding to the first inquiry based on the first inquiry, and instructs the second autonomous mobile body, which is one of the plurality of autonomous mobile bodies different from the first autonomous mobile body, to conduct a second inquiry on the second person based on the first inquiry.

[0166] (Postscript 2)

[0167] The autonomous mobile body application system according to Appendix 1 is characterized in that,

[0168] When the collaborative processing unit receives the first answer from the second person to the second inquiry from the second autonomous mobile body, it instructs the first autonomous mobile body to give a second answer to the first person based on the first answer.

[0169] (Note 3)

[0170] The autonomous mobile body application system according to Appendix 1 or 2 is characterized in that,

[0171] The application device has a database of people whose names and skills are associated with each person, and the collaboration processing unit selects the second person based on the database.

[0172] (Postscript 4)

[0173] The autonomous mobile body operation system according to any one of Appendices 1 to 3 is characterized in that,

[0174] The autonomous mobile body has a camera unit, a voice generation unit, and a sound receiving unit.

[0175] (Note 5)

[0176] The autonomous mobile body application system according to Appendix 4 is characterized in that...

[0177] The application device determines the person based on the image information captured by the camera unit of the autonomous mobile body, and has a person location storage unit that stores the person and the location of the person in association. The cooperation processing unit selects the second autonomous mobile body based on the person location storage unit.

[0178] (Note 6)

[0179] The autonomous mobile body application system according to Appendix 5 is characterized in that,

[0180] The second autonomous mobile body has a camera unit that captures images of the second person stored in the person location storage unit.

[0181] (Note 7)

[0182] The autonomous mobile body application system according to Appendix 6 is characterized in that...

[0183] The first inquiry is obtained by the sound receiving unit of the first autonomous mobile body.

[0184] (Note 8)

[0185] The autonomous mobile body application system according to Appendix 7 is characterized in that,

[0186] The application device has a feedback processing unit that, when receiving feedback from the first autonomous mobile body based on the second response from the first person, causes the person database and the collaboration processing unit to reflect the feedback.

[0187] (Note 9)

[0188] The autonomous mobile body application system according to Appendix 8 is characterized in that...

[0189] The first inquiry is performed in correspondence with the speech generated by the speech generation unit of the first autonomous mobile body.

[0190] (Postscript 10)

[0191] The autonomous mobile body operation system according to any one of Appendices 1 to 9 is characterized in that,

[0192] While the cooperative processing unit issues the instruction to the second autonomous mobile body, it also instructs a third autonomous mobile body, which is one of the plurality of autonomous mobile bodies different from the first and second autonomous mobile bodies, to perform an action corresponding to the first inquiry.

[0193] (Postscript 11)

[0194] The autonomous mobile body application system according to Appendix 4 is characterized in that...

[0195] If the collaborative processing unit is unable to select the second person from the person database, it instructs at least one of the plurality of autonomous mobile entities that is different from the first autonomous mobile entity to issue a second inquiry based on the first inquiry from the speech generation unit.

[0196] (Postscript 12)

[0197] The autonomous mobile body application system according to Appendix 4 is characterized in that...

[0198] The application device has a database storage unit that stores data in the person database based on the image and sound information received by the shooting unit and the sound receiving unit of the autonomous mobile body.

[0199] (Postscript 13)

[0200] The autonomous mobile body application system according to Appendix 12 is characterized in that,

[0201] The database storage unit stores data in the person's database using information obtained from the person's portable terminal.

[0202] (Postscript 14)

[0203] The autonomous mobile body operation system according to any one of Appendices 1 to 13 is characterized in that,

[0204] The first autonomous mobile body and the second autonomous mobile body are any one of cleaning robots, transport robots, nursing robots, guiding robots, and medical robots.

[0205] (Postscript 15)

[0206] A method for utilizing autonomous mobile bodies, which employs multiple autonomous mobile bodies, is characterized by comprising the following steps:

[0207] The character selection step involves receiving a first inquiry from a first autonomous mobile body, which is one of the plurality of autonomous mobile bodies, and selecting a second character capable of responding to the first inquiry based on the first inquiry.

[0208] The autonomous mobile body selection step involves selecting a second autonomous mobile body as one of the plurality of autonomous mobile bodies that is different from the first autonomous mobile body; and

[0209] The instruction step instructs the second autonomous mobile body to conduct a second inquiry on the second person based on the first inquiry.

[0210] (Postscript 16)

[0211] The method for using an autonomous mobile body as described in Appendix 15 is characterized in that,

[0212] The autonomous mobile entity application method has the following response instruction steps: receiving a first response from the second person to the second inquiry from the second autonomous mobile entity, and instructing the first autonomous mobile entity to provide a second response to the first person based on the first response.

[0213] (Postscript 17)

[0214] A computer-readable recording medium containing an autonomous mobile device application program, characterized in that the autonomous mobile device application program causes a computer using multiple autonomous mobile devices to perform the following steps:

[0215] The character selection step involves receiving a first inquiry from a first autonomous mobile body, which is one of the plurality of autonomous mobile bodies, and selecting a second character capable of responding to the first inquiry based on the first inquiry.

[0216] The autonomous mobile body selection step involves selecting a second autonomous mobile body as one of the plurality of autonomous mobile bodies that is different from the first autonomous mobile body; and

[0217] The instruction step instructs the second autonomous mobile body to conduct a second inquiry on the second person based on the first inquiry.

[0218] (Postscript 18)

[0219] The computer-readable recording medium according to Appendix 17, which records an application program for an autonomous mobile body, is characterized in that...

[0220] The autonomous mobile entity application causes the computer to execute the following response instruction steps: receiving the first response from the second person to the second inquiry from the second autonomous mobile entity, and instructing the first autonomous mobile entity to provide a second response to the first person based on the first response.

Claims

1. An autonomous mobile entity application system, comprising multiple autonomous mobile entities and an application device capable of communicating with the multiple autonomous mobile entities, characterized in that, The application device has a cooperation processing unit that, when receiving a first inquiry from a first autonomous mobile body, which is one of the plurality of autonomous mobile bodies, selects a second person capable of responding to the first inquiry based on the first inquiry, and instructs the second autonomous mobile body, which is one of the plurality of autonomous mobile bodies different from the first autonomous mobile body, to conduct a second inquiry on the second person based on the first inquiry.

2. The autonomous mobile body application system according to claim 1, characterized in that, When the collaborative processing unit receives the first answer from the second person to the second inquiry from the second autonomous mobile body, it instructs the first autonomous mobile body to give a second answer to the first person based on the first answer.

3. The autonomous mobile body application system according to claim 2, characterized in that, The application device has a database of people whose names and skills are associated with each person, and the collaboration processing unit selects the second person based on the database.

4. The autonomous mobile body application system according to claim 3, characterized in that, The autonomous mobile body has a camera unit, a voice generation unit, and a sound receiving unit.

5. The autonomous mobile body application system according to claim 4, characterized in that, The application device determines the person based on the image information captured by the camera unit of the autonomous mobile body, and has a person location storage unit that stores the person and the location of the person in association. The cooperation processing unit selects the second autonomous mobile body based on the person location storage unit.

6. The autonomous mobile body application system according to claim 5, characterized in that, The second autonomous mobile body has a camera unit that captures images of the second person stored in the person location storage unit.

7. The autonomous mobile body application system according to claim 6, characterized in that, The first inquiry is obtained by the sound receiving unit of the first autonomous mobile body.

8. The autonomous mobile body application system according to claim 7, characterized in that, The application device has a feedback processing unit that, when receiving feedback from the first autonomous mobile body based on the second response from the first person, causes the person database and the collaboration processing unit to reflect the feedback.

9. The autonomous mobile body application system according to claim 8, characterized in that, The first inquiry is performed in correspondence with the speech generated by the speech generation unit of the first autonomous mobile body.

10. The autonomous mobile body application system according to claim 1, characterized in that, While the cooperative processing unit issues the instruction to the second autonomous mobile body, it also instructs a third autonomous mobile body, which is one of the plurality of autonomous mobile bodies different from the first and second autonomous mobile bodies, to perform an action corresponding to the first inquiry.

11. The autonomous mobile body application system according to claim 4, characterized in that, If the collaborative processing unit is unable to select the second person from the person database, it instructs at least one of the plurality of autonomous mobile entities that is different from the first autonomous mobile entity to issue the second inquiry based on the first inquiry from the speech generation unit.

12. The autonomous mobile body application system according to claim 4, characterized in that, The application device has a database storage unit that stores data in the person database based on the image and sound information received by the shooting unit and the sound receiving unit of the autonomous mobile body.

13. The autonomous mobile body application system according to claim 12, characterized in that, The database storage unit stores data in the person's database using information obtained from the person's portable terminal.

14. The autonomous mobile body application system according to any one of claims 1 to 13, characterized in that, The first autonomous mobile body and the second autonomous mobile body are any one of cleaning robots, transport robots, nursing robots, guiding robots, and medical robots.

15. A method for using autonomous mobile bodies, comprising using multiple autonomous mobile bodies, characterized in that, The autonomous mobile body application method includes the following steps: The character selection step involves receiving a first query from a first autonomous mobile body, which is one of the plurality of autonomous mobile bodies, and selecting a second character capable of responding to the first query based on the first query. The autonomous mobile body selection step involves selecting a second autonomous mobile body as one of the plurality of autonomous mobile bodies that is different from the first autonomous mobile body. as well as The instruction step instructs the second autonomous mobile body to conduct a second inquiry on the second person based on the first inquiry.

16. The method for using an autonomous mobile body according to claim 15, characterized in that, The autonomous mobile entity application method has the following response instruction steps: receiving a first response from the second person to the second inquiry from the second autonomous mobile entity, and instructing the first autonomous mobile entity to provide a second response to the first person based on the first response.

17. A computer-readable recording medium containing an application program for an autonomous mobile entity, characterized in that, The autonomous mobile unit application causes a computer using multiple autonomous mobile units to perform the following steps: The character selection step involves receiving a first inquiry from a first autonomous mobile body, which is one of the plurality of autonomous mobile bodies, and selecting a second character capable of responding to the first inquiry based on the first inquiry. The autonomous mobile body selection step involves selecting a second autonomous mobile body as one of the plurality of autonomous mobile bodies that is different from the first autonomous mobile body. as well as The instruction step instructs the second autonomous mobile body to conduct a second inquiry on the second person based on the first inquiry.

18. The computer-readable recording medium according to claim 17, which records an application program for an autonomous mobile entity, is characterized in that... The autonomous mobile entity application causes the computer to execute the following response instruction steps: receiving the first response from the second person to the second inquiry from the second autonomous mobile entity, and instructing the first autonomous mobile entity to provide a second response to the first person based on the first response.

Images