A machine-human interaction system and method

Abstract

A kind of man-machine interaction system of robot, the weight and / or volume change information of article is collected by counting unit, the unloading article quantity and / or type of current destination is identified, the still or dynamic image of taking article body is collected by taking article body identification system, when the unloading article quantity and / or type identified by robot is inconsistent with the unloading article quantity and / or type preset in system, the still or dynamic image of taking article body collected can be compared with the taking article body data stored in system, it is judged whether taking article body is the taking article body corresponding to current destination or the taking article body from remaining destination, and according to the judgment result, the voice prompt device is used to issue the voice prompt of needing to carry out man-machine interaction to taking article side, and finally the exact abnormal taking article information is recognized by the man-machine interaction intervention of operating party.The abnormal taking article information that is different from setting procedure is recognized by robot in process article conveying by real-time man-machine interaction.

Description

Technical Field

[0001] This invention relates to the field of robot control application technology, specifically to a robot human-computer interaction system and method. Background Technology

[0002] In recent years, intelligent logistics has not only been widely used in professional logistics and warehousing services, but has also been developed in manufacturing and other real industries, promoting the transformation of industrial production towards greater efficiency and economy. By using logistics robots to replace workers in handling goods, not only is labor saved, but the planned and controllable transportation of goods can also be achieved, thereby integrating the transportation behavior of logistics robots with the production process and creating intelligent production lines.

[0003] Currently, in manufacturing production sites, feeding or supplying robots can basically assist production lines in the automatic transport and process positioning of materials or parts, and can also perform qualitative and quantitative identification of the transported materials or parts. However, for some abnormal events in the process of transporting goods, such as operators picking up parts across processes, the lack of automatic identification often disrupts the robot's established transport procedures and replenishment plans. Summary of the Invention

[0004] The purpose of this invention is to solve the problems mentioned in the background art and to propose a robot human-machine interaction system and method. Through real-time human-machine interaction, in the process of conveying items in a step-by-step manner, the robot is helped to identify abnormal item picking information that is different from the set program, so as to provide conditions for subsequent item conveying and replenishment planning.

[0005] The technical solution of the present invention is as follows: A robot human-machine interaction system is disclosed. The robot is a robot that performs the task of transporting items. The robot carries multiple items and transports them sequentially to multiple pick-up destinations according to a path program or running map. The robot is equipped with a counting unit that can identify the unloaded item information at each destination through changes in weight and / or volume. The robot is equipped with a pick-up subject recognition system, a voice prompt device, and a human-machine interaction interface. When the robot's counting unit identifies that the quantity and / or type of unloaded items at the current destination is inconsistent with the system's preset quantity and / or type of unloaded items, the robot uses the pick-up subject recognition system to identify whether the current pick-up subject is the pick-up subject corresponding to the current destination or a pick-up subject from another destination. Based on the recognition result, the robot uses the voice prompt device to issue a voice prompt to the pick-up party asking whether human-machine interaction is required. The human-machine interaction interface is used by the pick-up party to input abnormal pick-up information that can be recognized by the robot after performing an abnormal pick-up action.

[0006] A further aspect of the aforementioned robot human-computer interaction system is that the robot identifies each object retrieval destination based on location codes.

[0007] A further aspect of the aforementioned robot human-machine interaction system is that the counting unit includes a weight collector, which collects weight changes to allow the robot to identify the quantity and / or type of items to be unloaded at each destination.

[0008] A further aspect of the aforementioned robot human-machine interaction system is that the counting unit includes an image acquisition device or a contour recognition device, which collects volume changes to allow the robot to identify the quantity and / or type of unloaded items at each destination.

[0009] A further aspect of the aforementioned robot human-computer interaction system is that the object retrieval subject identification system includes a visual recognition system, which collects static or dynamic images of the object retrieval subject to help the robot identify whether the current object retrieval subject is the object retrieval subject corresponding to the current destination or the object retrieval subject from other destinations.

[0010] A further aspect of the aforementioned robot human-computer interaction system is that the human-computer interaction interface adopts a touch screen mode and / or a mechanical key mode, providing a button associated with the object retrieval destination and / or a button associated with the object retrieval subject, allowing the robot to identify abnormal object retrieval information through external human-computer interaction commands.

[0011] A human-computer interaction method based on the robot human-computer interaction system described above includes the following steps: S01: The robot performing the item delivery task carries multiple items and delivers them sequentially to multiple pick-up destinations according to the path program or running map. Each destination contains location code information. The robot identifies each pick-up destination based on the location code and retrieves the quantity and / or type of items to be picked up at each pick-up destination. At the current destination, the robot collects the weight and / or volume change information of the items through the counting unit, identifies the quantity and / or type of items to be unloaded at the current destination, and collects static or dynamic images of the pick-up subject through the pick-up subject identification system. When the quantity and / or type of items currently unloaded identified by the robot is consistent with the quantity and / or type of items pre-set by the system, the delivery task to the next destination is executed. When the quantity and / or type of items currently unloaded identified by the robot is inconsistent with the quantity and / or type of items pre-set by the system, the process proceeds to step S02. S02: The robot compares the static or dynamic image of the object-retrieving entity collected by the object-retrieving entity recognition system with the object-retrieving entity data stored in the system to determine whether the object-retrieving entity is the object-retrieving entity corresponding to the current destination or the object-retrieving entity from other destinations. Based on the determination result, the robot issues a voice prompt to the object-retrieving party through the voice prompt device, indicating that human-computer interaction is required. S03: The operator inputs abnormal object retrieval information into the robot through the touch screen and / or mechanical keys of the human-machine interface according to the voice prompts. The touch screen and / or mechanical keys provide an object retrieval destination association button and / or an object retrieval subject association button, so that the robot can identify the abnormal object retrieval information of the current destination through external human-machine interaction commands.

[0012] A further aspect of the aforementioned robot human-computer interaction method is that, in step S02, when the robot determines that the object-retrieving entity is the object-retrieving entity corresponding to the current destination, it issues a voice prompt to the object-retrieving party through a voice prompt device, requiring verification of the quantity and / or type of the object. After waiting for the object-retrieving party to confirm through the human-computer interaction interface or after multiple voice prompts, the robot executes the delivery task to the next destination. When the robot determines that the object-retrieving entity is the object-retrieving entity from another destination, it issues a voice prompt to the object-retrieving party through a voice prompt device, requiring verification of the address and / or identity. After waiting for the object-retrieving party to confirm through the human-computer interaction interface or after multiple voice prompts, the robot executes the delivery task to the next destination.

[0013] A further embodiment of the above-mentioned robot human-computer interaction method is that the robot is a material feeding or parts supply robot in a manufacturing production site.

[0014] The beneficial effects of this invention are as follows: The robot human-machine interaction system and method proposed in this invention collects information on changes in the weight and / or volume of items through a counting unit, identifies the quantity and / or type of items to be unloaded at the current destination, and simultaneously collects static or dynamic images of the retrieval subject through a retrieval subject identification system. When the quantity and / or type of items currently unloaded identified by the robot is inconsistent with the quantity and / or type of items pre-set by the system, the system can compare the collected static or dynamic images of the retrieval subject with the retrieval subject data stored in the system to determine whether the retrieval subject is the retrieval subject corresponding to the current destination or the retrieval subject from another destination. Based on the determination result, a voice prompt device is used to issue a voice prompt to the retrieval party requiring human-machine interaction. Finally, through the human-machine interaction intervention of the operator, the robot can identify the exact abnormal retrieval information. Through this real-time human-machine interaction, in the process of standardized item transportation, the robot can identify abnormal retrieval information that deviates from the set program, which can provide a basis and support for the subsequent robot to update the item transportation plan and replenishment plan. Attached Figure Description

[0015] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0016] In the attached diagram: Figure 1This is a schematic diagram of the robot human-computer interaction system provided in an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the object retrieval state in a certain process of the robot human-computer interaction system and method provided in the embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of multiple types of goods transportation modes in the robot human-computer interaction system and method provided in the embodiments of the present invention.

[0019] The meanings of the labels in the attached figures: 10-Robot, 11-Weight collector, 12-Visual recognition system, 13-Voice prompt device, 14-Human-computer interface, 15-Radar, 20-Item. Detailed Implementation

[0020] Exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. It should be noted that these embodiments are provided to enable a more thorough understanding of this disclosure and to fully convey the scope of this disclosure to those skilled in the art. This disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0021] Example 1 See Figure 1 , Figure 1 This is a schematic diagram of the robot human-machine interaction system provided in this embodiment. The robot 10 referred to in this embodiment is an example of a material feeding or parts supply robot in a typical manufacturing production site. The robot 10 performs the task of transporting items, carrying multiple items 20, and sequentially transports the items to multiple pick-up destinations according to a path program or running map. The robot 10 identifies each pick-up destination based on a location code.

[0022] Robot 10 is equipped with a counting unit that can identify unloaded item information at each destination by changes in weight and / or volume, such as... Figure 1 As shown, taking the use of weight collector 11 as an example, the weight collector 11 collects weight changes so that the robot 10 can identify the quantity and / or type of items to be unloaded at each destination.

[0023] Alternatively, the counting unit may also include an image acquisition unit or a contour recognition unit to acquire volume changes, allowing the robot 10 to identify the quantity and / or type of unloaded items at each destination.

[0024] It's easy to understand that robot 10 identifies the quantity of unloaded items at each destination. In this invention, identifying the "type" of unloaded items at each destination means that even on a standardized production line, different pick-up destinations (process stations) may require using the same robot 10 to transport different items, such as... Figure 3 This demonstrates multiple types of goods transport tasks performed using the same robot 10. Figure 3 In the illustration, the different items 20 are shown by their differences in volume.

[0025] To achieve the purpose of this invention, the robot 10 is equipped with a subject recognition system, a voice prompt device 13, and a human-computer interaction interface 14.

[0026] Specifically, the object retrieval entity recognition system is configured to identify whether the current object retrieval entity is the one corresponding to the current destination or the one from another destination. The voice prompt device 13 is configured to issue a voice prompt requiring human-machine interaction to the object retrieval party when the robot 10 detects abnormal object retrieval. Abnormal object retrieval mainly refers to retrieval that is not carried out according to the program plan and retrieval from a destination (process station) other than the current retrieval destination. The human-machine interaction interface 14 is used by the object retrieval party to input abnormal object retrieval information that can be recognized by the robot 10 after performing an abnormal object retrieval action. Specifically, the object retrieval subject identification system includes a visual recognition system 12, which collects static or dynamic images of the object retrieval subject, allowing the robot 10 to identify whether the current object retrieval subject is the object retrieval subject corresponding to the current destination or the object retrieval subject from other destinations. The human-computer interaction interface 14 adopts a touch screen mode and / or a mechanical key mode, providing an object retrieval destination association button and / or an object retrieval subject association button, allowing the robot 10 to identify abnormal object retrieval information through external human-computer interaction commands.

[0027] In the above embodiments, it can be understood that when referring to "object retrieval subject", the present invention may refer to the operator or the object retrieval device or instrument operated by the operator. Regardless of which one it refers to, it should be a relatively stable target that can be distinguished and identified relative to the object retrieval destination (process position). When referring to "object retrieval party", it generally refers to the operator subject.

[0028] The term "voice prompt device" as used in this invention refers to a device that can provide at least voice prompts, not a device that can only provide voice prompts. Therefore, various commonly used screen display devices with voice functionality fall within the scope of "voice prompt device" as defined in this invention.

[0029] In addition, Figure 1-3Although the loading system is not shown, those skilled in the art can apply any suitable automatic loading system disclosed in the prior art to the robot 10 of the present invention without hindrance. In this case, the automatic loading system is also preferably controlled by the same vision system, radar system and main screen panel.

[0030] As an alternative implementation, the aforementioned automated loading system can also be an independent operating system set up at the item storage point, whose location is also identified by location coding as the location (destination) that the robot 10 can recognize.

[0031] Example 2 This embodiment provides a human-computer interaction method based on the robot human-computer interaction system described in Embodiment 1. Specifically, it includes the following steps: S01: The robot 10, which performs the task of transporting items, carries multiple items 20 and transports the items 20 sequentially to multiple pick-up destinations according to the path program or running map. Each destination contains location code information. The robot 10 identifies each pick-up destination based on the location code and retrieves the quantity and / or type of items to be picked up at each pick-up destination. At the current destination, the robot 10 collects the weight and / or volume change information of the items through the counting unit, identifies the quantity and / or type of unloaded items at the current destination, and collects static or dynamic images of the pick-up subject through the pick-up subject identification system. When the quantity and / or type of unloaded items identified by the robot 10 is consistent with the quantity and / or type of unloaded items preset by the system, the delivery task to the next destination is executed. When the quantity and / or type of unloaded items identified by the robot 10 is inconsistent with the quantity and / or type of unloaded items preset by the system, the process proceeds to step S02.

[0032] S02: The robot 10 compares the static or dynamic image of the object being picked up by the object being picked up by the object being picked up by the object being picked up by the object being picked up with the object being picked up data pre-stored in the system, determines whether the object being picked up is the object being picked up corresponding to the current destination or the object being picked up from other destinations, and issues a voice prompt to the object being picked up through the voice prompt device 13 to indicate that human-computer interaction is required.

[0033] In this step, when the robot 10 determines that the person picking up the item is the same person picking up the item corresponding to the current destination, it sends a voice prompt to the person picking up the item via the voice prompt device 13, requiring verification of the quantity and / or type of the item. After waiting for the person picking up the item to confirm via the human-machine interface 14 or after multiple voice prompts, the robot 10 executes the delivery task to the next destination. When the robot 10 determines that the person picking up the item is from another destination, it sends a voice prompt to the person picking up the item via the voice prompt device 13, requiring verification of the address and / or identity. After waiting for the person picking up the item to confirm via the human-machine interface 14 or after multiple voice prompts, the robot 10 executes the delivery task to the next destination.

[0034] S03: The operator inputs abnormal object retrieval information into the robot 10 through the touch screen and / or mechanical keys of the human-machine interface 14 according to the voice prompts. The touch screen and / or mechanical keys provide an object retrieval destination association button and / or an object retrieval subject association button, so that the robot 10 can identify the abnormal object retrieval information of the current destination through external human-machine interaction commands.

[0035] Therefore, based on the human-computer interaction system and method of this invention, the robot can collect information on changes in the weight (or volume) of items through a counting unit, identify the quantity and / or type of items to be unloaded at the current destination, and simultaneously collect static or dynamic images of the retrieval subject through a retrieval subject identification system. When the quantity and / or type of items currently unloaded identified by the robot is inconsistent with the quantity and / or type of items preset by the system, the robot can compare the collected static or dynamic images of the retrieval subject with the retrieval subject data stored in the system to determine whether the retrieval subject is the retrieval subject corresponding to the current destination or a retrieval subject from another destination (or a newly identified unfamiliar retrieval subject). Based on the judgment result, the robot will issue a voice prompt to the retrieval party that human-computer interaction is required through a voice prompt device. Finally, the human-computer interaction intervention of the operator helps the robot identify the exact abnormal retrieval information. This operation basically does not increase the operator's workload, but avoids misjudgment caused by program recognition.

[0036] Through this real-time human-machine interaction, in the process of conveying items in a modular way, the robot can identify abnormal picking information that deviates from the set program. This can provide a basis and support for the robot to update the item conveying plan and replenishment plan in the future. For example, in steps S01-S03 provided in embodiment 2, the robot updates the current conveying task progress at any time according to the picking data input by the picking party, and determines whether the items can be conveyed to each picking destination of the entire line under the current item loading capacity. When it is identified as not, a replenishment plan is generated in time and a replenishment command is automatically output, thereby minimizing the idle travel during the automatic operation period and improving the operating efficiency.

[0037] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations, additions, or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A robot-human interaction system, the robot being a robot that performs an article transport task, the robot carrying a plurality of articles and sequentially transporting the articles to a plurality of article pickup destinations in accordance with a path program or a movement map, the robot being provided with a counting unit that is capable of identifying unloaded article information for each destination by changes in weight and / or volume, characterized by, The robot is equipped with a retrieval subject recognition system, a voice prompt device, and a human-machine interface. When the robot's counting unit identifies that the quantity and / or type of items currently being unloaded by the retrieval subject at the current destination is inconsistent with the quantity and / or type of items preset by the system, the robot uses the retrieval subject recognition system to identify whether the current retrieval subject is the retrieval subject corresponding to the current destination or the retrieval subject from another destination. Based on the recognition result, the robot uses the voice prompt device to issue a voice prompt to the retrieval subject asking whether human-machine interaction is required. The human-machine interface is used by the retrieval subject to input abnormal retrieval information that can be recognized by the robot after performing an abnormal retrieval action.

2. The robot human-computer interaction system according to claim 1, characterized in that, The robot identifies each item retrieval destination based on its location code.

3. The robot human-computer interaction system according to claim 1, characterized in that, The counting unit includes a weight collector, which collects weight changes to help the robot identify the quantity and / or type of items to be unloaded at each destination.

4. The robot human-computer interaction system according to claim 1, characterized in that, The counting unit includes an image acquisition unit or a contour recognition unit, which collects volume changes to help the robot identify the quantity and / or type of items to be unloaded at each destination.

5. A robot human-computer interaction system according to claim 1, characterized in that, The object retrieval subject identification system includes a visual recognition system, which collects static or dynamic images of the object retrieval subject to help the robot identify whether the current object retrieval subject is the object retrieval subject corresponding to the current destination or the object retrieval subject from other destinations.

6. The robot human-computer interaction system according to claim 1, characterized in that, The human-computer interaction interface adopts a touch screen mode and / or mechanical key mode, and provides buttons associated with the object retrieval destination and / or the object retrieval subject, so that the robot can identify abnormal object retrieval information through external human-computer interaction commands.

7. A human-computer interaction method based on the robot human-computer interaction system according to claim 1, characterized in that, Includes the following steps: S01: The robot performing the item delivery task carries multiple items and delivers them sequentially to multiple pick-up destinations according to the path program or running map. Each destination contains location code information. The robot identifies each pick-up destination based on the location code and retrieves the quantity and / or type of items to be picked up at each pick-up destination. At the current destination, the robot collects the weight and / or volume change information of the items through the counting unit, identifies the quantity and / or type of items to be unloaded at the current destination, and collects static or dynamic images of the pick-up subject through the pick-up subject identification system. When the quantity and / or type of items currently unloaded identified by the robot is consistent with the quantity and / or type of items pre-set by the system, the delivery task to the next destination is executed. When the quantity and / or type of items currently unloaded identified by the robot is inconsistent with the quantity and / or type of items pre-set by the system, the process proceeds to step S02. S02: The robot compares the static or dynamic image of the object-retrieving entity collected by the object-retrieving entity recognition system with the object-retrieving entity data stored in the system to determine whether the object-retrieving entity is the object-retrieving entity corresponding to the current destination or the object-retrieving entity from other destinations. Based on the determination result, the robot issues a voice prompt to the object-retrieving party through the voice prompt device, indicating that human-computer interaction is required. S03: The operator inputs abnormal object retrieval information into the robot through the touch screen and / or mechanical keys of the human-machine interface according to the voice prompts. The touch screen and / or mechanical keys provide an object retrieval destination association button and / or an object retrieval subject association button, so that the robot can identify the abnormal object retrieval information of the current destination through external human-machine interaction commands.

8. The human-computer interaction method according to claim 7, characterized in that, In step S02, when the robot determines that the object being retrieved is the object being retrieved corresponding to the current destination, it issues a voice prompt to the object being retrieved through a voice prompt device, requiring verification of the quantity and / or type of the object. After waiting for the object being retrieved to confirm through the human-machine interface or after multiple voice prompts, the robot executes the delivery task to the next destination. When the robot determines that the object being retrieved is the object being retrieved from another destination, it issues a voice prompt to the object being retrieved through a voice prompt device, requiring verification of the address and / or identity. After waiting for the object being retrieved to confirm through the human-machine interface or after multiple voice prompts, the robot executes the delivery task to the next destination.

9. The human-computer interaction method according to claim 7, characterized in that, The robot in question is a material feeding or parts supply robot used in manufacturing production sites.

Images