Data processing method and device for sales service based on humanoid robot

Abstract

The invention provides a data processing method and device for sales service based on a humanoid robot, applied to the humanoid robot, and the method comprises the steps: detecting an article demand of a target customer, obtaining an article information list, the article information list comprising a first category to which a reference article existing in a current service scene belongs, the reference article does not exist in the current service scene, but can be made through the Internet of Things server, and the second category belongs to the reference article; determining a target category to which a target article indicated by the article demand belongs; determining a target task flow; and interactively executing the target task flow with the Internet of Things server, so that the target customer obtains the target article. Thus, the humanoid robot generates the corresponding target task flow based on the current service scene and the article demand of the target customer, and provides the target article required by the target customer by executing the target task flow, thereby improving the intelligence and flexibility of the humanoid robot to provide the sales service. And compatibility of executing data processing in combination with the current service scene.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a data processing method and device for sales services based on a humanoid robot. Background Art

[0002] At present, in offline sales and service scenarios, it has gradually become a mainstream service method for merchants or stores to use humanoid robots to provide sales services to meet customer needs in order to improve service efficiency.

[0003] However, since customer needs are diverse, in addition to common business needs, customer needs may also include item needs. Although the current common humanoid robots can recognize and understand customer intentions, they usually need human assistance to solve customer item needs so that customers can obtain the corresponding items. This results in poor intelligence and efficiency in the sales services provided by humanoid robots, as well as a lack of fluency in data processing. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and device for sales services based on a humanoid robot. After detecting the item demand of a target customer, the humanoid robot determines the target category of the target item indicated by the item demand by obtaining a list of item information adapted to the current service scenario, thereby generating a corresponding target task flow; and then interacts with an Internet of Things server to execute the target task flow so that the target customer can obtain the corresponding target item, thereby improving the comprehensiveness of the sales service that the humanoid robot can provide, and improving the compatibility and flexibility of the humanoid robot in performing data processing in combination with the current service scenario.

[0005] In a first aspect, an embodiment of the present application provides a data processing method for sales services based on a humanoid robot, which is applied to a humanoid robot in an interactive service system, wherein the interactive service system also includes an Internet of Things server, and the Internet of Things server is used to control multiple reference smart devices in a current service scene, and the reference smart devices are used to perform corresponding reference tasks; the method includes: Detecting the item demand of the target customer, obtaining an item information list adapted to the current service scene, wherein the item demand is used to indicate the target item that the target customer currently needs to obtain, and the item information list is used to indicate a first category and a second category of reference items that the humanoid robot can provide, wherein the first category includes item types of reference items pre-placed in the current service scene, and the second category includes item types of reference items that the reference smart device can make by performing a reference task; Determine, according to the item information list, a target category to which the target item indicated by the item demand belongs, the target category being the first category or the second category; Determine a target task flow according to the target category, the type of the target item, and the scene arrangement information of the preset current service scene; and, Interact with the IoT server to execute the target task flow so that the target customer can obtain the target item.

[0006] In a second aspect, an embodiment of the present application provides a data processing device for sales services based on a humanoid robot, which is applied to a humanoid robot in an interactive service system. The interactive service system also includes an Internet of Things server, which is used to control multiple reference smart devices in a current service scene, and the reference smart devices are used to perform corresponding reference tasks; the device includes: an acquisition unit, configured to detect an item demand of a target customer and acquire an item information list adapted to a current service scenario, wherein the item demand is used to indicate a target item that the target customer currently needs to obtain, and the item information list is used to indicate a first category and a second category of reference items that the humanoid robot can provide, wherein the first category includes item types of reference items pre-placed in the current service scenario, and the second category includes item types of reference items that can be made by a reference smart device by performing a reference task; A category determination unit, used to determine, according to the item information list, a target category to which the target item indicated by the item demand belongs, the target category being the first category or the second category; A task flow determination unit, used to determine a target task flow according to a target category, an item type of a target item, and scene arrangement information of a preset current service scene; The execution unit is used to interact with the Internet of Things server to execute the target task flow so that the target customer obtains the target item.

[0007] In a third aspect, an embodiment of the present application provides a humanoid robot comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program comprises instructions for executing the steps in the first aspect of the embodiment of the present application.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps in the first aspect of the embodiment of the present application.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement some or all of the steps described in the first aspect of the embodiment of the present application.

[0010] It can be seen that in the embodiment of the present application, after detecting the target customer's demand for items, the humanoid robot obtains the list of item information adapted to the current service scenario to determine the target category of the target item indicated by the item demand, thereby generating a corresponding target task flow; and then interacts with the Internet of Things server to execute the target task flow. In this way, the humanoid robot can determine the customer's item demand through interaction with the customer, and generate and execute the target task flow based on the item demand to meet the customer's needs. In the process of executing the task flow, the humanoid robot can interact with other intelligent devices to realize a variety of service processes, thereby improving the intelligence and flexibility of the robot service. In combination with the other devices in the service scenario, the fluency and success rate of the service link system in processing information data are improved, and the compatibility of the humanoid robot in various service scenarios is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 is a structural block diagram of an interactive service system provided by an embodiment of the present application; Figure 2 It is a flowchart of a data processing method for sales service based on a humanoid robot provided in an embodiment of the present application; Figure 3 is a scenario diagram of a current service scenario provided by an embodiment of the present application; Figure 4 is a schematic diagram of a scene in which a humanoid robot performs a grasping operation provided by an embodiment of the present application; Figure 5 is a schematic diagram of a scene in which a humanoid robot performs a transfer operation provided by an embodiment of the present application; Figure 6 It is a block diagram of the functional units of a data processing device for sales services based on a humanoid robot provided in an embodiment of the present application; Figure 7 It is a block diagram of the functional units of another data processing device for sales services based on a humanoid robot provided in an embodiment of the present application; Figure 8 It is a structural block diagram of a humanoid robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0015] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0016] See also Figure 1 , Figure 1 is a structural block diagram of the interactive service system provided by the embodiment of the present application. Figure 1 As shown, the interactive service system 100 includes a humanoid robot 110, an Internet of Things server 120 and a plurality of reference smart devices 130, wherein the Internet of Things server 120 is used to control a plurality of reference smart devices 130 in the current service scene, and the reference smart devices 130 are used to perform corresponding reference tasks. The humanoid robot 110 is connected to the Internet of Things server 120 in communication, and the Internet of Things server 120 is connected to the plurality of reference smart devices 130 in communication. The humanoid robot 110 detects the item demand of the target customer 140 in the current service scene by voice interaction or by setting an interactive display screen, and then obtains the list of item information adapted to the current service scene to determine whether the target item indicated by the item demand belongs to the first category of reference items existing in the current service scene, or belongs to the second category of reference items that need to be made by the reference smart device 130, and then generates the corresponding target task flow. The humanoid robot 110 then interacts with the Internet of Things server 120 that can control the reference smart device 130 to perform the reference task, so as to execute the target task flow, so as to enable the target customer 140 to obtain the target item, that is, to meet the item demand.

[0017] Based on this, an embodiment of the present application provides a data processing method for sales services based on a humanoid robot. The embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0018] See also Figure 2 , Figure 2 1 is a flow chart of a data processing method for a sales service based on a humanoid robot provided in an embodiment of the present application. The method is applied to a humanoid robot 110 in an interactive service system 100. The interactive service system 100 also includes an Internet of Things server 120. The Internet of Things server 120 is used to control multiple reference smart devices 130 in the current service scene. The reference smart devices 130 are used to perform corresponding reference tasks. The method includes: Step S201: Detect the item demand of the target customer and obtain an item information list adapted to the current service scenario.

[0019] Among them, the item demand is used to indicate the target item that the target customer currently needs to obtain, and the item information list is used to indicate the first category and the second category of reference items that the humanoid robot can provide. The first category includes the types of reference items that are pre-placed in the current service scene, and the second category includes the types of reference items that the reference smart device can make by performing reference tasks. Since each service scene is different in meeting item demand, the humanoid robot can align with the current service scene at the data information level by obtaining the item information list, ensuring that the subsequent data processing process is based on the associated information of the current service scene for data analysis and processing, thereby enabling the sales service provided by the humanoid robot to adapt to the current service scene, improving the accuracy and success rate of the humanoid robot's execution of sales services, and compatibility in different service scenes. In addition, by obtaining the item information list to classify the reference items in the current service scene, it can ensure that targeted services are provided for the corresponding item needs of the target customers in the future, and improve the fluency and efficiency of the humanoid robot in performing data information processing.

[0020] Among them, the item demand refers to the demand for the use of a specific target item, for example, the target item corresponding to the demand "drinking orange juice" is "a cup with orange juice"; the target item corresponding to the demand "drinking water" is "a cup with drinking water"; the target item corresponding to the demand "eating candy" is "candy". The current service scene can be understood as a venue for selling services held by humans, such as a store, a manufacturer's site, an exhibition, etc. In this current service scene, a humanoid robot is used to perform sales services for the target object (i.e., the target customer). In addition to the humanoid robot, the current service scene also includes various reference smart devices controlled by the Internet of Things server, each of which is used to perform different functions to achieve corresponding different reference tasks. The item information list is used to record the item types of reference items that already exist in the current service scene and have finished products, that is, the first category; in addition, the item information list is also used to record the item types of reference items that can be produced by multiple reference smart devices set in the current service scene, that is, the second category.

[0021] For example, see Figure 3 , Figure 3 Schematic diagram of a current service scenario provided by an embodiment of the present application. Figure 3 As shown, taking the sales service corresponding to the current service scenario as the juice selling service as an example, the target customer interacts with the humanoid robot and says the sentence "I want to drink juice". At this time, the target item indicated by the target customer's item demand determined by the humanoid robot through semantic recognition is a cup of orange juice. Among them, the reference intelligent device includes a label printer and a bar robot arm installed on the bar. These reference intelligent devices are all associated with the target item (cup of orange juice) corresponding to the juice selling service. Furthermore, a juicer is also installed on the bar, and the reference task of the bar robot arm can be: grab oranges → put the oranges into the juicer → start the juicer to squeeze the oranges → pour the prepared orange juice into the cup on the bar. The reference task of the label printer is: print out the cartoon avatar corresponding to the target service customer. The reference task of the bar robot arm also includes: identify the cup of orange juice → stick the cartoon avatar on the cup of orange juice to form the target item → send an instruction to the Internet of Things server to indicate that the target item has been made. The IoT server will then notify the humanoid robot to obtain the target item (i.e. the cup of orange juice), and then deliver the orange juice to the target customer to meet the target customer's item needs.

[0022] In one possible embodiment, before obtaining a list of item information adapted to the current service scenario, the method further includes: the humanoid robot determines the target customer's item needs by responding to the target customer's interactive operation on an interactive display screen set for the humanoid robot, or responding to the target customer's voice interactive operation with the humanoid robot.

[0023] Among them, after the interactive display screen of the humanoid robot is started, a preset customer interface is displayed. The customer interface can select the type of demand of the target customer, such as business demand, item demand, etc. The actual demand type is determined by the product of the sales service provided by the current service scenario. When it is detected that the target customer selects the function or option related to "item demand" through the touch screen, it is determined that the target customer has item demand, and the customer can enter or select his own item demand in the following ways: 1. Text input: directly enter the name or description of the item on the touch screen. 2. Voice input: Using voice recognition technology, customers can directly say their needs. 3. Image recognition: Customers can let the robot identify the required items by taking pictures or uploading pictures. 4. List selection: If the robot has a preset list of common items, customers can select from the list. After the target customer completes the input of his own item demand, the humanoid robot can confirm whether his demand is accurate to the target customer through the interactive display screen or voice module. After waiting for an accurate answer (which can be a voice reply or a touch operation based on the selection control on the interactive display screen), the humanoid robot can determine that the item demand entered by the target customer is the item demand that needs to be processed for subsequent data processing.

[0024] Among them, taking the actual item demand of the target customer as "drinking orange juice" as an example, there are two specific ways to determine the item demand for the voice interaction operation between the target customer and the humanoid robot: 1. The target customer directly speaks the inquiry message corresponding to the item demand. For example, "I want to drink orange juice"; 2. The target customer first expresses the inquiry intention, and then speaks the inquiry message corresponding to the item demand based on the dialogue. For example, the target customer (to the humanoid robot): "I want to ask a question", the humanoid robot: "Okay, go ahead" (the words / content output by the robot in the dialogue are what the humanoid robot can get based on the preset human-computer dialogue model), the target customer: "Excuse me, do you have orange juice? I want to drink orange juice." The humanoid robot identifies the target customer's item demand based on voice recognition, semantic understanding and other operations.

[0025] It can be seen that in this example, when providing sales services, the current service scenario can determine the target customer's item needs through the diverse interaction methods between the humanoid robot and the target customer, and then execute the subsequent corresponding data processing process to meet the target customer's item needs, thereby improving the flexibility and intelligence of the humanoid robot in performing data processing and improving the service experience of the target customer.

[0026] Step S202: determining the target category to which the target item indicated by the item demand belongs according to the item information list.

[0027] The target category is the first category or the second category.

[0028] Among them, this step is a prerequisite step, which is used by the humanoid robot to determine whether the current service scenario can meet the target customer's item needs. Only when the humanoid robot determines that the target item indicated by the item demand belongs to the item type of the reference item included in the first category or the second category, the humanoid robot will determine the target category of the target item and then perform subsequent data processing operations.

[0029] Among them, if the humanoid robot determines that the target item belongs neither to the first category nor to the second category, the humanoid robot can prompt the target customer to re-enter his item requirements through the built-in voice module or interactive display screen through a preset interaction method and item information list, and then restart step S201 until the humanoid robot determines the target category to which the target item indicated by the target customer's newly input item requirements belongs based on the item information list.

[0030] Step S203, determining the target task flow according to the target category, the type of the target item and the preset scene layout information of the current service scene.

[0031] Among them, the specific content in the scene layout information of the current service scene may include the current position information of the humanoid robot, the device position information of multiple reference intelligent devices, the item position information corresponding to each type of reference item in the first category, etc., specifically including various types of position information required for use in the subsequent data processing process. The scene layout information is used by the humanoid robot to plan its movement path required to meet the target customer's item needs, ensuring that the sales service provided by the humanoid robot for the target customer's item demand service can be performed efficiently and accurately. After determining the target category of the target item corresponding to the target customer's item demand, the humanoid robot can ensure the accurate and efficient execution of the subsequent target task flow through further information acquisition operations and data processing operations. The target task flow in this step enables the humanoid robot in this application to meet the target customer's item needs without human assistance, compared to service robots in common service scenarios, thereby improving the intelligence and efficiency of the sales service provided by the humanoid robot.

[0032] In one possible embodiment, a target task flow is determined based on a target category, an item type of a target item, and a preset scene layout information of a current service scene, including: if the target category is a first category, determining the item location information corresponding to the target item based on the scene layout information; determining a first task and a second task based on the item location information and the current location information of the humanoid robot; and determining a third task based on the item type of the target item; and determining the target task flow based on the execution order of the first task, the second task, the third task, and the first task.

[0033] Among them, the first task is used to instruct the humanoid robot to go to the item location corresponding to the item location information, the second task is used to instruct the humanoid robot to return to the current location corresponding to the current location information, and to perform an item delivery operation for the target customer to deliver the target item, and the third task is used to instruct the humanoid robot to perform a grasping operation on the target item. The execution order of the first task is greater than the execution order of the third task, and the execution order of the third task is greater than the execution order of the second task.

[0034] Among them, when the target category is the first category, it means that there is a target item in the current service scene, and the content included in the target task flow is the grabbing and delivery strategy for the target item. In this case, the working principle of the humanoid robot is as follows: the humanoid robot obtains the item location information where the target item is placed through the scene layout information, and then generates the first task of moving to the location where the target item is placed according to the current position and the item location information, and the second task of returning to the current position after grabbing the target item; the humanoid robot then determines how to identify the target item and perform the grabbing operation according to the type of the target item; after the humanoid robot generates the first task, the second task, and the third task, it determines the target task flow according to the execution order of the first task. The humanoid robot can then complete the grabbing of the target item and the delivery to the target customer by executing the target task flow, so that the target customer can obtain the target item. It should be noted that the item position corresponding to the item position information is a position that can ensure that the humanoid robot can identify the target item through the configured camera, and the target item is located within the range that the humanoid robot's mechanical arm and dexterous hand can grab, not the actual location of the target item.

[0035] Exemplarily, the second task instructs the humanoid robot to return to the current position corresponding to the current position information, which can be adaptively changed according to the current service scene. For example, if the previous service scene is a store, the second task can be changed to instruct the humanoid robot to go to the seat position corresponding to the target customer, and perform the item delivery operation for the target customer to deliver the target item to meet the item needs of the target customer. The seat position information corresponding to the seat position can be a data set that stores the corresponding relationship between the seat position information and the face image of each customer and sent to the humanoid robot by the staff in advance. The specific data processing process can be: when the humanoid robot interacts with the corresponding target customer, the camera configured by the humanoid robot performs face recognition to match the customer's face image to determine the corresponding seat position information and the item needs of the target customer; and after the humanoid robot obtains the target item corresponding to the item demand, the humanoid robot moves according to the seat position corresponding to the seat position information, thereby completing the item delivery operation for the target customer.

[0036] It can be seen that in this example, when the target category is the first category, the humanoid robot generates and executes the corresponding target task flow through the above data processing operations, so as to achieve the target needs of the target customers so that the target customers can obtain the target items, improve the intelligence and flexibility of the humanoid robot service, and improve the fluency and success rate of the humanoid robot in processing information data for implementing sales services.

[0037] In one possible embodiment, a target task flow is determined based on a target category, an item type of a target item, and a preset scene layout information of a current service scene, including: if the target category is the second category, determining a reference smart device associated with the target item as the target smart device; determining device location information of the target smart device based on the scene layout information; determining a fourth task and a fifth task based on the device location information and the current location information; and determining a sixth task based on the item type of the target item and the reference task corresponding to the target smart device; and determining a seventh task based on the item type of the target item; and determining the target task flow based on the execution order of the fourth task, the fifth task, the sixth task, the seventh task, and the second task.

[0038] Among them, the fourth task is used to instruct the humanoid robot to go to the device location indicated by the device location information, the fifth task is used to instruct the humanoid robot to return to the current location indicated by the current location information, and to perform an item delivery operation for the target customer to deliver the target item, the sixth task is used to instruct the humanoid robot to send a service request instruction to the Internet of Things server to make the target item, and the seventh task is used to instruct the humanoid robot to perform a grabbing operation on the target item after receiving a service response instruction sent by the Internet of Things server. The service response instruction is used to characterize the target smart device to complete the production of the target item. The execution order of the second task is that the execution order of the sixth task is greater than the execution order of the fourth task, the execution order of the fourth task is greater than the execution order of the seventh task, and the execution order of the seventh task is greater than the execution order of the fifth task.

[0039] Among them, when the target category is the second category, it means that the target item does not exist in the current service scene, then the target task flow includes making the target item, and the grabbing and delivery strategy for the target item. This step requires interaction with the IoT server to control the reference smart device to make the target item. It should be noted that the device position indicated by the device position information of the target smart device determined by the humanoid robot according to the scene layout information should be understood as the position of the manufactured target item that the humanoid robot can grab through the mechanical arm and dexterous hand. For example, when the target item is a cup filled with orange juice, the device position indicated by the device position information is the preset distance of the bar for placing the cup. The preset distance is determined according to the actual machine parameters of the humanoid robot and should be less than the maximum distance that the humanoid robot can grab items.

[0040] Among them, the sixth task instructs the Internet of Things server to control the reference smart device to produce the target item, which may be only the reference smart device executing the reference task to produce the target item, or the reference smart device executing the reference task to produce the target item and performing additional reference tasks for the target item (such as packaging services, labeling services, free gift services, etc., which are determined according to the service content of the sales service provided in the current service scenario, the item attributes of the target item, and the device functions that can be achieved by the reference device).

[0041] For example, when the target demand of the target customer is "drinking orange juice", the target item corresponding to the target demand is "a cup with orange juice", and the corresponding target intelligent devices include a label printer, a bar robot arm, and a batching station. The reference task performed by the bar robot arm can complete the goal of making a cup with orange juice, and the reference task performed by the label printer can perform an additional reference task for the target item, that is, to affix a cartoon avatar corresponding to the target customer to the cup with orange juice. The reference task performed by the batching station also belongs to an additional reference task for the target item. The content of the reference task can be to move the cup through the batching station, and when the cup moves to the target position, add sugar or ice cubes to the orange juice in the cup through the mixing machine above. When the target demand of the target customer is "eating candy", the target item corresponding to the target demand is "candy", and the corresponding target intelligent devices include a candy making machine and a packaging machine. The candy making machine can make the candy that the target customer wants by performing the corresponding reference task, and the reference task performed by the packaging machine is an additional complimentary service provided. The content of the reference task can be to provide customers with suitable wrapping paper to package candy according to the age of the target customer.

[0042] It can be seen that in this example, when the target category is the second category, the humanoid robot generates and executes the corresponding target task flow through the above data processing operations, thereby realizing the process of the IoT server controlling the reference smart device to make the target item, and the humanoid robot obtaining the target item and delivering it to the target customer, thereby improving the intelligence and flexibility of the humanoid robot service. It can be combined with other equipment in the service scenario to improve the fluency and success rate of the service link system in processing information data, thereby improving the compatibility of the service robot.

[0043] In one possible embodiment, a sixth task is determined based on the type of the target item and the reference task corresponding to the target smart device, including: determining necessary production information corresponding to the target item based on the type of the target item; determining that the target smart device indicated by the production device is a necessary smart device, and determining that the remaining target smart devices except the necessary smart device are non-essential smart devices; determining a first request instruction based on the production steps and item-related attributes corresponding to the necessary smart device; obtaining the waiting time limit and task time set of the current sales service; determining a second request instruction based on the waiting time limit and the task time set; determining a service request instruction based on the first request instruction and the second request instruction; and generating the sixth task based on the service request instruction.

[0044] Among them, the necessary production information is used to indicate the production steps required to produce the target item, the item production requirements, and the production equipment corresponding to the production steps. The first request instruction is used to instruct the Internet of Things server to control the corresponding necessary intelligent devices in sequence according to the production steps, and execute the reference tasks adapted to the item production requirements. The task time set includes the estimated task time corresponding to the reference task corresponding to each non-essential intelligent device. The second request instruction is used to instruct the Internet of Things server to control the non-essential intelligent devices whose estimated task time is less than the remaining time to execute the corresponding reference tasks after the first request instruction is executed. The remaining time is determined by the waiting time limit and the actual execution time corresponding to the first request instruction.

[0045] Among them, the humanoid robot instructs the Internet of Things server to control the necessary target intelligent devices corresponding to the target item by generating a first request instruction. The humanoid robot only sends the first request instruction to the Internet of Things server, and can efficiently and accurately obtain the target item made by the reference device. However, the humanoid robot cannot send the second request instruction to the Internet of Things server alone, because there is a precondition in the implementation logic of the second request instruction, and the precondition is that the first request instruction is executed. The principle of this design is that the non-essential intelligent device controlled by the second request instruction is used to execute the corresponding reference task to provide additional additional services for the target item. Whether the second request instruction is executed does not affect the humanoid robot to obtain the target item and deliver it to the target customer. The second request instruction is used as an option to provide the target customer with an additional sales service experience. In addition, in this embodiment, the humanoid robot generates a service request instruction based on the first request instruction and the second request instruction to send to the Internet of Things server. Whether to execute the second request instruction is determined by the Internet of Things server in the process of executing the request instruction, specifically, the reference task of the non-essential intelligent device that can still be executed is determined according to the remaining time after the necessary intelligent device has completed the production of the target item. Furthermore, if the remaining time is sufficient to execute n reference tasks of non-essential smart devices, the IoT server can determine n reference tasks in descending order of popularity based on historical data for execution, or a humanoid robot can interact with the target customer according to a preset interaction method (voice interaction or selection on an interactive display screen) to determine the reference tasks to be executed.

[0046] It can be seen that in this example, the humanoid robot divides the target intelligent devices into necessary intelligent devices and non-essential intelligent devices according to the actual production needs of the target items, and then generates corresponding request instructions to instruct the Internet of Things server to perform additional reference tasks according to the remaining time to provide additional item services on the premise of ensuring the production of the target items. While ensuring the waiting time of the target customers for the target items, the intelligence and flexibility of the humanoid robot in executing the item sales service is improved, thereby improving the service experience of the target customers.

[0047] In a possible embodiment, the grasping operation includes: obtaining an image to be identified corresponding to the position of the object captured by a camera configured for the humanoid robot; and obtaining object reference information adapted to the preset target object type; determining the target object in the image to be identified and the actual shape contour of the target object according to the reference shape contour corresponding to the target object; determining the relative position and orientation of the target object relative to the preset reference position according to the target object in the image to be identified and a pre-trained general pose estimation model; determining a target path from the reference position to the target grasping position according to the image to be identified; according to the joint point change curve determined by the target path, driving a motor to control the position change of each joint of the humanoid robot to move the target dexterous hand to the target grasping position; after the target dexterous hand reaches the target grasping position, controlling the target dexterous hand to perform a target grasping action to grasp the target object according to the reference surface area, the actual shape contour and the reference weight; after it is determined that the target object is grasped successfully, driving the motor to control the joint to move the target dexterous hand to the preset grasping fixed position.

[0048] Among them, the object reference information includes a reference shape contour, a reference surface area and a reference weight. The reference position is the initial position of the target dexterous hand of the humanoid robot. The target dexterous hand is the dexterous hand on the corresponding side selected according to the position of the target object in the image to be identified. The target path is the shortest collision-free path, and the target grasping position is determined by the relative position and orientation of the target object.

[0049] Among them, in this embodiment, the humanoid robot uses the configured camera to collect the image to be identified to identify the target object to be grasped and determine its actual shape contour, and then determine the relative position and orientation of the target object and the initial position of the target dexterous hand, and then determine the target path that the target dexterous hand needs to move. After that, the humanoid robot moves its target dexterous hand according to the target path, and then performs the target grasping action for the target object according to the associated information of the target object, to ensure that the humanoid robot can grasp the target object stably and will not drop it. In addition, after determining that the target object has been grasped successfully, the humanoid robot will also move the target dexterous hand to a preset fixed grasping position to facilitate subsequent movement in the current service scene and to facilitate the target customer to obtain the target object in the target dexterous hand of the humanoid robot.

[0050] It can be seen that in this example, the humanoid robot performs data analysis and processing on the image of the target object to be identified captured by the camera, including the selection of the target dexterous hand, the determination of the target path, and the selection of the target grasping action. Multi-level data analysis and processing is used to improve the stability and accuracy of the humanoid robot's grasping operation and to improve the intelligence of the humanoid robot in providing service for item needs.

[0051] In one possible embodiment, the target dexterous hand is controlled to perform a target grasping action to grasp the target object according to a reference surface area, an actual outer contour and a reference weight, including: determining the target number of fingers according to the reference surface area and a preset grasping range set; if the target object is determined to be a regular-shaped object according to the actual outer contour, determining the reference grasping position according to the geometric center or center of gravity corresponding to the actual outer contour; if the target object is determined to be an irregular-shaped object according to the actual outer contour, determining the reference grasping position according to the center of gravity distribution information of the target object; determining the finger attachment positions corresponding to the target number of fingers on the actual outer contour according to the reference grasping positions; determining the corresponding target grasping force according to the reference weight and the finger attachment positions corresponding to each mechanical finger; and controlling the target dexterous hand to perform the corresponding target grasping action to grasp the target object according to the target number of fingers, the corresponding finger attachment positions of each mechanical finger and the target grasping force.

[0052] The grasping range set is used to indicate the surface area range of the object that can be grasped when the target dexterous hand uses different numbers of mechanical fingers. Based on the grasping range set, the humanoid robot can determine the grasping action that matches the reference surface area of ​​the target object (i.e., how many mechanical fingers to use for grasping).

[0053] Among them, when the target object is an irregularly shaped object, the principle of the humanoid robot determining the reference grasping position according to the center of gravity distribution information of the target object is that the grasping position is usually selected to be close to the center of gravity of the object, which can minimize the instability and overturning risk during grasping, and determine the mass concentration area according to the center of gravity distribution information, and set the grasping point at the edge of the object in this area to provide better support and stability. In the actual grasping process, the humanoid robot also monitors the grasping state in real time through visual feedback and sensor data to adjust the target grasping force corresponding to each mechanical finger. Specifically, when the target object is detected to be falling, the target grasping force is gradually increased until the target object stops falling. In addition, in order to ensure the grasping success rate when the target grasping action is officially executed, the humanoid robot can be tested through a preset data simulation model to verify whether the selection of the finger attachment position and the corresponding target grasping force is appropriate, and determine whether the stability and balance of the target object during the grasping process are sufficient.

[0054] For example, see Figure 4 , Figure 4 Schematic diagram of a scene in which a humanoid robot performs a grasping operation provided by an embodiment of the present application. Figure 4As shown, the humanoid robot determines the reference grasping position according to the actual outer contour of the target object. The number of target fingers of the target dexterous hand corresponding to the current grasping action is 4, including the first mechanical finger, the second mechanical finger, the third mechanical finger and the fourth mechanical finger. Therefore, the corresponding reference grasping position includes the first reference grasping position corresponding to the first mechanical finger, the second reference grasping position corresponding to the second mechanical finger, the third reference grasping position corresponding to the third mechanical finger and the fourth reference grasping position corresponding to the fourth mechanical finger. The humanoid robot then determines the corresponding target grasping force according to the reference weight and the finger attachment position corresponding to each mechanical finger, and finally realizes the execution of the corresponding target grasping action according to the reference grasping position and the target grasping force corresponding to the first mechanical finger, the second mechanical finger, the third mechanical finger and the fourth mechanical finger respectively to grasp the target object (i.e. Figure 4 cup shown in ).

[0055] It can be seen that in this example, the humanoid robot determines the various parameters of the target execution action that needs to be performed in the end through the associated information of the target object, ensuring that the dexterous hand can stably grasp the target object, improving the accuracy and efficiency of the humanoid robot in achieving the target grasping action, and improving the intelligence of the humanoid robot in providing object demand services.

[0056] In one possible embodiment, the object transfer operation includes: playing a preset prompt voice to prompt the target customer to receive the target object; and, acquiring in real time an image to be analyzed captured by a camera centered on a fixed grasping position, and acquiring in real time the weight of the object captured by a sensor built into the target dexterous hand; determining, according to the type of the target object, a judgment condition for receiving the object for the target customer; if the image to be analyzed meets the judgment condition for receiving the object, or the weight change rate of the detected object weight is greater than a first preset change rate, it is determined that the target customer has successfully received the object; and, according to the target grasping force and finger attachment position corresponding to each mechanical finger, controlling the target dexterous hand to return to a preset initial state until it is detected that the weight change rate is greater than a second preset change rate, and the second preset change rate is greater than the first preset change rate.

[0057] The object receiving determination condition includes any one of the following: the target customer's hand has touched the target object, the target customer's hand has covered the target object, and the target customer's hand is located within a preset height area below the target dexterous hand.

[0058] Specifically, the target item characteristics corresponding to the first object receiving determination condition are: items that require direct hand contact to confirm receipt or operation. These items may include but are not limited to small items (such as keys, mobile phones), fragile items (need to be gently picked up). These items require direct physical contact to ensure safe and accurate delivery or operation. The target item characteristics corresponding to the second object receiving determination condition are: items that need to be fully or partially covered to protect privacy, prevent contamination or ensure safety. For example, food, medicine, etc., items may require additional protection measures during the delivery process to ensure that they are not disturbed by external factors before the recipient's hand is fully covered. The target item characteristics corresponding to the third object receiving determination condition are: heavy objects, items that are difficult to grasp directly by hand (such as smooth surfaces or high-temperature objects), or items that are not afraid of falling and being damaged (such as packaged food). Such items are often not suitable for direct grasping by human hands due to weight, shape, temperature or other physical characteristics.

[0059] For example, see Figure 5 , Figure 5 Schematic diagram of a scene in which a humanoid robot performs a transfer operation provided by an embodiment of the present application. Figure 5 As shown in FIG. 1 , according to the type of the target object grasped by the target dexterous hand, Figure 5 The illustrated legends correspond to the target dexterous hand for grabbing a cup, the target dexterous hand for grabbing an apple, and the target dexterous hand for grabbing a candy from left to right. Further, according to the target object characteristics corresponding to the target object, the target customer's receiving action predicted by the humanoid robot to obtain the target object is different, and the camera configured by the humanoid robot collects the corresponding image to identify the transfer area of ​​the receiving action is also different, wherein the transfer area includes the first transfer area, the second transfer area, and the third transfer area. Among them, the humanoid robot identifies whether the target customer's hand has touched the target object by collecting the image of the first transfer area, the humanoid robot identifies whether the target customer's hand has covered the target object by collecting the image of the second transfer area, and the humanoid robot identifies whether the target customer's hand is located in the preset height area below the target dexterous hand by collecting the image of the third transfer area. If the image to be analyzed by the humanoid robot meets the receiving judgment condition at this time, it will start to control the target dexterous hand to release the target object until it is detected that the weight change rate is greater than the second preset change rate, and it is determined that the transfer operation is completed.

[0060] The humanoid robot prompts the target customer to obtain the target item through a preset prompt voice, and at the same time, performs different data processing processes according to the type of the target item and the actual weight of the item, so as to ensure that the target customer can be accurately determined to receive the target item, and avoid the situation where the humanoid robot lets go when the target customer is not ready to receive the target item. In this embodiment, the humanoid robot converts the associated data corresponding to the target item to accurately identify whether the target customer has received the item successfully, and then completes the delivery action for the target item.

[0061] It can be seen that in this example, the corresponding data processing process is executed according to parameters such as the type of target item and the actual item weight, which improves the accuracy of data processing performed by the humanoid robot, achieves the stability of the transfer action and the intelligence of providing item demand services, and improves the service experience of target customers.

[0062] Step S204: interact with the Internet of Things server to execute the target task flow so that the target customer obtains the target item.

[0063] Among them, the humanoid robot provided in this application in this step can realize autonomous linkage with the reference smart device through the Internet of Things server in the current service scenario, realizing the compatibility of the humanoid robot in different service scenarios, and the humanoid robot can provide target customers with sales services adapted to the current service scenario through interaction with the Internet of Things server. Compared with common service robots, it can provide diversified sales services and is more efficient.

[0064] Among them, when the humanoid robot determines that the target category of the target item is the first category, the humanoid robot can directly obtain the target item to deliver it to the target customer. When the humanoid robot determines that the target category of the target item is the second category, the humanoid robot performs the sixth task to send a service request instruction to the Internet of Things server. The Internet of Things server will accept the instruction to control the corresponding target intelligent device to perform the corresponding reference task according to the order and requirements indicated by the service request instruction, and finally the target intelligent device will produce the target item. After the target intelligent device completes the execution according to the service request instruction, the Internet of Things server will determine that the target item has been produced and generate a service response instruction based on this. Afterwards, the Internet of Things server sends a service response instruction to the humanoid robot to notify the humanoid robot to grab the target item. Finally, the humanoid robot delivers the target item to the target customer through the grabbing operation and delivery operation adapted to the target item to meet the target customer's item needs.

[0065] visible, Figure 2This is a flow chart of a data processing method for a sales service based on a humanoid robot provided in an embodiment of the present application. After detecting the target customer's demand for items, the humanoid robot obtains a list of item information adapted to the current service scenario to determine the target category of the target item indicated by the item demand, thereby generating a corresponding target task flow; and then interacts with the Internet of Things server to execute the target task flow so that the target customer can obtain the corresponding target item. In this way, the humanoid robot can determine the customer's item demand through interaction with the customer, and generate and execute the target task flow based on the item demand to meet the customer's needs. In the process of executing the task flow, the humanoid robot can interact with other intelligent devices to realize a variety of service processes, thereby improving the intelligence and flexibility of the robot service. In combination with the other devices in the service scenario, the fluency and success rate of the service link system in processing information data are improved, and the compatibility of the humanoid robot in various service scenarios is improved.

[0066] The following is an embodiment of the device of the present application. The embodiment of the device of the present application and the embodiment of the method of the present application are of the same concept and are used to execute the method described in the embodiment of the present application. For the convenience of explanation, the embodiment of the device of the present application only shows the part related to the embodiment of the device of the present application. For the specific technical details not disclosed, please refer to the description of the embodiment of the method of the present application, which will not be repeated here one by one.

[0067] The embodiment of the present application provides a data processing device for sales services based on a humanoid robot. The data processing device for sales services based on a humanoid robot is applied to a humanoid robot 110 in an interactive service system 100. The interactive service system 100 also includes an Internet of Things server 120. The Internet of Things server 120 is used to control multiple reference smart devices 130 in the current service scene. The reference smart devices 130 are used to perform corresponding reference tasks. Specifically, the data processing device for sales services based on a humanoid robot is used to execute the steps performed by the humanoid robot in the above data processing method for sales services based on a humanoid robot. The data processing device for sales services based on a humanoid robot provided in the embodiment of the present application may include modules corresponding to the corresponding steps.

[0068] The embodiment of the present application can divide the data processing device for sales services based on humanoid robots into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. The division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0069] In the case of dividing each functional module into corresponding functional modules, Figure 6It is a block diagram of the functional units of a data processing device for sales services based on a humanoid robot provided in an embodiment of the present application; the data processing device 60 for sales services based on a humanoid robot is applied to a humanoid robot 110 in an interactive service system 100, and the interactive service system 100 also includes an Internet of Things server 120, and the Internet of Things server 120 is used to control multiple reference smart devices 130 in the current service scene, and the reference smart devices 130 are used to execute corresponding reference tasks. The data processing device 60 for sales service based on a humanoid robot includes: an acquisition unit 601, which is used to detect the item demand of a target customer and acquire an item information list adapted to the current service scene, wherein the item demand is used to indicate the target item that the target customer currently needs to obtain, and the item information list is used to indicate the first category and the second category of reference items that the humanoid robot can provide, wherein the first category includes the item types of reference items pre-placed in the current service scene, and the second category includes the item types of reference items that can be made by the reference smart device by executing a reference task; a category determination unit 602, which is used to determine, based on the item information list, the target category to which the target item indicated by the item demand belongs, wherein the target category is the first category or the second category; a task flow determination unit 603, which is used to determine the target task flow based on the target category, the item type of the target item and the preset scene layout information of the current service scene; and an execution unit 604, which is used to interact with the Internet of Things server to execute the target task flow so that the target customer obtains the target item.

[0070] In one possible embodiment, in terms of determining the target task flow according to the target category, the item type of the target item and the scene layout information of the preset current service scene, the task flow determination unit 603 is specifically used to: if the target category is the first category, determine the item location information corresponding to the target item according to the scene layout information; determine the first task and the second task according to the item location information and the current position information of the humanoid robot, the first task being used to instruct the humanoid robot to go to the item location corresponding to the item location information, the second task being used to instruct the humanoid robot to return to the current location corresponding to the current location information, and perform an item delivery operation for the target customer to deliver the target item; and, determining the third task according to the item type of the target item, the third task being used to instruct the humanoid robot to perform a grasping operation on the target item; determining the target task flow according to the execution order of the first task, the second task, the third task and the first task, the execution order of the first task being that the execution order of the first task is greater than the execution order of the third task, and the execution order of the third task is greater than the execution order of the second task.

[0071] In one possible embodiment, in terms of determining the target task flow according to the target category, the item type of the target item, and the preset scene layout information of the current service scene, the task flow determination unit 603 is specifically used to: if the target category is the second category, determine the reference smart device associated with the target item as the target smart device; determine the device location information of the target smart device according to the scene layout information; determine the fourth task and the fifth task according to the device location information and the current location information, the fourth task being used to instruct the humanoid robot to go to the device location indicated by the device location information, the fifth task being used to instruct the humanoid robot to return to the current location indicated by the current location information, and to perform an item delivery operation for the target customer to deliver the target item; and, according to the item type and the target item, The sixth task is determined according to the reference task corresponding to the target smart device, and the sixth task is used to instruct the humanoid robot to send a service request instruction to the Internet of Things server to produce the target item; and the seventh task is determined according to the type of the target item, and the seventh task is used to instruct the humanoid robot to perform a grabbing operation on the target item after receiving the service response instruction sent by the Internet of Things server, and the service response instruction is used to characterize the target smart device to complete the production of the target item; according to the execution order of the fourth task, the fifth task, the sixth task, the seventh task and the second task, the target task flow is determined, and the execution order of the second task is that the execution order of the sixth task is greater than the execution order of the fourth task, the execution order of the fourth task is greater than the execution order of the seventh task, and the execution order of the seventh task is greater than the execution order of the fifth task.

[0072] In a possible embodiment, in determining the sixth task according to the object type of the target object and the reference task corresponding to the target smart device, the task flow determination unit 603 is specifically used to: determine the necessary production information corresponding to the target object according to the object type of the target object, the necessary production information is used to indicate the production steps required to produce the target object, the object production requirements, and the production equipment corresponding to the production steps; determine that the target smart device indicated by the production equipment is a necessary smart device, and determine that the remaining target smart devices except the necessary smart device are non-essential smart devices; determine a first request instruction according to the production steps corresponding to the necessary smart devices and the object-related attributes, the first request instruction is used to instruct the Internet of Things server to generate the target object according to the production steps. Control the corresponding necessary smart devices in sequence to execute reference tasks that are adapted to the requirements of item production; obtain the waiting time limit and task duration set of the current sales service, the task duration set including the estimated task duration corresponding to the reference task corresponding to each non-essential smart device; determine a second request instruction according to the waiting time limit and the task duration set, the second request instruction is used to instruct the Internet of Things server to control the non-essential smart devices whose estimated task duration is less than the remaining time to execute the corresponding reference tasks after the first request instruction is executed, the remaining time being determined by the waiting time limit and the actual execution time corresponding to the first request instruction; determine a service request instruction according to the first request instruction and the second request instruction; generate a sixth task according to the service request instruction.

[0073] In a possible embodiment, the grasping operation includes: obtaining an image to be identified corresponding to the position of an object captured by a camera configured for the humanoid robot; and obtaining object reference information adapted to the type of a preset target object, the object reference information including a reference shape contour, a reference surface area, and a reference weight; determining the target object in the image to be identified and the actual shape contour of the target object according to the reference shape contour corresponding to the target object; determining the relative position and orientation of the target object relative to a preset reference position according to the target object in the image to be identified and a pre-trained general pose estimation model, the reference position being the initial position of the target dexterous hand of the humanoid robot, and the target dexterous hand being the target object located in the image to be identified. the dexterous hand of the corresponding side selected by the position; according to the image to be recognized, a target path from the reference position to the target grasping position is determined, the target path is a collision-free path with the shortest distance, and the target grasping position is determined by the relative position and orientation of the target object; according to the joint point change curve determined by the target path, the driving motor controls the joints of the humanoid robot to change the position so as to move the target dexterous hand to the target grasping position; after the target dexterous hand reaches the target grasping position, the target dexterous hand is controlled to perform the target grasping action to grasp the target object according to the reference surface area, the actual outer contour and the reference weight; after it is determined that the target object is grasped successfully, the driving motor controls the joints to move the target dexterous hand to a preset grasping fixed position.

[0074] In one possible embodiment, in terms of controlling the target dexterous hand to perform a target grasping action to grasp the target object based on a reference surface area, an actual outer contour and a reference weight, the task flow determination unit 603 is specifically used to: determine the target number of fingers based on the reference surface area and a preset grasping range set, the grasping range set being used to indicate the surface area range of objects that can be grasped when the target dexterous hand uses different numbers of mechanical fingers; if the target object is determined to be a regular-shaped object based on the actual outer contour, determine the reference grasping position based on the geometric center or center of gravity corresponding to the actual outer contour; if the target object is determined to be an irregular-shaped object based on the actual outer contour, determine the reference grasping position based on the center of gravity distribution information of the target object; determine the finger attachment position corresponding to the target number of fingers on the actual outer contour based on the reference grasping position; determine the corresponding target grasping force based on the reference weight and the finger attachment position corresponding to each mechanical finger; control the target dexterous hand to perform the corresponding target grasping action to grasp the target object based on the target number of fingers, the corresponding finger attachment position of each mechanical finger and the target grasping force.

[0075] In one possible embodiment, the object transfer operation includes: playing a preset prompt voice to prompt the target customer to receive the target object; and, acquiring in real time an image to be analyzed captured by a camera centered on a fixed grasping position, and acquiring in real time the weight of the object captured by a sensor built into the target dexterous hand; determining a receiving judgment condition for the target customer according to the type of the target object, the receiving judgment condition including any one of the following: the target customer's hand has touched the target object, the target customer's hand has covered the target object, and the target customer's hand is located within a preset height area below the target dexterous hand; if the image to be analyzed meets the receiving judgment condition, or the weight change rate of the detected object weight is greater than a first preset change rate, it is determined that the target customer has successfully received the object; and, according to the target grasping force and finger attachment position corresponding to each mechanical finger, controlling the target dexterous hand to return to a preset initial state until it is detected that the weight change rate is greater than a second preset change rate, and the second preset change rate is greater than the first preset change rate.

[0076] In the case of integrated units, such as Figure 7 As shown, Figure 7 This is a block diagram of the functional units of another data processing device for sales services based on a humanoid robot provided in an embodiment of the present application. Figure 7In the embodiment, the data processing device 60 for sales service based on humanoid robot includes: a processing module 702 and a communication module 701. The processing module 702 is used to control and manage the actions of the data processing device for sales service based on humanoid robot, for example, the steps of the acquisition unit 601, the category determination unit 602, the task flow determination unit 603 and the execution unit 604, and / or other processes for executing the technology described herein. The communication module 701 is used to support the interaction between the data processing device for sales service based on humanoid robot and other devices. Figure 7 As shown, the data processing device for sales services based on a humanoid robot may include a storage module 703, and the storage module 703 is used to store program codes and data of the data processing device for sales services based on a humanoid robot.

[0077] Among them, the processing module 702 can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 701 can be a transceiver, an RF circuit or a communication interface, and the like. The storage module 703 can be a memory.

[0078] All relevant contents of each scenario involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here. Figure 2 The data processing method for sales service based on humanoid robot is shown.

[0079] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0080] Figure 8 is a structural block diagram of a humanoid robot provided in an embodiment of the present application. Figure 8 As shown, the humanoid robot 110 may include one or more of the following components: a processor 801, a memory 802 coupled to the processor 801, wherein the memory 802 may store one or more computer programs 803, and the one or more computer programs 803 may be configured to implement the methods described in the above embodiments when executed by one or more processors 801. The humanoid robot here is the humanoid robot 110 in the above embodiments.

[0081] The processor 801 may include one or more processing cores. The processor 801 uses various interfaces and lines to connect the various parts of the entire humanoid robot 110, and executes various functions and processes data of the humanoid robot 110 by running or executing instructions, programs, code sets or instruction sets stored in the memory 802, and calling data stored in the memory 802. Optionally, the processor 801 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 801 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), and a modem. Among them, the CPU mainly processes the operating system, customer interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 801, but may be implemented separately through a communication chip.

[0082] The memory 802 may include a random access memory (RAM) or a read-only memory (ROM). The memory 802 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the humanoid robot 110 during use, etc.

[0083] It is understandable that the humanoid robot 110 may include more or fewer structural elements than those in the above structural block diagram, which is not limited here.

[0084] An embodiment of the present application also provides a computer storage medium, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, part or all of the steps of any method recorded in the above method embodiments are implemented.

[0085] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0086] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0087] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are only schematic; for example, the division of units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0090] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a volatile memory or a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM) and direct RAM bus random access memory (DR RAM), among other media that can store program code.

[0091] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions without departing from the spirit and scope of the present invention, and can make various changes and modifications, including the combination of the above-mentioned different functions and implementation steps, including software and hardware implementation methods, all of which are within the scope of protection of the present invention.

Claims

1. A data processing method for sales service based on a humanoid robot, characterized in that: A humanoid robot applied to an interactive service system, wherein the interactive service system further comprises an Internet of Things server, wherein the Internet of Things server is used to control a plurality of reference smart devices in a current service scene, wherein the reference smart devices are used to perform corresponding reference tasks; and the method comprises: Detecting the item demand of the target customer, and acquiring an item information list adapted to the current service scene, wherein the item demand is used to indicate the target item that the target customer currently needs to obtain, and the item information list is used to indicate a first category and a second category of reference items that the humanoid robot can provide, wherein the first category includes item types of reference items pre-placed in the current service scene, and the second category includes item types of reference items that the reference smart device can make by performing the reference task; Determining, according to the item information list, a target category to which the target item indicated by the item demand belongs, the target category being the first category or the second category; Determining a target task flow according to the target category, the type of the target item, and the scene arrangement information of the preset current service scene; and, The target task flow is executed by interacting with the Internet of Things server so that the target customer obtains the target item.

2. The method according to claim 1, characterized in that The step of determining the target task flow according to the target category, the type of the target item, and the preset scene arrangement information of the current service scene includes: If the target category is the first category, determining the object location information corresponding to the target object according to the scene layout information; Determining a first task and a second task according to the item location information and the current location information of the humanoid robot, wherein the first task is used to instruct the humanoid robot to go to the item location corresponding to the item location information, and the second task is used to instruct the humanoid robot to return to the current location corresponding to the current location information, and to perform an item delivery operation for a target customer to deliver the target item; and Determining a third task according to the type of the target object, wherein the third task is used to instruct the humanoid robot to perform a grasping operation on the target object; The target task flow is determined according to the execution order of the first task, the second task, the third task and the first task, wherein the execution order of the first task is greater than the execution order of the third task, and the execution order of the third task is greater than the execution order of the second task.

3. The method according to claim 2, characterized in that The step of determining the target task flow according to the target category, the type of the target item, and the preset scene arrangement information of the current service scene includes: If the target category is the second category, determining the reference smart device associated with the target item as the target smart device; Determining device location information of the target smart device according to the scene layout information; Determining a fourth task and a fifth task according to the device location information and the current location information, wherein the fourth task is used to instruct the humanoid robot to go to the device location indicated by the device location information, and the fifth task is used to instruct the humanoid robot to return to the current location indicated by the current location information, and to perform an item delivery operation for a target customer to deliver the target item; and Determining a sixth task according to the type of the target object and the reference task corresponding to the target smart device, wherein the sixth task is used to instruct the humanoid robot to send a service request instruction to the Internet of Things server to produce the target object; and Determining a seventh task according to the type of the target object, the seventh task is used to instruct the humanoid robot to perform the grabbing operation on the target object after receiving the service response instruction sent by the Internet of Things server, the service response instruction is used to characterize the target smart device to complete the production of the target object; The target task flow is determined according to the execution order of the fourth task, the fifth task, the sixth task, the seventh task and the second task, wherein the execution order of the second task is that the execution order of the sixth task is greater than the execution order of the fourth task, the execution order of the fourth task is greater than the execution order of the seventh task, and the execution order of the seventh task is greater than the execution order of the fifth task.

4. The method according to claim 3, characterized in that: The determining the sixth task according to the object type of the target object and the reference task corresponding to the target smart device includes: Determine necessary production information corresponding to the target item according to the item type of the target item, wherein the necessary production information is used to indicate production steps required for producing the target item, item production requirements, and production equipment corresponding to the production steps; Determining that the target smart device indicated by the production device is a necessary smart device, and determining that the remaining target smart devices except the necessary smart device are non-essential smart devices; Determine a first request instruction according to the production steps corresponding to the necessary smart devices and the associated attributes of the item, wherein the first request instruction is used to instruct the Internet of Things server to sequentially control the corresponding necessary smart devices according to the production steps to perform a reference task adapted to the production requirements of the item; Obtaining a waiting time limit and a task time set for the current sales service, wherein the task time set includes an estimated task time corresponding to a reference task corresponding to each of the non-essential smart devices; Determine a second request instruction according to the waiting time limit and the task time set, the second request instruction is used to instruct the Internet of Things server to control the non-essential smart device whose estimated task time is less than the remaining time to perform the corresponding reference task after the first request instruction is executed, and the remaining time is determined by the waiting time limit and the actual execution time corresponding to the first request instruction; Determine the service request instruction according to the first request instruction and the second request instruction; The sixth task is generated according to the service request instruction.

5. The method according to any one of claims 2 to 4, characterized in that: The crawling operation includes: Acquire an image to be identified corresponding to the position of the object captured by a camera configured by the humanoid robot; and Acquire preset object reference information that matches the object type of the target object, wherein the object reference information includes a reference shape profile, a reference surface area, and a reference weight; Determining the target object in the image to be identified and the actual shape contour of the target object according to the reference shape contour corresponding to the target object; According to the target object in the image to be identified and a pre-trained general pose estimation model, determine the relative position and orientation of the target object relative to a preset reference position, wherein the reference position is the initial position of the target dexterous hand of the humanoid robot, and the target dexterous hand is the dexterous hand of the corresponding side selected according to the position of the target object in the image to be identified; Determine a target path from the reference position to a target grasping position according to the image to be recognized, wherein the target path is a collision-free path with the shortest distance, and the target grasping position is determined by the relative position and the orientation of the target object; According to the joint point change curve determined by the target path, the driving motor controls each joint of the humanoid robot to change position, so as to move the target dexterous hand to the target grasping position; After the target dexterous hand reaches the target grasping position, the target dexterous hand is controlled to perform a target grasping action to grasp the target object according to the reference surface area, the actual outer contour and the reference weight; After it is determined that the target object is grasped successfully, the motor is driven to control the joint to move the target dexterous hand to a preset grasping fixed position.

6. The method according to claim 5, characterized in that The step of controlling the target dexterous hand to perform a target grasping action to grasp the target object according to the reference surface area, the actual outer contour and the reference weight comprises: Determining a target number of fingers according to the reference surface area and a preset grasping range set, wherein the grasping range set is used to indicate the surface area range of objects that can be grasped by the target dexterous hand when using different numbers of mechanical fingers; If the target object is determined to be a regular-shaped object according to the actual outer contour, a reference grasping position is determined according to a geometric center or a center of gravity corresponding to the actual outer contour; If the target object is determined to be an irregularly shaped object according to the actual outer contour, the reference grasping position is determined according to the center of gravity distribution information of the target object; Determining finger attachment positions corresponding to the target number of fingers on the actual contour according to the reference grasping positions; Determining a corresponding target grasping force according to the reference weight and the finger attachment position corresponding to each mechanical finger; The target dexterous hand is controlled to perform a corresponding target grasping action to grasp the target object according to the target number of fingers, the corresponding finger attachment position of each mechanical finger and the target grasping force.

7. The method according to claim 6, characterized in that The item transfer operation includes: Playing a preset prompt voice to prompt the target customer to receive the target item; and, Acquire in real time the image to be analyzed centered on the fixed grasping position captured by the camera, and acquire in real time the weight of the object captured by the sensor built into the target dexterous hand; According to the type of the target object, determining the object acceptance judgment condition for the target customer, the object acceptance judgment condition including any one of the following: the target customer's hand has touched the target object, the target customer's hand has covered the target object, the target customer's hand is located in a preset height area below the target dexterous hand; If the image to be analyzed satisfies the object receiving determination condition, or if the weight change rate of the object detected is greater than a first preset change rate, it is determined that the target customer has successfully received the object; and According to the target grasping force and finger attachment position corresponding to each mechanical finger, the target dexterous hand is controlled to return to a preset initial state until it is detected that the weight change rate is greater than a second preset change rate, and the second preset change rate is greater than the first preset change rate.

8. A data processing device for sales services based on a humanoid robot, characterized in that: A humanoid robot applied to an interactive service system, wherein the interactive service system further comprises an Internet of Things server, wherein the Internet of Things server is used to control a plurality of reference smart devices in a current service scene, wherein the reference smart devices are used to execute corresponding reference tasks; The device comprises: an acquisition unit, configured to detect an item demand of a target customer, and acquire an item information list adapted to the current service scenario, wherein the item demand is used to indicate a target item that the target customer currently needs to obtain, and the item information list is used to indicate a first category and a second category of reference items that the humanoid robot can provide, wherein the first category includes item types of reference items pre-placed in the current service scenario, and the second category includes item types of reference items that the reference smart device can make by performing the reference task; a category determination unit, configured to determine, according to the item information list, a target category to which the target item indicated by the item demand belongs, wherein the target category is the first category or the second category; A task flow determination unit, configured to determine a target task flow according to the target category, the type of the target item, and the scene arrangement information of the preset current service scene; An execution unit is used to interact with the Internet of Things server to execute the target task flow so that the target customer obtains the target item.

9. A humanoid robot, characterized in that: The method comprises a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 7.

Images