Robot automatic interaction method, device, equipment and storage medium

Abstract

The application relates to intelligent decision-making and medical health technology, and discloses a robot automatic interaction method, device, equipment and storage medium. The method comprises the following steps: capturing a behavior image of a user based on an interaction robot, analyzing a human body posture of the behavior image by using a posture analysis model which is pre-trained in the interaction robot, constructing a behavior picture of the user according to the human body posture, performing limb action decomposition on the user to obtain a plurality of user decomposition actions, and calculating decomposition parameters of each user decomposition action; simulating the action picture of the user in the interaction robot according to the decomposition parameters to obtain a simulation action picture, configuring an interaction instruction of the simulation action picture, executing an interaction event corresponding to the simulation action picture through the interaction instruction, and obtaining an interaction result of the interaction robot. The application aims to improve the automatic interaction efficiency of the robot.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology, and in particular to a robot automatic interaction method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Interactive robots are robots that engage in human-machine dialogue with computer systems and operators or programmers, capable of independently performing functions such as trajectory planning and simple obstacle avoidance. Currently, there are many types of humanoid robots on the market, with complex delivery methods, including control via mobile apps, remote controls, and voice control. However, these interaction methods all have a certain learning time cost. In addition, the robot's pre-programmed movements can be too rigid. If users want robots to perform highly complex and customized movements, more complex programming is required. While some interactive robots use programming software that provides intelligent modular designs, allowing users to simply drag modules and set parameters to complete programming, this method still requires manual programming by the user, thus reducing the interaction efficiency of the robot. Therefore, a method is needed to improve the efficiency of automatic robot interaction. Summary of the Invention

[0003] This invention provides a method, apparatus, device, and storage medium for automatic robot interaction, the main purpose of which is to improve the efficiency of automatic robot interaction.

[0004] To achieve the above objectives, the present invention provides an automatic robot interaction method, comprising:

[0005] The system captures user behavior images based on an interactive robot, and analyzes the human posture of the behavior images using a posture analysis model pre-trained in the interactive robot.

[0006] Based on the human posture, a behavioral picture of the user is constructed to decompose the user's body movements, resulting in multiple user decomposed movements, and decomposition parameters for each user decomposed movement are calculated.

[0007] Based on the decomposition parameters, the user's actions are simulated in the interactive robot to obtain a simulated action screen. Interaction instructions for the simulated action screen are configured, and the corresponding interaction events are executed through the interaction instructions to obtain the interaction result of the interactive robot.

[0008] Optionally, the step of analyzing the human posture of the behavior image using a posture analysis model pre-trained in the interactive robot includes:

[0009] The background separation algorithm in the posture analysis model is used to separate the background of the behavior image to obtain the target image;

[0010] The convolutional layer in the pose analysis model is used to extract features from the target image to obtain image features;

[0011] The matching algorithm in the pose analysis model is used to perform feature matching between the image features and the feature poses in the preset pose feature database to obtain feature matching values;

[0012] When the feature matching value is greater than a preset threshold, the feature matching value corresponding to the feature matching value is taken as the human pose of the behavior image.

[0013] Optionally, the background separation algorithm includes:

[0014]

[0015] Where D(a, b) represents the target image, h x (a, b) represent the image corresponding to the x-th frame in the action image, h x+1 (a, b) represents the image corresponding to the (x+1)th frame in the action image, K represents the binarization threshold of the image corresponding to the xth frame and the image corresponding to the (x+1)th frame, and others represents other images in the action image.

[0016] Optionally, constructing the user's behavioral image based on the human posture includes:

[0017] The number of human body poses is counted, and the dynamic frame rate corresponding to the human body pose is calculated based on the number of poses.

[0018] The human body postures are sorted according to a preset sorting rule to obtain sorted postures, and a dynamic environment corresponding to the human body postures is created.

[0019] By combining the dynamic frame rate and the sorting posture, the user's behavior is generated in the dynamic environment.

[0020] Optionally, the step of decomposing the user's body movements to obtain multiple user-decomposed movements includes:

[0021] Identify the user image in the behavior screen;

[0022] Each limb part of the user image is labeled to obtain multiple user limb parts;

[0023] Extract the user limb movements corresponding to the multiple user limb parts respectively;

[0024] Each limb movement in the user's limb movements is decomposed into multiple user decomposed movements.

[0025] Optionally, calculating the decomposition parameters for each user decomposition action includes:

[0026] Based on the user image, construct a three-dimensional coordinate system corresponding to the user's decomposed actions;

[0027] Obtain the initial coordinates of the user's decomposed action in the three-dimensional coordinate system;

[0028] Run the user decomposition action in the three-dimensional coordinate system and obtain the termination coordinates corresponding to the user decomposition action;

[0029] The motion trajectory of the user's decomposed actions is recorded, and the decomposition parameters of each user's decomposed actions are calculated by combining the initial coordinates, the termination coordinates, and the motion trajectory.

[0030] Optionally, the step of simulating the user's actions in the interactive robot based on the decomposition parameters to obtain simulated action footage includes:

[0031] Obtain the simulation parameters corresponding to the decomposition parameters in the interactive robot, and define the simulation code for the simulation parameters;

[0032] A virtual user corresponding to the user is constructed in the interactive robot;

[0033] Based on the simulation code, the virtual user's actions are simulated in the interactive robot to obtain simulated action footage.

[0034] To address the above problems, the present invention also provides an automated robot interaction device, the device comprising:

[0035] The posture analysis module is used to capture user behavior images based on the interactive robot and analyze the human posture of the behavior images using a posture analysis model pre-trained in the interactive robot.

[0036] The motion decomposition module is used to construct the user's behavior scene based on the human posture, so as to decompose the user's limb movements, obtain multiple user decomposed movements, and calculate the decomposition parameters of each user decomposed movement.

[0037] The interaction execution module is used to simulate the user's action screen in the interactive robot according to the decomposition parameters, obtain the simulated action screen, configure the interaction instructions of the simulated action screen, execute the interaction events corresponding to the simulated action screen through the interaction instructions, and obtain the interaction result of the interactive robot.

[0038] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0039] At least one processor; and,

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the robot autonomous interaction method described above.

[0042] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described robot automatic interaction method.

[0043] This invention captures user behavior images using an interactive robot and analyzes the human posture in the behavior images using a pre-trained posture analysis model within the interactive robot. The posture analysis model can analyze the user's corresponding posture in the behavior images to facilitate subsequent action decomposition. This invention constructs a user behavior scene based on the human posture, converting the human posture into a corresponding behavior scene, which is then used for subsequent action decomposition. Furthermore, this invention simulates the user's action scene in the interactive robot based on the decomposition parameters, obtaining a simulated action scene for the interactive robot to learn from and for configuring corresponding interaction commands. Therefore, the robot automatic interaction method, apparatus, device, and storage medium provided by this invention can improve the efficiency of robot automatic interaction. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating an embodiment of the robot automatic interaction method provided by the present invention;

[0045] Figure 2 This is a functional block diagram of a robot automatic interaction device provided in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the structure of an electronic device for implementing the robot automatic interaction method according to an embodiment of the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0049] This application provides a method for automated robot interaction. In this application, the executing entity of the automated robot interaction method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the automated robot interaction method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0050] Reference Figure 1 The diagram shown is a flowchart illustrating an automatic robot interaction method according to an embodiment of the present invention. In this embodiment, the automatic robot interaction method includes steps S1-S5:

[0051] S1. Capture user behavior images based on the interactive robot, and analyze the human posture of the behavior images using a posture analysis model pre-trained in the interactive robot.

[0052] This invention captures user behavior images based on an interactive robot, and analyzes the human posture of the behavior images using a posture analysis model pre-trained in the interactive robot. The posture analysis model can be used to analyze the user's posture in the behavior images, so as to facilitate subsequent action decomposition of the human posture.

[0053] For example, in the healthcare field, interactive robots can capture behavioral images of doctors performing procedures such as auscultation and examination on patients, and analyze human posture to simulate these procedures within the interactive robot. The interactive robot involves a computer system engaging in human-computer dialogue with an operator or programmer to control and operate the robot. It can independently perform functions such as trajectory planning and simple obstacle avoidance, but is still subject to external control. The behavioral images are captured by the interactive robot showing the user's movements, and the human posture is the user's action posture within these behavioral images. Furthermore, the interactive robot's camera can also capture these behavioral images.

[0054] As an embodiment of the present invention, the step of analyzing the human posture of the behavior image using a posture analysis model pre-trained in the interactive robot includes: performing background separation on the behavior image using a background separation algorithm in the posture analysis model to obtain a target image; performing feature extraction on the target image using a convolutional layer in the posture analysis model to obtain image features; performing feature matching on the image features and feature postures in a preset posture feature database using a matching algorithm in the posture analysis model to obtain a feature matching value; and when the feature matching value is greater than a preset threshold, using the feature matching value corresponding to the feature matching value as the human posture of the behavior image.

[0055] The target image is the image obtained after background separation of the behavior image, removing background images that are irrelevant to the user from the behavior image. The image feature is an image with representational significance in the target image. The feature matching value is the degree of matching between the image feature and the feature pose. The preset threshold can be 0.8, or it can be set according to the actual business scenario.

[0056] Furthermore, as an optional embodiment of the present invention, the background separation algorithm includes:

[0057]

[0058] Where D(a, b) represents the target image, h x (a, b) represent the image corresponding to the x-th frame in the action image, h x+1 (a, b) represents the image corresponding to the (x+1)th frame in the action image, K represents the binarization threshold of the image corresponding to the xth frame and the image corresponding to the (x+1)th frame, and others represents other images in the action image.

[0059] Furthermore, as an optional embodiment of the present invention, the step of using the matching algorithm in the pose analysis model to perform feature matching between the image features and the feature poses in a preset pose feature database to obtain feature matching values ​​includes:

[0060] The feature matching value can be calculated using the following formula:

[0061]

[0062] Among them, T d Let G represent the feature matching value between the image feature and the pose feature database, R represent the total number of corresponding image features, i represent the initial feature of the image feature, and ln G represent the feature matching value between the image feature and the pose feature database. h MAXln G represents the mapping value of the h-th image feature. h MAXln F represents the maximum value among the h-th image feature mapping values.h MINlnG represents the maximum value of the mapping for the h-th feature pose. h Let F represent the minimum value of the h-th image feature mapping, MINln F h δ represents the minimum value of the mapping value of the h-th feature pose, and δ represents the matching coefficient between the image feature and the pose feature.

[0063] S2. Based on the human posture, construct the user's behavior screen to decompose the user's body movements, obtain multiple user decomposed movements, and calculate the decomposition parameters of each user decomposed movement.

[0064] This invention constructs a behavioral image of the user based on the human posture, which can be converted into a corresponding behavioral image, so as to facilitate subsequent action decomposition of the user based on the behavioral image. The behavioral image is the user's dynamic information, such as raising a hand or holding a stethoscope.

[0065] As an embodiment of the present invention, constructing the user's behavior screen based on the human body posture includes: counting the number of human body postures; calculating the dynamic frame rate corresponding to the human body posture based on the number of postures; sorting the human body postures according to a preset sorting rule to obtain sorted postures; creating a dynamic environment corresponding to the human body posture; and generating the user's behavior screen in the dynamic environment by combining the dynamic frame rate and the sorted posture.

[0066] Wherein, the number of poses is the total number of human poses, the dynamic frame rate is the frequency at which the human pose appears continuously on the display, the preset sorting rule is the order in which the human poses are formed, the sorted poses are the poses obtained by sorting each pose in the human poses in chronological order, and the dynamic environment is the dynamic environment in which the human poses are formed. Furthermore, the number of human poses can be counted using the COUNT function, the dynamic frame rate corresponding to the human pose can be calculated using the fixed time frame method, the human poses can be sorted according to the preset sorting rule using Shell sort, and the dynamic environment corresponding to the human pose can be created using the virtual machine of the interactive robot.

[0067] This invention decomposes the user's limb movements to obtain multiple user decomposed movements. The user's movements can be decomposed into local movements corresponding to each part of the body, so that the interactive robot can interact better. The multiple user decomposed movements are multiple local movements obtained by decomposing the user's overall movements, such as raising a hand, holding a stethoscope to listen to the heart, etc.

[0068] As an embodiment of the present invention, the step of decomposing the user's limb movements to obtain multiple user decomposed movements includes: identifying the user image of the user in the behavior screen, and labeling each limb part of the user image to obtain multiple user limb parts, extracting the user limb movements corresponding to the multiple user limb parts respectively, and decomposing each limb movement in the user limb movements to obtain multiple user decomposed movements.

[0069] The user image is the image of the user in the behavior scene, and the multiple user limb parts are the limb parts of the user image that have actions. Furthermore, the user image in the behavior scene can be identified by OCR recognition technology, and each limb part of the user image can be labeled by a labeling tool, such as a color labeling tool. The user limb actions corresponding to the multiple user limb parts can be extracted by the above-mentioned OCR recognition technology, and the image corresponding to the user limb actions can be segmented by a frame segmentation tool to decompose each limb action in the user limb actions.

[0070] This invention can understand the motion parameters of the multiple user decomposed actions by calculating the decomposition parameters of each user decomposed action. The decomposition parameters are the parameters corresponding to the user decomposed action, such as raising the hand, the angle and height of the arm raised, etc.

[0071] As an embodiment of the present invention, the calculation of decomposition parameters for each user decomposition action includes: constructing a three-dimensional coordinate system corresponding to the user decomposition action based on the user image, obtaining the initial coordinates of the user decomposition action in the three-dimensional coordinate system, running the user decomposition action in the three-dimensional coordinate system, obtaining the termination coordinates corresponding to the user decomposition action, and recording the motion trajectory of the user decomposition action, and calculating the decomposition parameters for each user decomposition action by combining the initial coordinates, the termination coordinates, and the motion trajectory.

[0072] Wherein, the initial coordinates are the initial coordinate point information corresponding to the user decomposition action in the three-dimensional coordinate system, the termination coordinates are the coordinate point information corresponding to the end of the user decomposition action in the three-dimensional coordinate system, and the motion trajectory is the action path of the user decomposition action during its execution.

[0073] Furthermore, as an optional embodiment of the present invention, the three-dimensional coordinate system corresponding to the user decomposed action can be constructed by a coordinate system tool. The coordinate system tool is compiled by a scripting language. The initial coordinates and the termination coordinates can be obtained by the coordinate extractor in the three-dimensional coordinate system. The motion trajectory of the user decomposed action can be recorded by a trajectory recorder. The decomposition parameters of each user decomposed action can be calculated by the OpenPose algorithm.

[0074] S3. Based on the decomposition parameters, simulate the user's action screen in the interactive robot to obtain a simulated action screen, configure the interaction instructions for the simulated action screen, and execute the interaction events corresponding to the simulated action screen through the interaction instructions to obtain the interaction result of the interactive robot.

[0075] This invention simulates the user's actions in the interactive robot according to the decomposition parameters to obtain simulated action images, which facilitates the interactive robot's learning and provides a guarantee for the subsequent configuration of corresponding interaction commands. The simulated action images are the user's actions simulated by the interactive robot according to the decomposition parameters.

[0076] As an embodiment of the present invention, the step of simulating the user's action screen in the interactive robot according to the decomposition parameters to obtain a simulated action screen includes: obtaining the simulation parameters corresponding to the decomposition parameters in the interactive robot, defining the simulation code of the simulation parameters, constructing a virtual user corresponding to the user in the interactive robot, and simulating the action screen of the virtual user in the interactive robot according to the simulation code to obtain a simulated action screen.

[0077] Wherein, the simulation parameters are the virtual data corresponding to the decomposition parameters in the interactive robot, the simulation code is the computer language corresponding to the simulation parameters, and the virtual user is a simulated three-dimensional image built in the interactive robot based on the user. Furthermore, the simulation parameters corresponding to the decomposition parameters in the interactive robot can be obtained through a simulation data generator, and the simulation parameters can be coded using the Java language.

[0078] This invention configures the interactive instructions of the simulated action screen, which facilitates subsequent interaction with the interactive robot by executing the interactive instructions. The interactive instructions are instructions to control the interactive robot to perform action simulation.

[0079] As an embodiment of the present invention, configuring the interactive instructions of the simulated action screen includes: extracting the action category in the simulated action screen, identifying the limb category corresponding to the action category, combining the action category and the limb category to generate multiple action instructions in the interactive robot, and merging the action instructions to obtain the interactive instructions of the simulated action screen.

[0080] The action category refers to the type of action performed by the simulated user in the simulated action screen, such as raising a hand or holding a stethoscope. The limb category refers to the limb corresponding to the action category in the simulated action screen. The multiple action instructions are the commands to execute the corresponding actions. Furthermore, the action category in the simulated action screen can be extracted using the find function, and the multiple action instructions can be generated using an instruction generator.

[0081] This invention executes the interactive events corresponding to the simulated action screen through the interactive instructions, obtains the interaction results of the interactive robot, and thus completes the action simulation between the interactive robot and the user. It can understand the interaction accuracy and efficiency of the interactive robot. The interactive events are the events that the robot needs to imitate, and the interaction results are the results obtained by the interactive robot executing the interactive instructions. Furthermore, the execution of the interactive events corresponding to the simulated action screen can be realized by the controller in the interactive robot.

[0082] This invention captures user behavior images using an interactive robot and analyzes the human posture in the behavior images using a pre-trained posture analysis model within the interactive robot. The posture analysis model can analyze the user's posture in the behavior images to facilitate subsequent action decomposition. Based on the human posture, this invention constructs a user behavior scene, converting the human posture into a corresponding behavior scene, which is then used for subsequent action decomposition. Furthermore, based on the decomposition parameters, this invention simulates the user's action scene in the interactive robot to obtain a simulated action scene, facilitating the interactive robot's learning and providing a basis for configuring corresponding interaction commands. Therefore, the robot automatic interaction method provided by this invention can improve the efficiency of robot automatic interaction.

[0083] like Figure 2 The diagram shown is a functional block diagram of an automatic robot interaction device provided in an embodiment of the present invention.

[0084] The robot automatic interaction device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the robot automatic interaction device 100 may include a posture analysis module 101, a motion decomposition module 102, and an interaction execution module 103. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and is stored in the memory of the electronic device.

[0085] In this embodiment, the functions of each module / unit are as follows:

[0086] The posture analysis module 101 is used to analyze the human posture of the behavior image based on the user's behavior image captured by the interactive robot and using a posture analysis model pre-trained in the interactive robot.

[0087] The motion decomposition module 102 is used to construct the user's behavior scene based on the human posture, so as to decompose the user's limb movements, obtain multiple user decomposed movements, and calculate the decomposition parameters of each user decomposed movement.

[0088] The interaction execution module 103 is used to simulate the user's action screen in the interactive robot according to the decomposition parameters, obtain the simulated action screen, configure the interaction instructions of the simulated action screen, execute the interaction event corresponding to the simulated action screen through the interaction instructions, and obtain the interaction result of the interactive robot.

[0089] In detail, each module in the robot automatic interaction device 100 described in this application embodiment adopts the same characteristics as described above during use. Figure 1 The same technical means are used in the automatic robot interaction method described in the article, and can produce the same technical effect, so it will not be repeated here.

[0090] like Figure 3 The diagram shown is a structural schematic of an electronic device 1 for implementing an automatic robot interaction method according to an embodiment of the present invention.

[0091] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program, such as a robot automatic interaction method program, stored in the memory 11 and capable of running on the processor 10.

[0092] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device 1, connecting various components of the electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing robot automatic interaction method programs) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0093] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a robot's automatic interaction method program, but also to temporarily store data that has been output or will be output.

[0094] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0095] The communication interface 13 is used for communication between the electronic device 1 and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0096] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0097] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0098] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0099] The robot automatic interaction method program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions, which, when run in the processor 10, can achieve the following:

[0100] The system captures user behavior images based on an interactive robot, and analyzes the human posture of the behavior images using a posture analysis model pre-trained in the interactive robot.

[0101] Based on the human posture, a behavioral picture of the user is constructed to decompose the user's body movements, resulting in multiple user decomposed movements, and decomposition parameters for each user decomposed movement are calculated.

[0102] Based on the decomposition parameters, the user's actions are simulated in the interactive robot to obtain a simulated action screen. Interaction instructions for the simulated action screen are configured, and the corresponding interaction events are executed through the interaction instructions to obtain the interaction result of the interactive robot.

[0103] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.

[0104] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0105] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0106] The system captures user behavior images based on an interactive robot, and analyzes the human posture of the behavior images using a posture analysis model pre-trained in the interactive robot.

[0107] Based on the human posture, a behavioral picture of the user is constructed to decompose the user's body movements, resulting in multiple user decomposed movements, and decomposition parameters for each user decomposed movement are calculated.

[0108] Based on the decomposition parameters, the user's actions are simulated in the interactive robot to obtain a simulated action screen. Interaction instructions for the simulated action screen are configured, and the corresponding interaction events are executed through the interaction instructions to obtain the interaction result of the interactive robot.

[0109] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0110] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0112] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0113] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0114] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0115] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for automatic robot interaction, characterized in that, The method includes: The system captures user behavior images based on an interactive robot, and analyzes the human posture of the behavior images using a posture analysis model pre-trained in the interactive robot. Based on the human posture, a behavioral picture of the user is constructed to decompose the user's body movements, resulting in multiple user decomposed movements, and decomposition parameters for each user decomposed movement are calculated. Based on the decomposition parameters, the user's action screen is simulated in the interactive robot to obtain a simulated action screen, and the interaction instructions of the simulated action screen are configured. Through the interaction instructions, the interaction events corresponding to the simulated action screen are executed to obtain the interaction result of the interactive robot. The step of constructing the user's behavioral image based on the human posture includes: The number of human body poses is counted, and the dynamic frame rate corresponding to the human body pose is calculated based on the number of poses. The human body postures are sorted according to a preset sorting rule to obtain sorted postures, and a dynamic environment corresponding to the human body postures is created. By combining the dynamic frame rate and the sorting posture, the user's behavior is generated in the dynamic environment; The process of decomposing the user's body movements yields multiple user-decomposed movements, including: Identify the user image in the behavior screen; Each limb part of the user image is labeled to obtain multiple user limb parts; Extract the user limb movements corresponding to the multiple user limb parts respectively; Each limb movement in the user's limb movements is decomposed into multiple user decomposed movements.

2. The robot automatic interaction method as described in claim 1, characterized in that, The step of analyzing the human posture of the behavior image using a posture analysis model pre-trained in the interactive robot includes: The background separation algorithm in the posture analysis model is used to separate the background of the behavior image to obtain the target image; The convolutional layer in the pose analysis model is used to extract features from the target image to obtain image features; The matching algorithm in the pose analysis model is used to perform feature matching between the image features and the feature poses in the preset pose feature database to obtain feature matching values; When the feature matching value is greater than a preset threshold, the feature matching value corresponding to the feature matching value is taken as the human pose of the behavior image.

3. The robot automatic interaction method as described in claim 2, characterized in that, The background separation algorithm includes: in, Represents the target image. This represents the image corresponding to the x-th frame in the action image. Represents the first in the behavior image The image corresponding to the x-th frame, K represents the image corresponding to the x-th frame and the image corresponding to the x-th frame. The binarization threshold of the image corresponding to the frame. Other images in the behavior image.

4. The robot automatic interaction method as described in claim 1, characterized in that, The calculation of decomposition parameters for each user decomposition action includes: Based on the user image, construct a three-dimensional coordinate system corresponding to the user's decomposed actions; Obtain the initial coordinates of the user's decomposed action in the three-dimensional coordinate system; Run the user decomposition action in the three-dimensional coordinate system and obtain the termination coordinates corresponding to the user decomposition action; Record the motion trajectory of the user's decomposed actions, and calculate the decomposition parameters for each user's decomposed actions by combining the initial coordinates, the termination coordinates, and the motion trajectory.

5. The robot automatic interaction method as described in claim 1, characterized in that, The step of simulating the user's actions in the interactive robot based on the decomposition parameters to obtain simulated action footage includes: Obtain the simulation parameters corresponding to the decomposition parameters in the interactive robot, and define the simulation code for the simulation parameters; A virtual user corresponding to the user is constructed in the interactive robot; Based on the simulation code, the virtual user's actions are simulated in the interactive robot to obtain simulated action footage.

6. A robot automatic interaction device for implementing the robot automatic interaction method as described in any one of claims 1 to 5, characterized in that, The device includes: The posture analysis module is used to capture user behavior images based on the interactive robot and analyze the human posture of the behavior images using a posture analysis model pre-trained in the interactive robot. The motion decomposition module is used to construct the user's behavior scene based on the human posture, so as to decompose the user's limb movements, obtain multiple user decomposed movements, and calculate the decomposition parameters of each user decomposed movement. The interaction execution module is used to simulate the user's action screen in the interactive robot according to the decomposition parameters, obtain the simulated action screen, configure the interaction instructions of the simulated action screen, execute the interaction events corresponding to the simulated action screen through the interaction instructions, and obtain the interaction result of the interactive robot.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the robot autonomous interaction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot automatic interaction method as described in any one of claims 1 to 5.

Images