Processing method and device for social interaction with a robot

Abstract

A processing method for social interaction with a robot is disclosed, which is applicable to an interactive robot and includes: converting a sound detected within an activity field of the interactive robot into a sound signal; performing speech recognition on the sound signal to generate a speech string, and determining whether the speech string matches a first command keyword; when the speech string matches the first command keyword, controlling a camera circuit to capture an image, and selecting a new model from multiple candidate character models stored in a storage based on the image being captured; and updating a retrieval database stored in the storage by utilizing the new model, so as to display the new model on a display of the interactive robot.

Description

Technical Field

[0001] This disclosure relates to a technology for robot interaction, and more particularly to a method and apparatus for processing robot social interactions. Prior Technology

[0002] In recent years, robots have been widely used in homes, such as robotic vacuum cleaners, assistive robots, and robotic pets. However, typical household robots usually have only simple and singular functions, making it difficult for them to engage in complex social activities with users. Furthermore, loneliness is a common problem among people living alone and the elderly in modern society, with the loneliness being particularly severe among the elderly, even affecting their mental health. Therefore, how to utilize household robots to engage in social interaction with users and improve their psychological well-being is a problem that those skilled in the art urgently need to solve. Summary of the Invention

[0003] The main purpose of this disclosure is to provide a method and device for processing social interaction of robots, which can enhance the social interaction of interactive robots in general home use and improve the psychological state of users through social interaction.

[0004] To achieve the above objectives, this disclosure proposes a method for processing social interactions of robots, applicable to an interactive robot, comprising: Step a) When a sound is detected in an activity area of ​​the interactive robot using a sound receiving circuit, the sound is converted into a sound signal; Step b) Using a processor, the sound signal is subjected to speech recognition to generate a speech string, and it is determined whether the speech string matches a first instruction keyword; Step c) When the voice string matches the first instruction keyword, the processor controls a camera circuit to capture an image, and selects a new model from multiple candidate character models in a storage device based on the image; and Step d) The processor updates a database in the storage with the new model so that the new model can be displayed on a display of the interactive robot.

[0005] To achieve the above objectives, this disclosure proposes a processing device for robot interaction, disposed in an interactive robot, comprising: A camera circuit, configured to capture an image; A sound receiving circuit is configured to convert sound into an audio signal when sound is detected in an activity area of ​​the interactive robot; A storage device configured to store an acquisition database, multiple candidate character models, and multiple instructions; and A processor, connected to the camera circuit, the radio circuit, and the memory, accesses the plurality of instructions to perform the following actions: Action a) Perform speech recognition on the sound signal to generate a speech string, and determine whether the speech string matches a first instruction keyword; Action b) When the voice string matches the first instruction keyword, control the camera circuit to capture the image, and select a new model from the multiple candidate character models based on the image; and Action c) Update the acquired database with the new model so that the new model can be displayed on a monitor on the interactive robot.

[0006] Compared to previous technologies, this invention combines voice recognition with the image and voice interaction of interactive robots, and utilizes interactive robots to engage in dialogue with users and visual interaction with virtual characters, thereby enhancing the social interaction of interactive robots and improving the user's psychological state through social interaction. Simple Explanation of the Diagram

[0007] Figure 1A illustrates a block diagram of a processing device for robot social interaction in some embodiments.

[0008] Figure 1B illustrates a schematic diagram of an interactive robot in an activity area in some embodiments.

[0009] Figure 2 illustrates a flowchart of a method for processing robot social interactions as disclosed in some embodiments.

[0010] Figure 3 illustrates a schematic diagram of the selected candidate role models disclosed in some embodiments.

[0011] Figure 4 illustrates a flowchart of several steps further included in the processing method for robot social interaction disclosed in some embodiments.

[0012] Figure 5 illustrates a schematic diagram of multiple role groups disclosed in some embodiments.

[0013] Figure 6 illustrates a flowchart of several steps further included in the processing method for robot social interaction disclosed in some other embodiments. Implementation

[0014] Referring to FIG1A, FIG1A illustrates a block diagram of a processing apparatus 100 for robot social interaction disclosed in some embodiments. As shown in FIG1A, in this embodiment, the processing apparatus 100 for robot social interaction includes a camera circuit 110, a microphone circuit 120, a processor 130, and a storage device 140. The processor 130 is coupled to the camera circuit 110, the microphone circuit 120, and the storage device 140.

[0015] In this embodiment, a processing device 100 for robot social interaction is disposed on the interactive robot to perform various controls or operations on the interactive robot. In some embodiments, the interactive robot can move in a specific activity area and is capable of engaging in social interactions with the user as described in the following paragraphs. In other words, the interactive robot is designed to communicate naturally with the user in a specific activity area and to respond according to the user's behavior as described in the following paragraphs. In some embodiments, the interactive robot can be any type of home robot with voice and image interaction capabilities (e.g., a doll-type robot or a humanoid social assistance robot). In some embodiments, the activity area can be any type of place for the user to move around or rest (e.g., a living room or bedroom). In some embodiments, the user can be an ordinary person, an elderly person, or a person with mobility impairment in the activity area.

[0016] In this embodiment, the camera circuit 110 is used to capture images. In some embodiments, the camera circuit 110 may be located at any position on the interactive robot where the activity area can be captured (e.g., the position of a surface on the interactive robot facing the direction of movement of the interactive robot). In some embodiments, the camera circuit 110 may be implemented by any image capturing circuit (e.g., optical camera circuit, infrared camera circuit, or 3D camera circuit, etc.).

[0017] In this embodiment, the sound receiving circuit 120 is used to convert sound into an audio signal when sound is detected in the activity area of ​​the interactive robot. In some embodiments, the sound receiving circuit 120 may be disposed at any location on the interactive robot where sound in the activity area can be detected (e.g., at a location on a surface of the interactive robot facing the direction of movement of the interactive robot). In some embodiments, the sound receiving circuit 120 may be implemented by any microphone with sound-capturing function (e.g., a condenser microphone, a piezoelectric microphone, or an optical microphone, etc.).

[0018] In this embodiment, the storage 140 stores the acquisition database 141 and a plurality of instructions. In some embodiments, the acquisition database 141 is used to store the acquisition models OM1~OMS (e.g., virtual two-dimensional or three-dimensional image models) already obtained by the user, where S equals the number of virtual two-dimensional or three-dimensional image models already obtained by the user. In some embodiments, the instructions can be implemented by any software or firmware. In this embodiment, the processor 130 is used to access these instructions to execute the processing method for robot social interaction described in subsequent paragraphs. In some embodiments, the processor 130 is further used to perform various controls on various circuits or components on the interactive robot (e.g., displays, audio circuits, or steering mechanisms, etc.).

[0019] In some embodiments, the storage 140 further stores multiple candidate character models CM1~CMT, which are all virtual two-dimensional or three-dimensional image models that the user may obtain when interacting with the interactive robot. In other words, when interacting with the interactive robot, the user may obtain several candidate character models from the candidate character models CM1~CMT, and these candidate character models will be stored in the acquisition database 141 as acquisition models OM1~OMS. In some embodiments, the candidate character models CM1~CMT are pre-designed by designers and stored in the storage 140.

[0020] The following describes a scenario where the processor 130 controls an interactive robot using a practical example. Referring also to FIG1B, FIG1B illustrates a schematic diagram of an interactive robot 200 in an activity area F1 as disclosed in some embodiments. As shown in FIG1B, in this embodiment, a processing device 100 for robot social interaction is provided on the interactive robot 200, which includes a display 210, a sonar sensor 220, and an audio circuit 230. The display 210 is used to display images for visual interaction with the user. The sonar sensor 220 is used to detect the distance to obstacles in front of the interactive robot 200 in the direction of its movement to prevent collisions. The audio circuit 230 is used to play sounds for auditory interaction with the user.

[0021] In some embodiments, the sonar sensor 220 is positioned on a surface of the interactive robot 200 facing the direction of movement. Here, the sonar sensor 220 can detect obstacles (e.g., obstacle OST1 or obstacle OST2) in front of the interactive robot 200 in the direction of movement. In the activity field F1, the processor 130 can control the interactive robot 200 to continue moving in the direction of movement D1 when the sonar sensor 220 detects that the distance between itself and the obstacle OST1 is not less than a distance threshold. Conversely, when the sonar sensor 220 detects that the distance between itself and the obstacle OST1 is less than the distance threshold, the processor 130 can control the interactive robot 200 to adjust its current direction of movement D1 to direction of movement D2 (e.g., turn 45 degrees clockwise).

[0022] Next, when the sonar sensor 220 detects that the distance between the robot and the obstacle OST2 in front is not less than a distance threshold, the processor 130 can control the interactive robot 200 to continue moving in the direction of movement D2. Conversely, when the sonar sensor 220 detects that the distance between the robot and the obstacle OST2 in front is less than the distance threshold, the processor 130 can control the interactive robot 200 to adjust its current direction of movement D2 to direction of movement D3 (e.g., turning 45 degrees clockwise). In some embodiments, the distance threshold may be preset by the user or obtained statistically from multiple experiments (e.g., 10 cm).

[0023] In some embodiments, the display 210 can be implemented using any type of display screen (e.g., a light-emitting diode screen, an organic light-emitting diode screen, or a cathode ray tube screen). In some embodiments, the sonar sensor 220 can be implemented using any type of sonar (e.g., active sonar, passive sonar, or reflective sonar). In some embodiments, the broadcasting circuit 230 can be implemented using any type of loudspeaker for playing sound (e.g., a dynamic loudspeaker, an electrostatic loudspeaker, or a dome loudspeaker).

[0024] In some embodiments, the camera circuit 110 and the audio recording circuit 120 may also be disposed on the surface of the interactive robot 200 facing the direction of movement. In this way, the camera circuit 110 can take pictures in the direction of movement of the interactive robot 200, and the audio recording circuit 120 can also record audio in the direction of movement of the interactive robot 200.

[0025] Referring also to Figure 2, which illustrates a flowchart of a robot social interaction processing method disclosed in some embodiments, this robot social interaction processing method is applicable to the robot social interaction processing apparatus 100 of Figure 1A. As shown in Figure 2, firstly, in step S210, the sound receiving circuit 120 converts sound into a sound signal when it detects sound in the activity field F1 of the interactive robot 200. In other words, as long as the sound receiving circuit 120 detects sound in the activity field F1, the sound receiving circuit 120 can convert the detected sound into a sound signal.

[0026] In step S220, processor 130 performs speech recognition on the sound signal to generate a speech string, and determines whether the speech string matches the first instruction keyword. When the speech string matches the first instruction keyword, processor 130 executes step S230. Otherwise, when the speech string does not match the first instruction keyword, processor 130 executes step S250 as described in the following paragraph (details to follow).

[0027] In some embodiments, speech recognition can be implemented by any algorithm (e.g., the whisper-small model proposed by OpenAI, the hidden Markov model (HMM), the Gaussian mixture model (GMM), or the long short-term memory model (LSTM model)) used to accurately map linguistic units (e.g., syllables or words) in speech to their corresponding written forms.

[0028] In some embodiments, the first instruction keyword may be associated with one of a plurality of keyword types. In some embodiments, the plurality of keyword types include any type of item placed in a typical residence (e.g., fruit, furniture, plants, and appliances). For example, when a user wants to set the keyword type associated with the first instruction keyword to fruit, the user can use the processor 130 to further set the first instruction keyword to "detect fruit". And when a user wants to set the keyword type associated with the first instruction keyword to appliances, the user can use the processor 130 to further set the first instruction keyword to "detect appliances".

[0029] In some embodiments, the processor 130 determines whether a first instruction keyword exists in the speech string. When the first instruction keyword exists in the speech string, the processor 130 determines that the speech string matches the first instruction keyword. Conversely, when the first instruction keyword does not exist in the speech string, the processor 130 determines that the speech string does not match the first instruction keyword. For example, assuming the first instruction keyword is "detect fruit" and the speech string is "I want to detect fruit", the processor 130 can determine that the first instruction keyword exists in the speech string and determines that the speech string matches the first instruction keyword.

[0030] In step S230, the processor 130 controls the camera circuit 110 to capture images and selects a new model from candidate character models CM1 to CMT based on the images, where T can be any positive integer. In some embodiments, image recognition can be implemented by any algorithm (e.g., YOLOv8n model, faster region-based convolutional neural network (faster R-CNN) model, or histogram of oriented gradients (HOG) algorithm, etc.) used to identify specific objects (i.e., objects corresponding to one of the multiple candidate object types) from the images.

[0031] In some embodiments, the processor 130 performs image recognition on the image to determine whether there is an object in the image that corresponds to one of a plurality of candidate object types. When there is an object in the image that corresponds to one of the candidate object types, the processor 130 selects a candidate character model from the candidate character models CM1~CMT that corresponds to one of the candidate object types as a new model. Conversely, when there is no object in the image that corresponds to one of the candidate object types, the processor 130 determines whether the audio receiving circuit 120 generates a new sound signal, and executes step S220 again when the audio receiving circuit 120 generates a new sound signal.

[0032] In some embodiments, the keyword type associated with the first instruction keyword indicates multiple candidate object types. For example, assuming the keyword type associated with the first instruction keyword is "fruit," this keyword type indicates oranges, guavas, and strawberries, etc. And assuming the keyword type associated with the first instruction keyword is "appliance," this keyword type indicates electric fans, refrigerators, and televisions, etc. In some embodiments, the multiple candidate object types correspond one-to-one to a subset of the candidate role models.

[0033] In some embodiments, the candidate character models CM1~CMT include multiple common models and multiple rare models, with the number of common models being much greater than the number of rare models. Each of the common and rare models has a pre-defined character level. In some embodiments, the character levels of the common and rare models can also be pre-defined by the designer. In some embodiments, a portion of the candidate character models are all common models and have the lowest character level (e.g., all are character level 1).

[0034] The following example illustrates the generation of the new model. Referring to Figure 3, which shows a schematic diagram of the candidate role model CM1 selected in some embodiments, as disclosed in Figure 3, assuming that the keyword type related to the first instruction keyword is fruit and the processor 130 identifies the object OJ1 corresponding to the orange among multiple candidate object types from the image IMG1, the processor 130 can select the candidate role model CM1 corresponding to the orange from multiple candidate role models CM1~CMT as the new model, wherein the candidate role model CM1 is a normal model and has a role level of 1.

[0035] Assuming the keyword type related to the first instruction keyword is fruit and the processor 130 identifies the object OJ2 corresponding to guava among multiple candidate object types from the image IMG2, the processor 130 can select the candidate character model CM3 corresponding to guava from multiple candidate character models CM1~CMT as the new model. The candidate character model CM3 is also a normal model and has a character level of 1.

[0036] Assuming the keyword type related to the first instruction keyword is fruit and the processor 130 identifies the object OJ3 corresponding to strawberry among multiple candidate object types from the image IMG3, the processor 130 can select the candidate character model CM5 corresponding to strawberry from multiple candidate character models CM1~CMT as the new model. The candidate character model CM5 is also a normal model and has a character level of 1.

[0037] Returning to Figure 2, in step S240, the processor 130 updates the acquisition database 141 using the newly added model, so as to display the newly added model on the display 210 of the interactive robot 200. In other words, the user can view the image of the newly added model on the display 210 of the interactive robot 200 to know the currently acquired new model. In some embodiments, the processor 130 stores the newly added model as a new acquisition model in the acquisition database 141. In this way, the processor 130 can continuously collect new models to update the acquisition models OM1~OMS in the acquisition database 141 using the above method.

[0038] Through the steps described above, in this disclosure, a user can emit a sound corresponding to a specific first command keyword when the interactive robot 200 moves in front of an item corresponding to one of the candidate object types (e.g., an orange). When the processor 130 determines that the user has emitted a sound corresponding to the specific first command keyword and that the item has been photographed, the processor 130 can obtain a new model corresponding to this item for the user to view. In this way, the user can use voice commands to instruct the interactive robot 200 to obtain various new models for interaction with the interactive robot 200.

[0039] Referring also to Figure 4, Figure 4 illustrates a flowchart of several steps S250-S290 further included in the processing method for robot social interaction disclosed in some embodiments. As shown in Figure 4, in step S250, the processor 130 determines whether the voice string matches the second instruction keyword. When the voice string matches the second instruction keyword, the processor 130 executes step S260. Conversely, when the voice string does not match the second instruction keyword, the processor 130 executes step S290. In some embodiments, the second instruction keyword is related to an upgrade instruction (e.g., the second instruction keyword is "upgrade the character" or "the character wants to upgrade," etc.).

[0040] In some embodiments, the processor 130 determines whether a second instruction keyword exists in the voice string. When the second instruction keyword exists in the voice string, the processor 130 determines that the voice string matches the second instruction keyword. Conversely, when the second instruction keyword does not exist in the voice string, the processor 130 determines that the voice string does not match the second instruction keyword. For example, assuming the second instruction keyword is "upgrade the character" and the voice string is "I want to upgrade the character now", the processor 130 can determine that the second instruction keyword exists in the voice string and determine that the voice string matches the second instruction keyword.

[0041] In step S260, the processor 130 reads the acquired models OM1~OMS and their respective role levels from the acquisition database 141, and assigns a role number to each of the acquired models OM1~OMS. In some embodiments, the processor 130 controls the display 210 of the interactive robot 200 to sequentially display the acquired models OM1~OMS and their respective role numbers, so that the user can view the currently acquired models OM1~OMS and their respective role numbers. In some embodiments, the processor 130 sequentially assigns different role numbers to the acquired models OM1~OMS. In some embodiments, the processor 130 may assign role numbers to the acquired models OM1~OMS sequentially from smallest to largest (e.g., sequentially assigning role numbers 1~S to the acquired models OM1~OMS).

[0042] In step S270, when one of the role numbers is selected, the processor 130 selects the role model to be upgraded from the acquired models OM1~OMS according to the selected role number, and upgrades the role level of the role model to be upgraded. In some embodiments, when the sound receiving circuit 120 detects another sound in the activity field F1 of the interactive robot 200, it converts the other sound into another sound signal. Then, the processor 130 performs speech recognition on the other sound signal to generate another speech string, and determines whether the other speech string matches one of the multiple role numbers. Then, when the other speech string matches one of the role numbers, the processor 130 selects the acquired model corresponding to the matched role number as the role model to be upgraded.

[0043] In some embodiments, the processor 130 determines whether a character number exists in another speech string. When it is determined that a character number exists in the other speech string, the processor 130 determines that the other speech string matches a character number. Conversely, when it is determined that a character number does not exist in the other speech string, the processor 130 determines that the other speech string does not match any of these character numbers and executes step S290. For example, assuming that the character number of one acquisition model is 5 and the other speech string is "character number 5", the processor 130 can determine that the character number of this acquisition model exists in the other speech string and determine that the other speech string matches the character number of this acquisition model.

[0044] In step S280, the processor 130 updates the character model to be upgraded according to the increased character level, so as to display the updated character model to be upgraded on the display 210 of the interactive robot 200. In some embodiments, the processor 130 selects from a plurality of candidate virtual characters that match the character model to be upgraded and the increased character level as the updated character model to be upgraded.

[0045] In some embodiments, multiple candidate character models are divided into multiple character groups, and the candidate character models in each character group have character levels. The processor 130 selects a character group that includes the character model to be upgraded. Then, the processor 130 retrieves the candidate character model corresponding to the upgraded character level from the selected character group as the updated character model to be upgraded. In some embodiments, when it is impossible to retrieve a candidate character model corresponding to the upgraded character level from the selected character group, the processor 130 controls the broadcasting circuit 230 of the interactive robot 200 to play a sound corresponding to a prompt statement, wherein the prompt statement indicates that the selected character model to be upgraded cannot be upgraded (e.g., the prompt statement is "Character cannot be upgraded").

[0046] The following uses practical examples to illustrate role groups and role levels. Referring also to Figure 5, Figure 5 illustrates a schematic diagram of multiple role groups G1~G3 disclosed in some embodiments. As shown in Figure 5, candidate role models CM1~CM6 are used as examples. Candidate role models CM1~CM6 can be divided into three role groups G1~G3, where role group G1 includes candidate role models CM1~CM2, role group G2 includes candidate role models CM3~CM4, and role group G3 includes candidate role models CM5~CM6.

[0047] In character group G1, candidate character models CM1~CM2 have character levels 1~2 respectively. In character group G2, candidate character models CM3~CM4 also have character levels 1~2 respectively. In character group G3, candidate character models CM5~CM6 also have character levels 1~2 respectively. Assuming the character model to be upgraded is candidate character model CM1 with character level 1, processor 130 can upgrade character level 1 to character level 2, and obtain candidate character model CM2 with character level 2 from character group G1 as the updated character model to be upgraded.

[0048] Furthermore, assuming the character model to be upgraded is candidate character model CM5 with character level 1, processor 130 can upgrade character level 1 to character level 2, and obtain candidate character model CM6 corresponding to character level 2 from character group G3 as the updated character model to be upgraded. It is worth noting that although this example uses three character groups, each containing two candidate character models, in practical applications, character groups can include more candidate character models, and all candidate character models can be divided into more character groups.

[0049] In step S290, the processor 130 generates a response statement based on the speech string, and plays the sound corresponding to the response statement via the broadcasting circuit 230 of the interactive robot 200. In some embodiments, the processor 130 uses language model processing to generate the response statement corresponding to the speech string. In other words, when the user simply wants to chat (i.e., the speech string cannot match the first instruction keyword and the second instruction keyword), the processor 130 can use language model processing to generate a response statement to reply to the speech string converted from the user's voice, and control the broadcasting circuit 230 to play the sound corresponding to this response statement to the user for chat interaction. In some embodiments, language model processing can be implemented by any large language model used for dialogue (e.g., generative pre-trained transformers (GPT), bidirectional encoder representations from transformers (BERT), or T5 (text-to-text transfer transformer) model).

[0050] Through the above steps, this disclosure can, when determining whether a voice string matches a second instruction keyword, upgrade the role level of one of the acquisition models based on the role number corresponding to the user's re-pronounced voice, and display the image of the acquisition model with the upgraded role level on the display 210 for the user to view. Furthermore, when the user simply wants to chat, this disclosure utilizes language model processing to generate a reply statement, which is then played back using the audio circuit 230 for the user to hear.

[0051] Referring also to Figure 6, Figure 6 illustrates a flowchart of several steps S610-S620 further included in the processing method for robot social interaction disclosed in some other embodiments. As shown in Figure 6, in step S610, the processor 130 times the interaction time of the interactive robot 200, and randomly selects one of the ordinary models as the extracted model every preset time interval during the interaction time. In step S620, the processor 130 stores the extracted model in the acquisition database 141 for display on the display 210 of the interactive robot 200.

[0052] In other words, after a preset time interval, the processor 130 uses a card-drawing mechanism to randomly select one of the common models from the candidate character models CM1 to CMT as the selected model and stores it in the acquisition database 141 (i.e., using this selected model as the new acquisition model). The user can see on the display 210 of the interactive robot 200 that a common model is acquired by the interactive robot 200 after this preset time interval. In some embodiments, the interaction time is the time elapsed after the interactive robot 200 is turned on. In some embodiments, the preset time can be pre-set by the user (e.g., 1 minute).

[0053] In some embodiments, the processor 130 further selects one of the rare models as another extracted model every other preset time period that is longer than a preset time period during the interaction time. Then, the processor 130 stores the other extracted model in the acquisition database 141 so as to display the other extracted model on the display 210 of the interactive robot 200.

[0054] In other words, after another preset time period exceeding the preset time, the processor 130 will also use a card-drawing mechanism to randomly select one of the rare models from all the candidate character models CM1~CMT as another model to be stored in the acquisition database 141 (that is, this other model is also used as the new acquisition model). The user can watch on the display 210 of the interactive robot 200 that the rare model will be acquired by the interactive robot 200 after this other preset time period. In some embodiments, this other preset time period can also be preset by the user (e.g., 5 minutes).

[0055] In some embodiments, the processor 130 further stores the upgrade card model in the acquisition database 141 every other preset time interval less than a preset time during the interaction time, so as to display the upgrade card model on the display 210 of the interactive robot 200, wherein the upgrade card model is used to upgrade the role level of one of the acquisition models in the acquisition database 141. In some embodiments, the upgrade card model is a two-dimensional or three-dimensional model of a virtual card indicating the upgrade of one of the acquisition models (e.g., a two-dimensional card model with the string "upgrade").

[0056] In other words, every other preset time interval less than a preset time, the processor 130 stores an upgrade card model in the acquisition database 141. The user can then see on the display 210 of the interactive robot 200 that the upgrade card model is acquired by the interactive robot 200 every other preset time interval. In some embodiments, this other preset time interval can also be preset by the user (e.g., 30 seconds).

[0057] In some embodiments, when determining whether the aforementioned voice string matches the second instruction keyword, the processor 130 may further determine whether an upgrade card model exists in the database 141. The processor 130 will only begin executing steps S260-S280 when an upgrade card model exists in the database 141. Conversely, when an upgrade card model does not exist in the database 141, the processor 130 controls the broadcasting circuit 230 of the interactive robot 200 to play a sound corresponding to another prompt statement, wherein the prompt statement indicates that an upgrade card model has not been obtained (e.g., the other prompt statement is "No upgrade card").

[0058] Through the above steps, this disclosure can continuously acquire common models, rare models, or upgrade card models, and display these models on the display 210 of the interactive robot 200 for the user to view. In this way, the user can engage in dialogue and visual interaction with the interactive robot 200 according to their needs and the images they see.

[0059] In summary, the method and apparatus for processing robot social interaction disclosed herein can perform voice recognition to determine the user's desired command to the interactive robot. When the user sees the interactive robot move near a specific object, the user can give the robot a command to identify the object. The method and apparatus disclosed herein will then display a virtual character corresponding to the object on the interactive robot's display screen for the user to see. In this way, the method and apparatus disclosed herein can utilize a typical household interactive robot to engage in dialogue and various visual interactions with the user, thereby enhancing the robot's social interaction and improving the user's psychological state through social interaction. Furthermore, when the user wants to upgrade the virtual character's appearance, the user can give the robot an upgrade command and the character's number to be upgraded. The method and apparatus disclosed herein will then display the upgraded virtual character on the interactive robot's display screen for the user to see. Thus, the method and apparatus disclosed herein further allows the user to interact with the virtual character. On the other hand, the method and device for handling robot social interaction disclosed herein can also engage in chat interaction with users without requiring user commands. In this way, the method and device for handling robot social interaction disclosed herein greatly enhance the level of communication with users.

[0060] Although this disclosure has been shown above with reference to embodiments, it is not intended to limit this disclosure. Anyone skilled in the art may make some modifications and refinements without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure shall be determined by the appended claims.

[0061] 100: Processing device for robot social interaction 110: Camera Circuit 120: Radio circuit 130: Processor 140: Storage 141: Accessing the Database OM1~OMS: Obtaining the model CM1~CMT: Candidate Character Models 200: Interactive Robot 210: Monitor 220: Sonar sensor 230: Broadcasting Circuit D1~D3: Movement direction OST1~OST2: Obstacles F1: Activity Area S210~S290, S610~S620: Steps IMG1~IMG3: Images OJ1~OJ3: objects G1~G3: Role Groups

Claims

1. A method for processing social interactions of robots, applicable to an interactive robot, comprising: Step a) When a sound is detected in an activity area of ​​the interactive robot using a sound receiving circuit, the sound is converted into a sound signal; Step b) A processor performs speech recognition on the sound signal to generate a speech string, and determines whether the speech string matches a first instruction keyword; Step c) When the speech string matches the first instruction keyword, the processor controls a camera circuit to capture an image, and selects a new model from multiple candidate character models in a storage based on the image; Step d) The processor updates an acquisition database in the storage using the new model, and displays the new model on a display of the interactive robot; Step e) The processor determines whether the speech string matches a second instruction keyword; Step f) When the speech string matches the second instruction keyword, the processor reads multiple acquisition models and a character level for each of the multiple acquisition models from the acquisition database, and assigns a character number to each of the multiple acquisition models; Step g) When one of the character numbers is selected, the processor selects a character model to be upgraded from the plurality of acquired models according to the selected character number and increases the character level of the character model to be upgraded; and step h) The processor updates the character model to be upgraded according to the increased character level so as to display the updated character model to be upgraded on the display of the interactive robot.

2. The method for processing robot social interactions as described in claim 1, wherein step c) includes: Image recognition is performed on the image to determine whether there is an object in the image that corresponds to one of the multiple candidate object types; And when an object corresponding to one of the candidate object types exists in the image, the processor selects the candidate character model corresponding to one of the candidate object types from the plurality of candidate character models as the new model.

3. The method for processing robot social interactions as described in claim 1, wherein step g) includes: The sound receiving circuit converts another sound into another sound signal when it detects another sound in the activity area of ​​the interactive robot. The processor performs speech recognition on the other sound signal to generate another speech string and determines whether the other speech string matches one of the multiple role numbers. When the other speech string matches one of the role numbers, the processor selects the acquisition model corresponding to the matched role number as the role model to be upgraded.

4. The method for processing robot social interactions as described in claim 1, wherein step h) includes: Step h1) Using the processor, select one of the multiple candidate virtual characters that matches the character model to be upgraded and the level of the upgraded character, and use it as the updated character model to be upgraded.

5. The robot control method as described in claim 4, wherein the plurality of candidate role models are divided into plurality of role groups, and the candidate role model in each role group has the role level, wherein step h1) includes: Using this processor, select the character group that includes the character model to be upgraded; And through the processor, the candidate character model corresponding to the upgraded character level is obtained from the selected character group, and used as the updated character model to be upgraded.

6. The method for processing robot social interaction as described in claim 1, wherein the plurality of candidate role models includes a plurality of common models and a plurality of rare models, the number of the plurality of common models being much greater than the number of the plurality of rare models, and wherein the control method further includes: The processor times an interaction period of the interactive robot, and during the interaction period, after each preset time interval, randomly selects one of the ordinary models as an extraction model; the processor stores the extraction model in the acquisition database, and displays the extraction model on the display of the interactive robot.

7. The method for processing robot social interactions as described in claim 6 further includes: Using the processor, during the interaction time, after each additional preset time exceeding the preset time, one of the rare models is randomly selected as another extraction model; and using the processor, the other extraction model is stored in the acquisition database so as to display the other extraction model on the display of the interactive robot.

8. The method for processing robot social interactions as described in claim 6 further includes: Using the processor, during the interaction time, every other preset time less than the preset time, an upgrade card model is stored in the acquisition database, and the upgrade card model is displayed on the display of the interactive robot, wherein the upgrade card model is used to upgrade the role level of one of the multiple acquisition models in the acquisition database.

9. A processing device for robot interaction, disposed in an interactive robot, comprising: A camera circuit configured to capture an image; a microphone circuit configured to convert sound into an audio signal when sound is detected in an activity area of ​​the interactive robot; a storage device configured to store an acquisition database, multiple candidate character models, and multiple instructions; and a processor connected to the camera circuit, the microphone circuit, and the storage device, and accessing the multiple instructions to perform the following actions: Action a) performing speech recognition on the audio signal to generate a speech string, and determining whether the speech string matches a first instruction keyword; Action b) when the speech string matches the first instruction keyword, controlling the camera circuit to capture the image, and selecting a new model from the multiple candidate character models based on the image; Action c) updating the acquisition database using the new model to display the new model on a display on the interactive robot; Action d) using the processor to determine whether the speech string matches a second instruction keyword. Action e) When the voice string matches the second instruction keyword, the processor reads multiple acquisition models and a role level for each of the multiple acquisition models from the acquisition database, and assigns a role number to each of the multiple acquisition models; Action f) When one of the role numbers is selected, the processor selects a role model to be upgraded from the multiple acquisition models according to the selected role number, and increases the role level of the role model to be upgraded; and Action g) The processor updates the role model to be upgraded according to the increased role level, so as to display the updated role model to be upgraded on the display of the interactive robot.

Images