Interaction method and device, interaction robot, storage medium and product

Abstract

The invention provides an interaction method and device, an interaction robot, a storage medium and a product. Relates to robots or other related fields. The method comprises the steps of obtaining first data of a target user, determining a group category of the target user according to the first data, if the group category of the target user is a target group category, moving to a target position of the target user, generating a corresponding holographic projection image according to target propaganda data corresponding to the target group category, and displaying the holographic projection image. The interactive robot acquires the first data of the user to identify the user of the target group category, moves to the position of the user of the target group category and displays the target propaganda data in a holographic projection mode, so that the resource consumption of manpower, material resources, time and the like is reduced while directional propaganda is performed on the user of the target group, and the propaganda efficiency is improved. And the propaganda efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of robotics or other related fields, and in particular to an interactive method, device, interactive robot, storage medium and product. Background Art

[0002] In the daily production process of activities, event publicity and product introduction are one of the important means to promote products and activities.

[0003] The relevant publicity methods mainly include distributing leaflets by publicity personnel and conducting fixed-point display and publicity at the stationed locations.

[0004] However, the above-mentioned promotional methods require a large amount of manpower, material resources and time, and are highly dependent on the ability and experience of the promoters, making the types of promotional objects and products susceptible to the subjective influence of the promoters, resulting in low efficiency of promotion. Summary of the invention

[0005] The present application provides an interactive method, device, interactive robot, storage medium and product, which obtain the first data of a target user and determine the group category of the target user through the interactive robot. When the target user is in the target group category, the interactive robot moves to the location of the target user and displays the target promotion data through holographic projection, so as to solve the problem of low efficiency of existing promotion and publicity.

[0006] In a first aspect, the present application provides an interaction method, which is applied to an interactive robot. The interaction method comprises:

[0007] Acquire first data of a target user; wherein the first data includes voice data and / or image data;

[0008] Determine, according to the first data, a group category of the target user, where the group category is a target group category or a non-target group category;

[0009] If the group category of the target user is the target group category, the target position of the target user is moved to generate and display a corresponding holographic projection image according to the target promotion data corresponding to the target group category.

[0010] In a possible implementation manner, determining the group category of the target user according to the first data includes:

[0011] Identify the first data to determine characteristic attributes of the target user, where the characteristic attributes include age attributes and / or gender attributes, and the identification method includes voice recognition method and / or image recognition method;

[0012] If the age attribute of the target user matches the target age attribute, and the gender attribute of the target user matches the target gender attribute, then the group category of the target user is determined to be the target group category.

[0013] In a possible implementation manner, identifying the first data to determine the characteristic attributes of the target user includes:

[0014] If the first data includes the image data, performing face recognition on the image data to obtain a face recognition result;

[0015] The face recognition result is input into a first neural network model to obtain the age attribute and / or gender attribute of the target user.

[0016] In a possible implementation manner, identifying the first data to determine the characteristic attributes of the target user includes:

[0017] If the first data includes the voice data, identifying target text information corresponding to the voice data;

[0018] Generate corresponding reply information according to the target text information and display it;

[0019] The age attribute and / or gender attribute of the target user is determined according to the voice data and the reply information.

[0020] In a possible implementation, if the first data includes the voice data, identifying the target text information corresponding to the voice data includes:

[0021] Segmenting the speech data to obtain speech data of multiple time windows, and extracting feature information of the speech data of each time window respectively;

[0022] Inputting feature information of the speech data in each of the time windows into a second neural network model to obtain speech recognition information;

[0023] Mapping the feature information of the speech data in each of the time windows to a text sequence model to obtain preprocessed text information;

[0024] The speech recognition information and the pre-processed text information are matched according to a speech recognition engine to obtain target text information.

[0025] In a possible implementation manner, generating and displaying corresponding reply information according to the target text information includes:

[0026] Performing intent recognition on the target text information to obtain the target intent;

[0027] Generate corresponding reply information according to the target intention;

[0028] Generate corresponding target voice data according to the reply information, and input the target voice data into the acoustic model to obtain voice playback data;

[0029] A first holographic projection image is generated according to the voice playback data and preset projection parameters, and the first holographic projection image is displayed on a display screen.

[0030] In a possible implementation, if the group category of the target user is a target group category, moving to the target position of the target user, generating and displaying a corresponding holographic projection image according to target promotion data corresponding to the target group category, includes:

[0031] If the group category of the target user is the target group category, obtaining the target location of the target user;

[0032] Planning a target path according to the target location, and moving toward the target location according to the target path;

[0033] If the target position is moved to, a second holographic projection image is generated according to the target promotional data, and the second holographic projection image is displayed on the display screen.

[0034] In a possible implementation, if the target position is moved to, a second holographic projection image is generated according to the target promotion data, and the second holographic projection image is displayed on the display screen, including:

[0035] If the target location is moved, target promotion data corresponding to the target group category is obtained;

[0036] A corresponding second holographic projection image is generated according to the target promotion data and preset projection parameters, and the second holographic projection image is displayed on the display screen.

[0037] In a possible implementation, the first data includes user identity authentication information, the interactive robot includes at least one pre-stored identity authentication information, and the method further includes:

[0038] Reading user identity authentication information in the first data;

[0039] matching the user identity authentication information with each of the at least one pre-stored identity authentication information;

[0040] If there is pre-stored identity authentication information matching the user identity authentication information in the at least one pre-stored identity authentication information, determining that the target user is a registered user;

[0041] Generate a login success message based on the user identity authentication information and display it on the display screen;

[0042] The age attribute and / or gender attribute of the target user is determined according to the user identity authentication information.

[0043] In a possible implementation, the method further includes:

[0044] If a radio message is received, then reading the data to be advertised in the radio message;

[0045] The data to be promoted is written into a radio tag, and the radio tag is displayed on a display screen.

[0046] In a second aspect, the present application provides an interactive device, which is applied to an interactive robot, and the interactive device includes:

[0047] The data acquisition module is used to acquire first data of a target user; wherein the first data includes voice data and / or image data;

[0048] The group determination module is used to determine the group category of the target user according to the first data, where the group category is a target group category or a non-target group category;

[0049] The display module is used to, if the group category of the target user is the target group category, move to the target position of the target user, generate and display a corresponding holographic projection image according to the target promotion data corresponding to the target group category.

[0050] In the third aspect, an embodiment of the present application provides an interactive robot, comprising: a movable base, a display screen, a radio sensor, at least one processor and a memory; the memory stores computer execution instructions; at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the interaction method involved in the first aspect and any item of the first aspect above.

[0051] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the interaction method involved in the first aspect and any item of the first aspect is implemented.

[0052] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the interaction method involved in the first aspect and any item of the first aspect.

[0053] The interactive method, device, interactive robot, storage medium and product provided by the present application obtain the first data of the target user through the interactive robot, determine the group category of the target user according to the first data, and if the group category of the target user is the target group category, move to the target position of the target user, and generate and display the corresponding holographic projection image according to the target promotion data corresponding to the target group category. Users of the target group category can be identified, moved to the location of the users of the target group category, and the target promotion data can be displayed by holographic projection, which enriches the promotion methods of the target promotion data, realizes targeted promotion for specific group users, reduces the consumption of resources such as manpower, material resources and time, and improves the promotion efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0055] Figure 1 A schematic diagram of the interactive system architecture provided in an embodiment of the present application;

[0056] Figure 2 A flowchart of an interactive method provided in an embodiment of the present application;

[0057] Figure 3 A flowchart of another interactive method provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of a loss function curve of the first neural network model provided in an embodiment of the present application;

[0059] Figure 5 A flowchart of an interactive method provided in an embodiment of the present application;

[0060] Figure 6 A flowchart of another interactive method provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of an application of an interaction method provided in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of an application of an interaction method provided in an embodiment of the present application;

[0063] Fig. 9 A flowchart of another interactive method provided in an embodiment of the present application;

[0064] Fig.10 A schematic diagram of an application of another interaction method provided in an embodiment of the present application;

[0065] Fig.11 A schematic diagram of the structure of an interactive device provided in an embodiment of the present application;

[0066] Fig.12 A schematic diagram of the structure of an interactive robot provided in an embodiment of the present application.

[0067] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0068] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0069] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0070] It should be noted that the interactive method, device, interactive robot, storage medium and product provided in the present application can be used in the field of robotics, and can also be used in any field other than the field of robotics. The application of the interactive method, device, interactive robot, storage medium and product is not limited.

[0071] In the related art, the promotion method used in daily production activities is mainly to distribute flyers through publicity personnel, and to conduct fixed-point display and publicity at the stationed points. However, the above-mentioned publicity method requires a large amount of manpower, material resources and time, is too dependent on the ability and experience of publicity personnel, the publicity objects are easily influenced by the subjective influence of publicity personnel, and the types of publicity products are more restricted, resulting in low publicity and promotion efficiency.

[0072] The present application obtains the first data of the target user through an interactive robot, determines the group category of the target user based on the first data of the target user, moves to the location of the target user when the target user is in the target group category, and displays the target promotional data corresponding to the target group category through holographic projection, which can enrich the display form of the promotional data, reduce manpower, material resources and promotion time, improve the matching degree between the promotional data and the user, and thus improve the efficiency of the promotion.

[0073] For ease of understanding, the following Figure 1 , the system architecture applicable to the embodiments of the present application is described.

[0074] Figure 1 This is a schematic diagram of the system architecture provided by the embodiment of this application. Figure 1 The interactive system includes an interactive robot 002 and a server 001, and the interactive robot 002 is communicatively connected with the server 001.

[0075] A storage space is provided in the server 001, and the interactive robot 002 can obtain data (such as the target age attribute and target gender attribute of the target user group, the target promotional data corresponding to the target user, etc.) from the server 001, and can also send the obtained data (such as radio NFC information, etc.) to the server 001 for storage.

[0076] In actual application, the interactive robot obtains the first data of the target user, determines the group category of the target user based on the first data, and when the group category of the target user is the target group category, generates and displays the corresponding holographic projection image based on the target promotion data corresponding to the target group category obtained from the server.

[0077] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0078] Figure 2 This is a flowchart of an interactive method provided in an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0079] S201. Acquire first data of a target user; wherein the first data includes voice data and / or image data.

[0080] The execution subject of the embodiment of the present application may be an interactive robot, or an interactive device arranged in the interactive robot. The interactive device may be implemented by software, or by a combination of software and hardware.

[0081] Optionally, the first data may include data related to the target user collected by a sensor device provided in the interactive robot. The above-mentioned sensor device includes but is not limited to audio sensor devices (such as microphones, etc.), cameras, keyboards, etc. The first data includes but is not limited to text data, voice data, and image data.

[0082] The interactive robot is provided with a cache space, and the first data may be stored in the cache space. Alternatively, the first data may be data obtained by the interactive robot from a server or other terminal device.

[0083] S202: Determine a group category of the target user according to the first data, where the group category is a target group category or a non-target group category.

[0084] Optionally, according to the first data of the target user, it is identified and determined that the group category of the target user is a target group category or a non-target group category.

[0085] Optionally, the group category of the user may be determined according to the age attribute and / or gender attribute of the target user.

[0086] The target group category refers to a group category divided based on the target age attribute and / or target gender attribute of the user.

[0087] Optionally, the age attribute of the target group category may be one or more age values, or one or more age ranges.

[0088] For example, the age attributes of the target group category include 25 years old, 25 years old, or the age attributes of the target group category include [20 years old-25 years old], [30 years old-40 years old].

[0089] S203: If the group category of the target user is the target group category, move to the target position of the target user, generate and display a corresponding holographic projection image according to the target promotion data corresponding to the target group category.

[0090] Among them, the interactive robot includes a display screen, and the display screen can be used to display a holographic projection image.

[0091] Optionally, if the group category of the target user is a target group category, target promotion data corresponding to the target group category is obtained.

[0092] Optionally, the target position of the target user is obtained, and the target position of the target user is moved to, and a corresponding holographic projection image is generated according to the target promotional data and displayed on a display screen.

[0093] The target promotion data may be data stored in the cache space of the interactive robot, or the target promotion data may be data obtained by the interactive robot from a server or other terminal devices.

[0094] The present application obtains the first data of the target user through an interactive robot, determines the group category of the target user based on the first data of the target user, moves to the location of the target user when the target user is in the target group category, and displays the target promotional data corresponding to the target group category through holographic projection, which can reduce the consumption of resources such as manpower, material resources and time, improve the matching degree between the target promotional data and the group users, and thus improve the efficiency of the promotion.

[0095] The target user group category can be determined in the following ways:

[0096] S2021. Identify the first data to determine characteristic attributes of the target user, where the characteristic attributes include age attributes and / or gender attributes, and the identification method includes voice recognition method and / or image recognition method.

[0097] Optionally, the first data of the target user is recognized by voice recognition and / or image recognition to determine characteristic attributes of the target user.

[0098] The characteristic attributes of the target user include age attribute and / or gender attribute.

[0099] The age attribute of the target user may be a specific age value or an age range.

[0100] S2022: If the age attribute of the target user matches the target age attribute, and the gender attribute of the target user matches the target gender attribute, determining that the group category of the target user is the target group category.

[0101] Optionally, if the age attribute of the target user matches the target age attribute of the target group category, and the gender attribute of the target user matches the target gender attribute of the target group category, then the group category of the target user is determined to be the target group category.

[0102] The target group category may include one or more. When the characteristic attributes of the target user meet the target characteristic attributes (such as the target age attribute and the target gender attribute) in one of the target group categories, the group category of the target user is determined to be the target group category.

[0103] If the target user has only one characteristic attribute, then when the characteristic attribute matches the corresponding target characteristic attribute in the target group category, the group category of the target user can be determined as the target group category.

[0104] For example, if the characteristic attributes of the target user only include the age attribute, when the age attribute of the target user matches the target age attribute in the target group category, the group category of the target user is determined to be the target group category.

[0105] Optionally, the gender attribute includes 2 genders, and the age attribute includes 7 age ranges.

[0106] For example, gender attributes include male and female. Age attributes include [0, 18), [18, 30), [30, 40), [40, 50), [50, 60), [60, 70), [70, 80), [80, 100), etc.

[0107] Optionally, the facial expression attributes (or emotional attributes) of the target user may also be identified through a deep learning algorithm. The facial expression attributes include seven types of expressions.

[0108] For example, facial expression attributes include but are not limited to happiness, anger, worry, thought, sadness, fear, and surprise.

[0109] The embodiment of the present application communicates with users by identifying the attributes of individuals in the crowd, such as expression, action, language, age, etc., to determine the group category of users. The appearance of the interactive robot can be changed through 3D holographic projection, and the target promotional data can be displayed through multiple images, making the target promotional data more intuitive and attractive. The efficiency of promotion is improved by using radio NFC tags.

[0110] Figure 3 This is a flowchart of another interactive method provided in the embodiment of the present application. Figure 3 , step S2022 of the method may include the following steps:

[0111] S20221. If the first data includes the image data, perform face recognition on the image data to obtain a face recognition result.

[0112] Optionally, if the first data of the target user includes image data, face recognition is performed on the image data to obtain a face recognition result in the first data.

[0113] Optionally, face recognition can be performed on the image data using a pre-trained face recognition model.

[0114] S2022. Input the face recognition result into a first neural network model to obtain the age attribute and / or gender attribute of the target user.

[0115] The first neural network model refers to a neural network model that is based on a data set and pre-trained and can be used to identify age attributes and / or gender attributes corresponding to input data (such as face recognition results). For example, the first neural network model is a convolutional neural network (CNN).

[0116] The data set includes but is not limited to the IMDB-WIKI data set.

[0117] Optionally, the output result of the pre-trained first neural network model can be verified and evaluated by the mean absolute error (MAE).

[0118] Among them, the evaluation formula of mean absolute error (MAE) is as follows:

[0119] ;

[0120] Where X represents the input training data, Represents the model prediction data, represents the actual data corresponding to the pre-input data, and M represents the number of input training data.

[0121] Optionally, model training and evaluation can be achieved through open source frameworks such as Keras and OpenCV frameworks.

[0122] Figure 4 A schematic diagram of a loss function curve of a first neural network model is provided.

[0123] Figure 4 The loss function curve in is: the loss function curve of the first neural network model after 100 iterations of pre-training. The solid line represents the average age loss. The dotted line represents the average absolute error value of age, the solid arrow represents the average loss value, and the dotted arrow represents the loss value.

[0124] according to Figure 4 It can be seen that the mean absolute error (MAE) of the first neural network model after pre-training is lower than the average loss value.

[0125] Optionally, the face recognition result in the first data is input into the first neural network model to obtain the age attribute and / or gender attribute of the target user.

[0126] Optionally, the remote terminal control system (GUI) of the interactive robot can be set up through a computer programming language, for example, Python language, Tkinter package.

[0127] Among them, the remote terminal control system of the interactive robot receives the age attribute and / or gender attribute sent by the first neural network module, compares it with the pre-set data, and returns the corresponding recognition result based on the comparison result. If the comparison result is consistent, the returned facial image data is called.

[0128] Figure 5 This is a flowchart of another interactive method provided in the embodiment of the present application. Figure 5 , step S2022 of the method may include the following steps:

[0129] S20223. If the first data includes the voice data, identify the target text information corresponding to the voice data.

[0130] Optionally, if the first data of the target user includes voice data, target text information corresponding to the voice data is identified to facilitate understanding of the intention of the target user's voice data.

[0131] S20224. Generate corresponding reply information according to the target text information and display it.

[0132] Optionally, corresponding reply information is generated and displayed according to the target text information to interact with the target user.

[0133] The reply information may include but is not limited to text information, voice information, and image information.

[0134] Optionally, the reply information may be displayed by holographic projection.

[0135] S20225. Determine the age attribute and / or gender attribute of the target user according to the voice data and the reply information.

[0136] Optionally, the age attribute and / or gender attribute of the target user is determined according to the voice data of the target user and the reply information corresponding to the voice data.

[0137] The target text information corresponding to the voice data can be identified in the following ways:

[0138] The speech data is segmented to obtain speech data of multiple time windows, and feature information of the speech data of each time window is extracted respectively; the feature information of the speech data of each time window is input into a second neural network model to obtain speech recognition information; the feature information of the speech data of each time window is mapped to a text sequence model to obtain preprocessed text information; the speech recognition information and the preprocessed text information are matched according to the speech recognition engine to obtain target text information.

[0139] Optionally, the speech data is divided into speech data of a plurality of time windows by using a speech signal framing technology.

[0140] Optionally, feature information of the speech data in each time window may be extracted by Mel Frequency Cepstrum Coefficient (MFCC).

[0141] The second neural network model refers to a pre-trained Convolutional Neural Networks (CNN) model for performing end-to-end speech recognition.

[0142] Optionally, the feature information of the speech data in each time window may be mapped to a text sequence model through a deep learning model (Connectionist Temporal Classification, CTC model).

[0143] The speech recognition engine may include but is not limited to a deep speech recognition engine (DeepSpeech).

[0144] The reply information corresponding to the target text information can be generated and displayed in the following ways:

[0145] Performing intent recognition on the target text information to obtain the target intent; generating corresponding reply information according to the target intent; generating corresponding target voice data according to the reply information, and inputting the target voice data into the acoustic model to obtain voice playback data; generating a first holographic projection image according to the voice playback data and preset projection parameters, and displaying the first holographic projection image on a display screen.

[0146] Optionally, when the target text information corresponding to the voice data of the target user is obtained, the target text information may be subjected to intent recognition to determine the target intent and demand of the target user.

[0147] The acoustic model may include but is not limited to a deep neural network (Tacotron) model.

[0148] If the reply information is text data, the reply information is converted into corresponding target voice data through Text To Speech (TTS) technology.

[0149] The preset projection parameters refer to the projection image parameters that are pre-set corresponding to the target group category and target promotional data.

[0150] The preset projection parameters may include parameters for setting the image of a person in the projected image. For example, the preset projection parameters include but are not limited to the gender, age, voiceprint information, etc. of the person in the projected image.

[0151] Optionally, the display screen can be used to display image data of a three-dimensional (3D) suspended holographic projection. The display screen includes chips, magnets, light-emitting diodes (LEDs) and propeller blade projectors. When the above-mentioned blade projectors rotate, a visual effect of a three-dimensional object appearing in the air can be presented. The interactive robot can transmit data to a cloud platform such as a server via a wireless network for serial use.

[0152] Optionally, when receiving the first holographic projection image, the holographic projection device arranged on the interactive robot starts the propeller blade projector and controls the operation of the internal diode according to preset projection parameters, thereby presenting the visual effect of a suspended 3D image.

[0153] Optionally, when moving to the target position of the target user, the holographic projection device on the interactive robot displays a first holographic projection image and transmits it through the audio device mounted on the movable base, such as replying to the information to generate corresponding voice playback data, target promotion data related to the target promotion object, or other communication voice data.

[0154] Optionally, the display screen of the interactive robot may include multiple controls, and the user may control the interactive robot by operating the controls. Exemplarily, functions such as pack_forget() and pack() in the interactive robot may be used to flexibly control the display and hiding of controls.

[0155] Optionally, the interactive robot includes a mainboard, which has the functions of data storage and processing and can control the movement of the interactive robot. Before the interactive robot is used, the mainboard can be connected to a network cable and a secure shell protocol (SecureShell, SSH for short) can be configured in advance, and data transmission software and visualization software can be set.

[0156] Optionally, in order to realize the control of the hardware part by the mainboard, an adapter board can be set up, and the functions of the adapter board are as follows: the adapter board realizes the expansion of functions by connecting with the expansion module of the mainboard, for example: connecting the tire motor of the interactive robot to the adapter board, connecting the speakers, speakers and other equipment to the adapter board, connecting the camera to the adapter board, and connecting the power supply to the adapter board.

[0157] Optionally, the holographic projection device can also be integrated with the core motherboard of the interactive robot, using a non-fixed installation, which is convenient for disassembly at any time. The preset projection parameters can be flexibly set according to user needs, and the operation is simple.

[0158] Holographic projection images are transparent and three-dimensional, creating a dynamic visual effect. Holographic projection images can improve the publicity efficiency of target publicity data.

[0159] The holographic projection image corresponding to the target promotional data may be generated and displayed in the following ways, including:

[0160] If the group category of the target user is the target group category, the target position of the target user is obtained; the target path is planned according to the target position, and the target position is moved according to the target path; if the target position is moved to, a second holographic projection image is generated according to the target promotional data, and the second holographic projection image is displayed on the display screen.

[0161] Optionally, the interactive robot includes but is not limited to a camera, an ultrasonic sensor, and an infrared obstacle avoidance sensor. In the process of moving to the target position, the existence of obstacles is determined based on the image data collected by the camera, the sensor data collected by the infrared obstacle avoidance sensor, and the sensor data collected by the ultrasonic sensor. Based on the distance between the interactive robot and each obstacle, motion decisions and motor control operations are performed according to the distance to avoid collision with the obstacle.

[0162] Optionally, the interactive robot can perform behavior recognition on the user based on the image data collected by the camera. When the target behavior is recognized, it is determined that the user has violated the rules and a prompt message can be generated and displayed.

[0163] The target behavior can be flexibly set according to the actual situation, and the prompt information includes but is not limited to text information, voice information, and image information.

[0164] Optionally, the movable range of the interactive robot may be preset, for example, the length, width or diameter of the movable range, etc. The interactive robot may display promotional data to the target user group within the above movable range.

[0165] The second holographic projection image can be generated and displayed in the following manner:

[0166] If it moves to the target position, the target promotion data corresponding to the target group category is obtained; the corresponding second holographic projection image is generated according to the target promotion data and the preset projection parameters, and the second holographic projection image is displayed on the display screen.

[0167] The second holographic projection image may be a single image data, or a video or animation composed of image data of multiple consecutive frames.

[0168] Optionally, the user can start the robot by dragging, pressing an external start button of the interactive robot, sending a start instruction to the interactive robot through a user terminal, etc.

[0169] Among them, the first data may include user identity authentication information, and the interactive robot includes at least one pre-stored identity authentication information.

[0170] The at least one pre-stored identity authentication information may be data stored in a cache space of the interactive robot, or the at least one pre-stored identity authentication information may be data obtained by the interactive robot from a server.

[0171] Figure 6 A flowchart of another interactive method provided in an embodiment of the present application. Figure 6 , the method may include the following steps:

[0172] S301. Read user identity authentication information in the first data.

[0173] The user identity authentication information in the first data may be image data collected by a camera, text data collected by a keyboard, voice data collected by a microphone, etc.

[0174] Figure 7 An application schematic diagram of an interaction method provided in an embodiment of the present application.

[0175] See also Figure 7 The interface used by the interactive robot to obtain user identity authentication information may include the user's registration account, registration password, login, registration, exit and other controls. The user can enter the registration account and registration password and operate the login, registration, exit and other controls to perform corresponding login, registration, exit and other functions.

[0176] S302: Match the user identity authentication information with each pre-stored identity authentication information in the at least one pre-stored identity authentication information.

[0177] The at least one pre-stored identity authentication information may be one or more of text data, voice data, image data, and the like.

[0178] Optionally, the user identity authentication information is matched with each pre-stored identity authentication information in at least one pre-stored identity authentication information according to a data type of the user identity authentication information.

[0179] S303: If there is pre-stored identity authentication information matching the user identity authentication information in the at least one pre-stored identity authentication information, determine that the target user is a registered user.

[0180] Among them, a registered user refers to a user who has been authenticated in advance based on identity authentication information and has passed the authentication.

[0181] The pre-stored identity authentication information refers to the identity authentication information of a registered user who has passed identity authentication in advance.

[0182] S304, generating a login success message based on the user identity authentication information and displaying it on a display screen;

[0183] S305: Determine the age attribute and / or gender attribute of the target user according to the user identity authentication information.

[0184] The successful login information may include but is not limited to one or more of text data, voice data, and image data.

[0185] Optionally, the display method of the successful login information may include a holographic projection method.

[0186] The user identity authentication information may include the user's gender attributes, age attributes, etc.

[0187] Optionally, when the target user successfully logs in, if a control instruction sent by the target user is received, a corresponding operation is performed according to the control instruction. The control instruction may be an instruction received from a terminal device of the registered user, or an instruction collected by a sensor device.

[0188] For example, when the target user logs in successfully, if a voice control instruction "modify projection parameters" is received from the target user, the preset projection parameters can be modified according to the voice control instruction.

[0189] For example, when the target user logs in successfully, if the target user receives a control instruction "modify projection parameters" by touching the touch display screen of the interactive robot, the preset projection parameters can be modified according to the above voice control instruction.

[0190] Figure 8 An application schematic diagram of an interaction method provided in an embodiment of the present application.

[0191] See also Figure 8 When the interactive robot is connected to the registered user's terminal device for communication, the registered user's terminal device can display data such as the interactive robot's device number, connection status, remaining power, and GPS signal in the display interface, and at the same time display controls such as "target group type and preset projection parameter setting", "target parameter setting", "activity range setting", "other settings", "start robot" and "turn off robot", so that the registered user can control the interactive robot with communication connection by operating the above controls.

[0192] Fig. 9 A flowchart of another interactive method provided in an embodiment of the present application. Fig. 9 , the method may include the following steps:

[0193] S401, if radio information is received, reading the data to be promoted in the radio information;

[0194] S402, writing the data to be promoted into a radio tag, and displaying the radio tag on a display screen.

[0195] The radio information refers to Near Field Communication (NFC) information.

[0196] The data to be promoted refers to the promotional data carried in the NFC signal.

[0197] For example, the interactive robot includes a graphical user interface filedialog component. The corresponding radio information may be text data, voice data, image data, etc. selected by the user through the filedialog component.

[0198] Optionally, near field communication information NFC technology is a short-range high-frequency radio technology that allows contactless point-to-point data transmission and exchange between electronic devices (within 10 cm). A gateway system that is compatible with contactless smart cards, including three working modes: point-to-point communication mode, reader / writer mode and NFC card emulation mode.

[0199] Among them, the NFC card simulation mode can simulate an NFC-enabled device (such as an infinity sensor) into a tag or contactless card. The RF domain of the NFC card and the contactless card reader is powered, so that the NFC-enabled device can provide information even when it is not powered on.

[0200] Fig.10 An application diagram of an interactive method provided in an embodiment of the present application.

[0201] See also Fig.10 When the interactive robot displays a radio tag on the display screen, if the user brings the user terminal device close to the card reader, the display interface of the user terminal device can display a prompt window such as "Do you want to open the NPC tag?". The user can read the data to be promoted carried by the radio tag by operating the above prompt window.

[0202] Optionally, the interactive robot can also collect data through a device with NFC function, and perform corresponding data processing operations through the processor of the interactive robot.

[0203] The embodiment of the present application uses an interactive robot to display specific target promotional data to the target user group, which can reduce the manpower cost, material cost, and time cost required for promotion; through computer vision technology and sound recognition technology, the attributes of the user group (such as age, expression, language, gender, etc.) are analyzed in multiple dimensions to determine the intention of the target user, locate the target user group, and improve the promotion efficiency; through the holographic projection method, the display form of the promotion object can be flexibly switched to reduce the risk of movement of the promotion object and improve the promotion efficiency. The method of converting radio information NFC tags simplifies user operations, makes the promotion method simpler and faster, and further improves the promotion efficiency.

[0204] Fig.11 This is a schematic diagram of the structure of an interactive device provided in an embodiment of the present application. Fig.11 The interactive device 10 includes: a data acquisition module 11, a group determination module 12, and a display module 13, wherein:

[0205] The data acquisition module 11 is used to acquire first data of the target user; wherein the first data includes voice data and / or image data;

[0206] The group determination module 12 is used to determine the group category of the target user according to the first data, and the group category is a target group category or a non-target group category;

[0207] The display module 13 is used to, if the group category of the target user is the target group category, move to the target position of the target user, generate and display a corresponding holographic projection image according to the target promotion data corresponding to the target group category.

[0208] In a possible implementation, the group determination module 12 is specifically configured to:

[0209] Identify the first data to determine characteristic attributes of the target user, where the characteristic attributes include age attributes and / or gender attributes, and the identification method includes voice recognition method and / or image recognition method;

[0210] If the age attribute of the target user matches the target age attribute, and the gender attribute of the target user matches the target gender attribute, then the group category of the target user is determined to be the target group category.

[0211] In a possible implementation, the group determination module 12 is specifically configured to:

[0212] If the first data includes the image data, performing face recognition on the image data to obtain a face recognition result;

[0213] The face recognition result is input into a first neural network model to obtain the age attribute and / or gender attribute of the target user.

[0214] In a possible implementation, the group determination module 12 is specifically configured to:

[0215] If the first data includes the voice data, identifying target text information corresponding to the voice data;

[0216] Generate corresponding reply information according to the target text information and display it;

[0217] The age attribute and / or gender attribute of the target user is determined according to the voice data and the reply information.

[0218] In a possible implementation, the group determination module 12 is specifically configured to:

[0219] Segmenting the speech data to obtain speech data of multiple time windows, and extracting feature information of the speech data of each time window respectively;

[0220] Inputting feature information of the speech data in each of the time windows into a second neural network model to obtain speech recognition information;

[0221] Mapping the feature information of the speech data in each of the time windows to a text sequence model to obtain preprocessed text information;

[0222] The speech recognition information and the pre-processed text information are matched according to a speech recognition engine to obtain target text information.

[0223] In a possible implementation, the group determination module 12 is specifically configured to:

[0224] Performing intent recognition on the target text information to obtain the target intent;

[0225] Generate corresponding reply information according to the target intention;

[0226] Generate corresponding target voice data according to the reply information, and input the target voice data into the acoustic model to obtain voice playback data;

[0227] A first holographic projection image is generated according to the voice playback data and preset projection parameters, and the first holographic projection image is displayed on a display screen.

[0228] In a possible implementation manner, the display module is specifically used for:

[0229] If the group category of the target user is the target group category, obtaining the target location of the target user;

[0230] Planning a target path according to the target location, and moving toward the target location according to the target path;

[0231] If the target position is moved to, a second holographic projection image is generated according to the target promotional data, and the second holographic projection image is displayed on the display screen.

[0232] In a possible implementation manner, the display module is specifically used for:

[0233] If the target location is moved, target promotion data corresponding to the target group category is obtained;

[0234] A corresponding second holographic projection image is generated according to the target promotion data and preset projection parameters, and the second holographic projection image is displayed on the display screen.

[0235] In a possible implementation, the first data includes user identity authentication information, the interactive robot includes at least one pre-stored identity authentication information, and the device further includes an authentication information reading module, a matching module, an authentication module, an information generating module, and an attribute determining module, wherein:

[0236] The authentication information reading module is used to read the user identity authentication information in the first data;

[0237] The matching module is used to match the user identity authentication information with each pre-stored identity authentication information in the at least one pre-stored identity authentication information;

[0238] The authentication module is configured to determine that the target user is a registered user if there is pre-stored identity authentication information matching the user identity authentication information in the at least one pre-stored identity authentication information;

[0239] The information generation module is used to generate a login success message based on the user identity authentication information and display it on the display screen;

[0240] The attribute determination module is used to determine the age attribute and / or gender attribute of the target user according to the user identity authentication information.

[0241] In a possible implementation, the device further includes a data reading module tag and a generating module, wherein:

[0242] The data reading module is used for reading the data to be advertised in the radio information if the radio information is received;

[0243] The label generation module is used to write the data to be promoted into a radio label and display the radio label on a display screen.

[0244] The interactive device provided in the embodiment of the present application can execute the method shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0245] Fig.12 This is a schematic diagram of the structure of the interactive robot provided in the embodiment of the present application. Fig.12 As shown, the interactive robot 20 may include: a transceiver 21, a processor 22, a memory 23, a display screen 24, a movable base 25, and a radio sensor 26; wherein the display screen 24, the movable base 25, and the radio sensor 26 are all communicatively connected to the processor 22.

[0246] The processor 22 executes the computer execution instructions stored in the memory, so that the processor 22 executes the scheme in the above embodiment. The processor 22 can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0247] The memory 23 is connected to the processor 22 via a system bus and completes communication between them. The memory 23 is used to store computer program instructions.

[0248] The transceiver 21 may be used to obtain first data of a target user, etc.

[0249] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0250] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the interaction method in the above embodiment.

[0251] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the interaction method in the above embodiment.

[0252] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the interaction method in the above embodiment can be implemented.

[0253] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0254] It should be further noted that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flow chart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0255] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0256] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0257] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc.

[0258] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0259] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0260] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0261] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An interactive method, characterized in that: Applied to an interactive robot, the method comprises: Acquire first data of a target user; wherein the first data includes voice data and / or image data; Determine, according to the first data, a group category of the target user, where the group category is a target group category or a non-target group category; If the group category of the target user is the target group category, the target position of the target user is moved to generate and display a corresponding holographic projection image according to the target promotion data corresponding to the target group category.

2. The method according to claim 1, characterized in that Determining the group category of the target user according to the first data includes: Identify the first data to determine characteristic attributes of the target user, where the characteristic attributes include age attributes and / or gender attributes, and the identification method includes voice recognition method and / or image recognition method; If the age attribute of the target user matches the target age attribute, and the gender attribute of the target user matches the target gender attribute, then the group category of the target user is determined to be the target group category.

3. The method according to claim 2, characterized in that The identifying the first data to determine the characteristic attributes of the target user includes: If the first data includes the image data, performing face recognition on the image data to obtain a face recognition result; The face recognition result is input into a first neural network model to obtain the age attribute and / or gender attribute of the target user.

4. The method according to claim 2, characterized in that: The identifying the first data to determine the characteristic attributes of the target user includes: If the first data includes the voice data, identifying target text information corresponding to the voice data; Generate corresponding reply information according to the target text information and display it; The age attribute and / or gender attribute of the target user is determined according to the voice data and the reply information.

5. The method according to claim 4, characterized in that If the first data includes the voice data, identifying target text information corresponding to the voice data includes: Segmenting the speech data to obtain speech data of multiple time windows, and extracting feature information of the speech data of each time window respectively; Inputting feature information of the speech data in each of the time windows into a second neural network model to obtain speech recognition information; Mapping the feature information of the speech data in each of the time windows to a text sequence model to obtain preprocessed text information; The speech recognition information and the pre-processed text information are matched according to a speech recognition engine to obtain target text information.

6. The method according to claim 4, characterized in that The step of generating and displaying corresponding reply information according to the target text information includes: Performing intent recognition on the target text information to obtain the target intent; Generate corresponding reply information according to the target intention; Generate corresponding target voice data according to the reply information, and input the target voice data into the acoustic model to obtain voice playback data; A first holographic projection image is generated according to the voice playback data and preset projection parameters, and the first holographic projection image is displayed on a display screen.

7. The method according to claim 1, characterized in that If the group category of the target user is the target group category, then move to the target position of the target user, generate and display a corresponding holographic projection image according to the target promotion data corresponding to the target group category, including: If the group category of the target user is the target group category, obtaining the target location of the target user; Planning a target path according to the target location, and moving toward the target location according to the target path; If the target position is moved to, a second holographic projection image is generated according to the target promotional data, and the second holographic projection image is displayed on the display screen.

8. The method according to claim 7, characterized in that If the target position is moved to, a second holographic projection image is generated according to the target promotion data, and the second holographic projection image is displayed on the display screen, including: If the target location is moved, target promotion data corresponding to the target group category is obtained; A corresponding second holographic projection image is generated according to the target promotion data and preset projection parameters, and the second holographic projection image is displayed on the display screen.

9. The method according to any one of claims 1 to 8, characterized in that The first data includes user identity authentication information, the interactive robot includes at least one pre-stored identity authentication information, and the method further includes: Reading user identity authentication information in the first data; matching the user identity authentication information with each of the at least one pre-stored identity authentication information; If there is pre-stored identity authentication information matching the user identity authentication information in the at least one pre-stored identity authentication information, determining that the target user is a registered user; Generate a login success message based on the user identity authentication information and display it on the display screen; The age attribute and / or gender attribute of the target user is determined according to the user identity authentication information.

10. The method according to any one of claims 1 to 8, characterized in that The method further comprises: If a radio message is received, then reading the data to be advertised in the radio message; The data to be promoted is written into a radio tag, and the radio tag is displayed on a display screen.

11. An interactive device, characterized in that: Applied to an interactive robot, the device comprises: a data acquisition module, a group determination module and a display module, wherein: The data acquisition module is used to acquire first data of a target user; wherein the first data includes voice data and / or image data; The group determination module is used to determine the group category of the target user according to the first data, where the group category is a target group category or a non-target group category; The display module is used to, if the group category of the target user is the target group category, move to the target position of the target user, generate and display a corresponding holographic projection image according to the target promotion data corresponding to the target group category.

12. An interactive robot, characterized in that: include: A display screen, a movable base, a wireless sensor, a processor, and a memory in communication with the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 10 when being executed by a processor.

Images