Interaction method and device

Abstract

The embodiment of the invention provides an interaction method and related equipment / products, and relates to the technical field of artificial intelligence. The interaction method comprises the following steps: determining whether a moving object exists in a target environment at the current moment; under the condition that the moving object exists, an information event at the current moment is obtained, and an interaction event of the moving object for the target environment at the current moment is determined; determining respective effective information amounts of the information event and the interaction event according to the moving object, and determining the information event or the interaction event as a target event based on the effective information amounts; and determining target interaction content based on the target event, and presenting the target interaction content to the moving object. According to the technical scheme, the event most critical to the moving object in a complex scene can be accurately recognized, and the interaction content matched with the requirement of the moving object is selected and presented according to the key event, so that the man-machine interaction effect is remarkably improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an interactive method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent robots are gradually being integrated into daily life, playing an important role in fields such as service, security, and healthcare.

[0003] However, traditional intelligent robots suffer from poor human-computer interaction due to their inability to accurately understand complex scenarios.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0005] This application provides an interactive method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems mentioned above.

[0006] One aspect of this application provides an interaction method, the method comprising: Determine whether there are moving objects within the target environment at the current moment; In the case of a moving object, acquire the information events at the current moment and determine the interaction events of the moving object with respect to the target environment at the current moment; Based on the moving object, determine the effective information content of each of the information event and the interaction event, and determine the information event or the interaction event as the target event based on the effective information content; and The target interactive content is determined based on the target event, and the target interactive content is presented to the moving object.

[0007] Optionally, the robot includes a spherical LED display device, which includes a spherical shell and a plurality of LEDs densely distributed on the spherical shell; Presenting the target interactive content to the moving object includes: Based on the target interactive content, a first control command is generated for the plurality of LEDs to control the on / off state of the plurality of LEDs; The first control command controls the switching state of the plurality of LEDs to form a corresponding LED dot matrix pattern. The LED dot matrix pattern includes Emoji patterns.

[0008] Optionally, presenting the target interactive content to the moving object further includes: Determine the location information of the moving object within the target environment; Based on the location information of the moving object within the target environment and the target interaction content, a second control command is generated for the plurality of LEDs to control the on / off state of the plurality of LEDs; The second control command controls the switching states of the plurality of LEDs to form an LED dot matrix pattern facing the moving object.

[0009] Optionally, the target interactive content includes target animation, target sound effects, and / or target text; The target animation is determined from a plurality of pre-configured animations based on the target event; the target sound effect is generated based on the interaction events of the moving object with respect to the target environment at the current moment and a pre-trained speech model; and the target text is generated based on the information events at the current moment and a pre-trained text model.

[0010] Optionally, determining the interaction events of the moving object with respect to the target environment at the current moment includes: Acquire sound data at the current moment, the sound data including multiple sound segments that are consecutive in time; The multiple sound segments are sequentially input into a pre-trained deep learning model according to the time sequence, so as to determine the interaction events of the moving object with respect to the target environment at the current moment through the deep learning model.

[0011] Optionally, the deep learning model determines the interaction events of the moving object with respect to the target environment at the current moment through the following operations: Audio analysis is performed on the multiple sound segments to determine the number of moving objects, the speech content of each moving object, the emotional state of each moving object, and / or the background sound of the target environment; Based on the number of moving objects, the voice content of each moving object, the emotional state of each moving object, and / or the background noise of the target environment, determine the interaction events of the moving objects with respect to the target environment at the current moment.

[0012] Optionally, retrieve information events at the current moment, including: Obtain pre-entered information, which includes the profile information and preference information of the mobile object; Based on the profile information and the preference information, the information event at the current moment is obtained; wherein the information event is matched with the moving object.

[0013] Optionally, determining whether there are moving objects within the target environment at the current moment includes: The system uses pre-configured infrared pyroelectric sensors, microwave radar sensors, and / or linear CCD sensors to determine whether there are moving objects in the target environment at the current moment. Specifically, the infrared pyroelectric sensor determines whether there is a moving object in the target environment at the current moment based on the infrared radiation of the target environment; the microwave radar sensor determines whether there is a moving object in the target environment at the current moment based on the microwave frequency of the target environment; and the linear CCD sensor determines whether there is a moving object in the target environment at the current moment based on the light intensity of the target environment.

[0014] Another aspect of this application provides an interactive device for a robot, the device comprising: The first determining module is used to determine whether there are moving objects in the target environment at the current moment; The acquisition module is used to acquire information events at the current moment and determine the interaction events of the mobile object with respect to the target environment at the current moment, in the case of a moving object; The second determining module is configured to determine the effective information content of the information event and the interaction event respectively based on the moving object, and determine the information event or the interaction event as a target event based on the effective information content; and The presentation module is used to determine the target interactive content based on the target event and present the target interactive content to the moving object.

[0015] Another aspect of this application provides a computer device, including: At least one processor; and A memory that is communicatively connected to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0016] Another aspect of this application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method described above.

[0017] Another aspect of this application provides a computer program product including a computer program that, when executed by a processor, implements the method described above.

[0018] The embodiments of this application employing the above technical solution may include the following advantages: When a moving object exists within the target environment at the current moment, information events and interaction events between the moving object and the target environment at the current moment are dynamically acquired. Based on the effective information content of each information event and interaction event, the information event or interaction event is determined as the target event. Then, based on the target event (e.g., the moving object is dancing), matching target interactive content (e.g., a happy animated emoticon or a dancing animation) is determined and displayed. The embodiments of this application can accurately identify the most critical events for the moving object in complex scenes, and select and present interactive content matching the needs of the moving object based on the critical events, thereby significantly improving the human-computer interaction effect. Attached Figure Description

[0019] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0020] Figure 1 A flowchart illustrating the interaction method according to Embodiment 1 of this application is shown schematically; Figure 2 Schematic illustration Figure 1 Flowchart of the sub-steps in step S102; Figure 3 Schematic illustration Figure 1 Flowchart of the sub-steps in step S102; Figure 4 The flowchart illustrates the steps a deep learning model takes to determine the interaction events of a moving object with respect to the target environment at the current moment. Figure 5 Schematic illustration Figure 1 Flowchart of the sub-steps in step S106; Figure 6 A flowchart illustrating an exemplary application implementation is shown. Figure 7 A flowchart illustrating an exemplary application implementation is shown. Figure 8 A schematic diagram illustrating a robot according to an embodiment of this application is shown. Figure 9 A block diagram of an interactive device according to Embodiment 2 of this application is schematically shown; and Figure 10 A schematic diagram of the hardware architecture of a computer device according to Embodiment 3 of this application is shown. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0022] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0023] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0024] First, a definition of the terminology used in this application is provided: Infrared pyroelectric sensor: A sensor based on the pyroelectric effect that generates an electrical signal by detecting temperature changes caused by infrared radiation emitted by an object.

[0025] Microwave radar sensor: an active sensor that detects targets based on electromagnetic waves in the microwave frequency band. It detects the distance, speed, angle and motion state of an object by emitting microwave signals and receiving the echo reflected from the target.

[0026] Linear CCD sensor: A linear array photoelectric sensor based on the charge-coupled device (CCD) principle, used to detect moving objects that are not part of an image. It converts light intensity distribution into a time-series electrical signal output through photoelectric conversion and charge transfer at the pixel level.

[0027] Deep learning models: a machine learning method based on artificial neural networks that automatically learns high-level feature representations of data through multi-layer nonlinear transformations. They achieve end-to-end feature extraction and pattern recognition through deep network structures.

[0028] Secondly, to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of this application, the relevant technologies are described below: (1) The robot uses image recognition technology, which leads to the frequent collection of users' daily life image data, increasing the risk of user privacy leakage, and thus causing users to develop a resistance mentality and be unwilling to use it for a long time. (2) It can only recognize simple emotional states of users (such as happy and sad), and cannot effectively judge the specific scene in the user's environment, resulting in limited interaction content between users and robots, a single interaction method, and difficulty in achieving true emotional companionship and deep interactive experience. (3) The accuracy of environmental event recognition is low. To this end, the embodiments of this application provide an interactive technical solution. In this technical solution, (1) user recognition is achieved through non-image multi-sensor fusion technology, which reduces the risk of user privacy leakage and ensures efficient and accurate preliminary detection of environmental events. (2) A deep learning model combined with slice-type continuous sound acquisition technology is used to deeply analyze detailed event information including voice content, number of people, user interaction methods, and background media playback status, thereby achieving a more accurate and deeper understanding and feedback of the environmental state. (3) A spherical screen is used as a display medium to provide vivid and diverse animated interactive feedback, significantly enhancing the emotional connection between the user and the robot, and enabling the robot to have a better family companionship effect. (4) A user feedback closed-loop algorithm is introduced to optimize the algorithm parameters and analysis model based on the data actively fed back by the user, so as to achieve continuous adaptive optimization of the algorithm and improve the robot's high accuracy and personalized effect, realizing the closed-loop automatic evolution of artificial intelligence algorithms. See the following text for details.

[0029] The technical solutions of this application are described below using a robot as the executing entity through multiple embodiments. It should be understood that these embodiments can be implemented in many different forms and should not be construed as being limited to the embodiments described herein.

[0030] Example 1 Figure 1 A flowchart illustrating an interactive method according to Embodiment 1 of this application is shown schematically.

[0031] like Figure 1 As shown, the interaction method may include steps S100 to S106, wherein: Step S100: Determine whether there are moving objects in the target environment at the current moment.

[0032] Step S102: In the case of a moving object, acquire the information event at the current moment and determine the interaction event of the moving object with respect to the target environment at the current moment.

[0033] Step S104: Based on the moving object, determine the effective information content of the information event and the interaction event respectively, and determine the information event or the interaction event as the target event based on the effective information content.

[0034] Step S106: Determine the target interactive content based on the target event, and present the target interactive content to the moving object.

[0035] The interactive method provided in this embodiment dynamically acquires information events and interaction events between the moving object and the target environment at the current moment, when there is a moving object in the target environment at the current moment. Based on the effective information content of each information event and interaction event, the information event or interaction event is determined as the target event. Then, according to the target event (e.g., the moving object is dancing), matching target interactive content (e.g., a happy animated emoticon or a dancing animation) is determined and displayed. This embodiment can accurately identify the most critical events for the moving object in complex scenes, and select and present interactive content that matches the needs of the moving object based on the critical events, thereby significantly improving the human-computer interaction effect.

[0036] The following combination Figure 1 The steps in steps S100 to S106 and other optional steps are described in detail.

[0037] Step S100 This determines whether there are moving objects in the target environment at the current moment.

[0038] The target environment can be a living room, bedroom, warehouse, shopping mall, etc. Moving objects can include users (e.g., customers, staff), smart devices (e.g., robot vacuum cleaners), animals, etc. It should be noted that the moving objects referred to in this application embodiment mainly refer to users. In practical applications, robots can use built-in image sensors or non-image sensors to detect whether the target environment changes in real time, thereby determining whether there are moving objects within the target environment. For example, an image sensor can continuously capture multiple images of the target environment and perform image comparison. If the position of the main subject in the image changes, it can be determined that there is a moving object within the target environment, such as a person moving in the living room. However, this detection method is prone to user privacy leakage. Therefore, this application embodiment creatively equips the robot with a non-image sensor to replace the image sensor. The non-image sensor can accurately identify whether there are moving objects within the target environment in real time without leaking user privacy (by capturing user images in real time). An exemplary solution is provided below.

[0039] In an optional embodiment, step S100 may include: determining whether there is a moving object in the target environment at the current moment using a pre-configured infrared pyroelectric sensor, microwave radar sensor, and / or linear CCD sensor. Specifically, the infrared pyroelectric sensor determines whether there is a moving object in the target environment at the current moment based on the infrared radiation of the target environment. The microwave radar sensor determines whether there is a moving object in the target environment at the current moment based on the microwave frequency of the target environment. The linear CCD sensor determines whether there is a moving object in the target environment at the current moment based on the light intensity of the target environment.

[0040] For example, a robot can determine whether there is a moving object in the target environment at the current moment through a pre-configured non-image motion detection module and a microprocessor. The non-image motion detection module includes an infrared pyroelectric sensor, a microwave radar sensor, and / or a linear CCD sensor. For instance, the robot can be configured with an infrared pyroelectric sensor, a microwave radar sensor, and a linear CCD sensor simultaneously, with different sensors distributed at different locations and forming a certain angle, thereby improving the accuracy of identifying moving objects. The infrared pyroelectric sensor can detect moving objects by monitoring changes in infrared radiation in the 8-14μm band at a sampling rate of 60Hz. The microwave radar sensor can use various modes, such as a 24GHz FMCW modulation mode, to detect moving objects through microwave frequencies. The linear CCD sensor can detect the light intensity of the target environment at a frequency of 1kHz, thereby identifying moving objects. For example, when a user returns home and opens the door to enter the living room, the infrared pyroelectric sensor captures the heat radiation emitted by the human body, the microwave radar sensor detects changes in displacement within the space, and the linear CCD sensor simultaneously monitors real-time changes in the object's outline. This multimodal sensor data can be transmitted to a microprocessor in real time to determine whether there are moving objects (users) in the target environment at the current moment.

[0041] In this embodiment, an infrared pyroelectric sensor, a microwave radar sensor, and a linear CCD sensor are used together to detect and determine whether there are moving objects in the target environment. This improves the accuracy and robustness of identifying moving objects, allowing the subsequent sound analysis module to start and execute efficiently, and effectively avoids the risk of image privacy data leakage.

[0042] In some embodiments, the validity of a moving object can be determined by calculating the confidence levels of different sensors. Specifically, after real-time detection of the target environment using different sensors, moving objects within the target environment are initially identified. Then, the confidence levels of the infrared pyroelectric sensor, microwave radar sensor, and / or linear CCD sensor can be calculated based on historical detection records. Next, a comprehensive confidence level is calculated based on the confidence levels of each sensor and preset weights. The comprehensive confidence level is compared with a preset judgment threshold to determine whether the detected moving object is valid.

[0043] Step S102 In the case of a moving object, acquire information events at the current moment and determine the interaction events of the moving object with respect to the target environment at the current moment.

[0044] For example, when the microprocessor determines that there is a moving object in the target environment, the microprocessor will connect to the Internet via Wi-Fi to obtain information events at the current moment. Information events may include, but are not limited to: (1) Environmental data events, such as: meteorological events that can be obtained through meteorological APIs, including real-time rainfall, sudden temperature changes (such as a drop of 3°C within 10 minutes), ultraviolet intensity, etc.; indoor environmental events that can be obtained through temperature and humidity sensors and air quality sensors, such as monitoring indoor environmental anomalies; security events that can be received from smart home gateways, such as smoke alarms, water immersion alarms, etc. (2) External information push events, such as: public information events that can be obtained through open platform APIs, such as breaking news (such as earthquake early warnings), traffic control information, etc.; commercial service events that can be obtained synchronously through e-commerce platform interfaces, such as express delivery arrival notifications, supermarket promotion information, etc.; personal schedule events that can be obtained through calendar synchronization functions, such as user-preset holiday reminders, birthday reminders, etc. (3) Notification events, such as: abnormal status events of component functions configured in the robot, such as infrared pyroelectric sensors having no signal for 5 minutes, battery level below 10%, etc. An exemplary information event acquisition scheme is provided below.

[0045] In optional embodiments, such as Figure 2 As shown, step S102 may include: Step S200: Obtain pre-entered information, which includes the portrait information and preference information of the mobile object.

[0046] Step S202: Based on the portrait information and the preference information, obtain the information event at the current moment; wherein the information event is matched with the moving object.

[0047] For example, pre-entered information can be information pre-entered into the robot by the user (e.g., the mobile object). This could include the user's voice, geographical location, gender, age, work style, and job content. Pre-entered information can be used to determine the information events at the current moment, and can subsequently be used to help determine the most important events for the mobile object at that moment.

[0048] User profile information can be used to describe a user's individual characteristics, specifically including age, gender, occupation, income, education level, and geographical location. Preference information can be dynamic data reflecting a user's subjective interests. Preference information can be obtained through user-initiated settings (such as interest tag subscriptions, liking / collecting content), or by acquiring and analyzing historical user behavior data (such as news click-through rate, reading completion rate, search keywords, page scrolling depth, etc.). It should be noted that the various types of data obtained in this application embodiment are obtained with the user's explicit authorization and have undergone anonymization processing to ensure user privacy and security.

[0049] In this embodiment, personalized news events can be matched to mobile objects in real time by using user profiles and preference information, which can achieve accurate personalized recommendations and further improve the user experience.

[0050] Interaction events can include the activities of moving objects within the target environment, as well as the number of moving objects involved, their emotions, and voice. There are various methods for acquiring interaction events, such as voiceprint recognition, image recognition, and machine learning. An exemplary solution is provided below.

[0051] In optional embodiments, such as Figure 3 As shown, step S102 may include: Step S300: Obtain the sound data at the current moment, the sound data including multiple sound segments that are consecutive in time.

[0052] Step S302: The multiple sound segments are sequentially input into a pre-trained deep learning model according to the time sequence, so as to determine the interaction event of the moving object with respect to the target environment at the current moment through the deep learning model.

[0053] In practical applications, when the presence of moving objects in the target environment is confirmed, a sound analysis module can acquire and analyze sound segments of a preset duration (e.g., 3 to 10 seconds) in real time, while simultaneously acquiring the next sound segment, thus forming a continuous feedback mechanism for sound analysis. Pre-trained deep learning models (e.g., sound recognition models) can be used to analyze the sound segments to extract multi-dimensional sound features, thereby accurately inferring the current interactive events occurring in the target environment, such as whether the user is working, interacting with friends, engaging in solo activities, or watching media content.

[0054] In this embodiment, the multiple sound segments are sequentially input into a pre-trained deep learning model for analysis according to the time sequence. This can efficiently and accurately determine interactive events, thereby improving the accuracy and efficiency of sound analysis and expanding the recognition range of interactive events.

[0055] In optional embodiments, such as Figure 4 As shown, the deep learning model determines the interaction events of the moving object with respect to the target environment at the current moment through the following operations: Step S400: Perform audio analysis on the multiple sound segments to determine the number of moving objects, the speech content of each moving object, the emotional state of each moving object, and / or the background sound of the target environment.

[0056] Step S402: Determine the interaction events of the mobile objects with respect to the target environment at the current moment based on the number of mobile objects, the voice content of each mobile object, the emotional state of each mobile object and / or the background sound of the target environment.

[0057] For example, the deep learning model can be a multi-task learning model, a speech content recognition model, or a composite model integrating multiple modules. Specifically, the spectral features of sound segments are extracted using a convolutional neural network (CNN) in the deep learning model. Based on the spectral features, different moving objects can be distinguished and their numbers counted using separation techniques, speech content can be parsed using a speech recognition model while analyzing emotional states using an attention-based model, and background sound types can be identified using an environmental sound classifier (such as YAMNet). Based on these multi-dimensional audio features, the interaction events between the moving object and the target environment at the current moment can be determined (such as "multi-person conversation," "working alone," etc.). In some embodiments, invalid sound segments can be filtered out by performing speech recognition and semantic analysis on each sound segment, reducing resource consumption.

[0058] In this embodiment, by comprehensively analyzing multi-dimensional audio features such as the number of moving objects, speech content, emotional state, and ambient background noise in the target environment through a deep learning model, interactive events that precisely match the target scene at the current moment can be identified, thereby achieving a more accurate and in-depth understanding and judgment of events occurring in the target environment.

[0059] Step S104 Based on the moving object, determine the effective information content of each of the information event and the interaction event, and determine the information event or the interaction event as the target event based on the effective information content.

[0060] Effective information content can be used to quantify the importance of each event to a mobile target audience. For example, the importance of an event can be comprehensively evaluated across multiple dimensions to determine its effective information content. These dimensions include timeliness, relevance, scope of impact, and user preferences. For instance, breaking news can be prioritized over current activities, a user operating a relevant device over irrelevant advertisements, and a major fire alarm over a single device malfunction. Based on the importance of each event, target events can be identified from informational or interactive events. Step S106 Based on the target event, the target interactive content is determined, and the target interactive content is presented to the moving object.

[0061] In practical applications, target interactive content adapted to mobile objects can be generated based on the event type and / or user characteristics (pre-entered information) of the target event. Specifically, the semantic features of the target event (such as environmental alarms, schedule reminders, or user emotional states) are first parsed. Then, based on the semantic features, the most relevant target interactive content, such as facial animations, can be matched from a pre-set database. Alternatively, target interactive content can be generated in real time based on semantic features and a trained large model. After determining the target interactive content, the robot can present it to the mobile object in real time, thereby achieving human-computer interaction and enhancing the user experience. The following will illustrate how to determine and present target interactive content using several examples.

[0062] In an optional embodiment, the target interactive content includes target animation, target sound effects, and / or target text; wherein the target animation is determined from a plurality of pre-configured animations based on the target event, the target sound effects are generated based on the interaction events of the moving object with respect to the target environment at the current moment and a pre-trained speech model, and the target text is generated based on the information events at the current moment and a pre-trained text model.

[0063] For example, the target animation could be facial animation, weather animation, etc. The target sound effect could be white noise that helps with sleep or a bedtime audio that the user likes. The target text could be suggestive text generated based on news time or weather information, etc.

[0064] In this embodiment, by presenting interactive content such as target animation, target sound effects, and target text to the moving object, the diversity of interactive content can be increased, thereby enhancing the emotional companionship and interactive experience between the user and the robot.

[0065] In an optional embodiment, the robot includes a spherical LED display device, which comprises a spherical shell and a plurality of LEDs densely distributed on the spherical shell. Step S106 may include: generating a first control command for the plurality of LEDs based on the target interactive content, to control the on / off state of the plurality of LEDs. The first control command forms a corresponding LED dot matrix pattern by controlling the on / off state of the plurality of LEDs. The LED dot matrix pattern includes Emoji patterns.

[0066] In practical applications, spherical LED display devices can be flexible LED displays or LED lamp bead arrays, such as... Figure 7 and Figure 8 As shown. Based on the target interactive content, the switching states of multiple LEDs in the spherical LED display device can be dynamically controlled to form dot matrix patterns such as Emojis, thereby achieving intuitive and vivid visual interaction. In this embodiment, the design of the spherical carrier enables the displayed expressions and animations to possess a strong sense of life and emotional expression, significantly enhancing the user's interactive experience at the psychological and emotional levels, and effectively improving the robot's acceptance and intimacy in the home environment.

[0067] In optional embodiments, such as Figure 5 As shown, step S106 may further include: Step S500: Determine the location information of the moving object within the target environment.

[0068] Step S502: Based on the location information of the moving object within the target environment and the target interactive content, a second control command is generated for the plurality of LEDs to control the on / off state of the plurality of LEDs; wherein, the second control command forms an LED dot matrix pattern facing the moving object by controlling the on / off state of the plurality of LEDs.

[0069] In practical applications, the presentation can be further optimized by incorporating the location information of the moving object. Specifically, the moving object can be located in real time using infrared pyroelectric sensors, microwave radar sensors, and linear CCD sensors to determine its location information. Then, based on the location information and the target interaction content, a second control command is generated for multiple LEDs. Next, the switching states of the multiple LEDs are controlled through the second control command to form an LED dot matrix pattern facing the moving object.

[0070] In this embodiment, determining the orientation of the LED dot matrix pattern based on the position of the moving object in the target environment can further improve the interaction between the user and the robot.

[0071] In some embodiments, a user feedback closed-loop algorithm can be introduced. Based on proactive user feedback, the algorithm parameters and analysis model are automatically optimized, thereby achieving continuous adaptive optimization of the algorithm and enabling the robot to maintain high accuracy and personalized performance over long-term operation. Specifically, user feedback data can be received through the interactive interface provided by a companion mobile app. For example, the app might ask the user, "How do you think of my performance this time?" The user can quickly select from options such as "Very good," "Good," and "Inappropriate" on the interactive interface. The robot then optimizes the algorithm in real time based on the feedback data, continuously improving the accuracy and personalization of event recognition.

[0072] To make this application easier to understand, the following is combined with... Figure 6 , Figure 7 and Figure 8 An example application is provided.

[0073] This application applies to a robot, which may include: a non-image motion detection module, a sound acquisition module, a sound analysis module, a microprocessor, and a spherical screen interaction module. The non-image motion detection module may include one or more combinations of an infrared pyroelectric sensor, a microwave radar sensor, and / or a linear CCD sensor.

[0074] Step S11: Determine whether there is a moving object (e.g., a person) in the target environment by using a combination of infrared pyroelectric sensor, microwave radar sensor and / or linear CCD sensor (non-image motion detection module).

[0075] Specifically, (1) the infrared pyroelectric sensor determines whether there is a moving object in the target environment at the current moment based on the infrared radiation of the target environment. (2) the microwave radar sensor determines whether there is a moving object in the target environment at the current moment based on the microwave frequency of the target environment. (3) the linear CCD sensor determines whether there is a moving object in the target environment at the current moment based on the light intensity of the target environment.

[0076] Step S12: If there is a moving object, obtain the information events at the current moment and the interaction events of the moving object with respect to the target environment at the current moment.

[0077] Specifically, (1) the microprocessor can connect to the Internet via Wi-Fi to obtain information events at the current moment, such as real-time weather and news. (2) Real-time sound slices are collected through a high-sensitivity microphone array in the sound acquisition module. The length of each collected sound slice can be set to 3-10 seconds. (3) The meaning represented in the sound slices is analyzed by a pre-trained deep learning model in the sound analysis module to obtain the interaction events of the moving object with respect to the target environment at the current moment. Specifically, the first layer algorithm is used to determine whether there is valid sound information in the sound slice. If there is no valid sound information in the sound slice, no interaction content is fed back. If there is valid sound information in the sound slice, the second layer algorithm is used to determine the emotion contained in the sound slice, whether it contains certain specific keywords, etc. The sound slice can include the number of moving objects, the voice content of each moving object, the emotional state of each moving object and / or the background sound of the target environment, etc.

[0078] Step S13: Using a microprocessor, determine the effective information content of each information event and interaction event based on the moving object, and determine the information event or interaction event as the target event based on the effective information content.

[0079] Step S14: The microprocessor determines the target interactive content based on the target event and sends the target interactive content to the spherical screen interaction module.

[0080] Step S15: Present the target interactive content to the moving object through the spherical screen interaction module (spherical LED display device).

[0081] Example 2 Figure 9 The diagram schematically illustrates an interactive device according to Embodiment 2 of this application. This device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program module referred to in the embodiments of this application refers to a series of computer program instruction segments capable of performing a specific function. The following description will specifically introduce the function of each program module in this embodiment. For example... Figure 9 As shown, the device 1000 may include: a first determining module 1100, an acquiring module 1200, a second determining module 1300, and a presenting module 1400, wherein: The first determining module 1100 is used to determine whether there is a moving object in the target environment at the current moment; The acquisition module 1200 is used to acquire information events at the current moment and determine the interaction events of the mobile object with respect to the target environment at the current moment when there is a moving object. The second determining module 1300 is configured to determine the effective information content of the information event and the interaction event respectively based on the moving object, and determine the information event or the interaction event as a target event based on the effective information content; and The presentation module 1400 is used to determine the target interactive content based on the target event and present the target interactive content to the moving object.

[0082] In an optional embodiment, the robot includes a spherical LED display device, which comprises a spherical housing and a plurality of LEDs densely distributed on the spherical housing; the presentation module 1400 is further configured to: Based on the target interactive content, a first control command is generated for the plurality of LEDs to control the on / off state of the plurality of LEDs; The first control command controls the switching state of the plurality of LEDs to form a corresponding LED dot matrix pattern. The LED dot matrix pattern includes Emoji patterns.

[0083] In an optional embodiment, the presentation module 1400 is further configured to: Determine the location information of the moving object within the target environment; Based on the location information of the moving object within the target environment and the target interaction content, a second control command is generated for the plurality of LEDs to control the on / off state of the plurality of LEDs; The second control command controls the switching states of the plurality of LEDs to form an LED dot matrix pattern facing the moving object.

[0084] In an optional embodiment, the target interactive content includes target animation, target sound effects, and / or target text; The target animation is determined from a plurality of pre-configured animations based on the target event; the target sound effect is generated based on the interaction events of the moving object with respect to the target environment at the current moment and a pre-trained speech model; and the target text is generated based on the information events at the current moment and a pre-trained text model.

[0085] In an optional embodiment, the acquisition module 1200 is further configured to: Acquire sound data at the current moment, the sound data including multiple sound segments that are consecutive in time; The multiple sound segments are sequentially input into a pre-trained deep learning model according to the time sequence, so as to determine the interaction events of the moving object with respect to the target environment at the current moment through the deep learning model.

[0086] In an optional embodiment, the deep learning model determines the interaction events of the moving object with respect to the target environment at the current moment through the following operations: Audio analysis is performed on the multiple sound segments to determine the number of moving objects, the speech content of each moving object, the emotional state of each moving object, and / or the background sound of the target environment; Based on the number of moving objects, the voice content of each moving object, the emotional state of each moving object, and / or the background noise of the target environment, determine the interaction events of the moving objects with respect to the target environment at the current moment.

[0087] In an optional embodiment, the acquisition module 1200 is further configured to: Obtain pre-entered information, which includes the profile information and preference information of the mobile object; Based on the profile information and the preference information, the information event at the current moment is obtained; wherein the information event is matched with the moving object.

[0088] In an optional embodiment, the first determining module is further configured to: The system uses pre-configured infrared pyroelectric sensors, microwave radar sensors, and / or linear CCD sensors to determine whether there are moving objects in the target environment at the current moment. Specifically, the infrared pyroelectric sensor determines whether there is a moving object in the target environment at the current moment based on the infrared radiation of the target environment; the microwave radar sensor determines whether there is a moving object in the target environment at the current moment based on the microwave frequency of the target environment; and the linear CCD sensor determines whether there is a moving object in the target environment at the current moment based on the light intensity of the target environment.

[0089] Example 3 Figure 10 This illustration schematically depicts a hardware architecture diagram of a computer device 10000 suitable for implementing an interactive method according to Embodiment 3 of this application. In some embodiments, the computer device 10000 may be a terminal device such as a smartphone, wearable device, tablet computer, personal computer, in-vehicle terminal, game console, virtual device, workbench, digital assistant, set-top box, robot, etc. In other embodiments, the computer device 10000 may be a rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc. Figure 10 As shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked with each other via a system bus. Wherein: The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of a computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as program code for interactive methods. Furthermore, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0090] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0091] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0092] It should be pointed out that, Figure 10 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0093] In this embodiment, the interactive method stored in memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiments of this application.

[0094] Example 4 This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the interactive method in the embodiments.

[0095] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Of course, the computer-readable storage medium may include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the interactive method in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0096] Example 5 This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in the above embodiments.

[0097] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computer devices. They can be centralized on a single computer device or distributed across a network of multiple computer devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computer device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0098] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. An interactive method, characterized in that, For use with a robot, the method includes: Determine whether there are moving objects within the target environment at the current moment; In the case of a moving object, acquire the information events at the current moment and determine the interaction events of the moving object with respect to the target environment at the current moment; Based on the moving object, determine the effective information content of each of the information event and the interaction event, and determine the information event or the interaction event as the target event based on the effective information content; and The target interactive content is determined based on the target event, and the target interactive content is presented to the moving object.

2. The method according to claim 1, characterized in that, The robot includes a spherical LED display device, which comprises a spherical shell and a plurality of LEDs densely distributed on the spherical shell; Presenting the target interactive content to the moving object includes: Based on the target interactive content, a first control command is generated for the plurality of LEDs to control the on / off state of the plurality of LEDs; The first control command controls the switching state of the plurality of LEDs to form a corresponding LED dot matrix pattern. The LED dot matrix pattern includes Emoji patterns.

3. The method according to claim 2, characterized in that, Presenting the target interactive content to the moving object further includes: Determine the location information of the moving object within the target environment; Based on the location information of the moving object within the target environment and the target interaction content, a second control command is generated for the plurality of LEDs to control the on / off state of the plurality of LEDs; The second control command controls the switching states of the plurality of LEDs to form an LED dot matrix pattern facing the moving object.

4. The method according to claim 1, characterized in that, The target interactive content includes target animation, target sound effects, and / or target text; The target animation is determined from a plurality of pre-configured animations based on the target event; the target sound effect is generated based on the interaction events of the moving object with respect to the target environment at the current moment and a pre-trained speech model; and the target text is generated based on the information events at the current moment and a pre-trained text model.

5. The method according to claim 1, characterized in that, Determining the interaction events of the moving object with respect to the target environment at the current moment includes: Acquire sound data at the current moment, the sound data including multiple sound segments that are consecutive in time; The multiple sound segments are sequentially input into a pre-trained deep learning model according to the time sequence, so as to determine the interaction events of the moving object with respect to the target environment at the current moment through the deep learning model.

6. The method according to claim 5, characterized in that, The deep learning model determines the interaction events of the moving object with respect to the target environment at the current moment through the following operations: Audio analysis is performed on the multiple sound segments to determine the number of moving objects, the speech content of each moving object, the emotional state of each moving object, and / or the background sound of the target environment; Based on the number of moving objects, the voice content of each moving object, the emotional state of each moving object, and / or the background noise of the target environment, determine the interaction events of the moving objects with respect to the target environment at the current moment.

7. The method according to claim 1, characterized in that, Retrieve information events at the current moment, including: Obtain pre-entered information, which includes the profile information and preference information of the mobile object; Based on the profile information and the preference information, the information event at the current moment is obtained; wherein the information event is matched with the moving object.

8. The method according to any one of claims 1 to 7, characterized in that, Determine whether there are moving objects in the target environment at the current moment, including: The system uses pre-configured infrared pyroelectric sensors, microwave radar sensors, and / or linear CCD sensors to determine whether there are moving objects in the target environment at the current moment. Specifically, the infrared pyroelectric sensor determines whether there is a moving object in the target environment at the current moment based on the infrared radiation of the target environment; the microwave radar sensor determines whether there is a moving object in the target environment at the current moment based on the microwave frequency of the target environment; and the linear CCD sensor determines whether there is a moving object in the target environment at the current moment based on the light intensity of the target environment.

9. An interactive device, characterized in that, For use in a robot, the device includes: The first determining module is used to determine whether there are moving objects in the target environment at the current moment; The acquisition module is used to acquire information events at the current moment and determine the interaction events of the mobile object with respect to the target environment at the current moment, in the case of a moving object; The second determining module is configured to determine the effective information content of the information event and the interaction event respectively based on the moving object, and determine the information event or the interaction event as a target event based on the effective information content; and The presentation module is used to determine the target interactive content based on the target event and present the target interactive content to the moving object.

10. A computer device, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in claims 1 to 8.

Images