Humanoid robot intelligent interaction method and system based on open source gap

Abstract

The invention discloses a humanoid robot intelligent interaction method and system based on an open source gap, and the method comprises the steps: obtaining the identity information and sensing parameters of a target object when the target object is sensed to enter a welcome area; identifying the identity of the target object according to the identity information, and playing a corresponding welcome word according to the identified identity; obtaining historical records or preference information of the target object according to the identity information, and analyzing the intention and demand of the target object in combination with the sensing parameters, the historical records or the preference information; and navigating the target object to a corresponding welcome area according to the intention and the demand, and recommending a specified activity or facility to the target object. According to the invention, the welcome efficiency of the humanoid robot is improved, and the humanized interaction and continuous optimization capabilities of the humanoid robot are enhanced.

Description

Technical Field

[0001] This invention relates to the field of humanoid robot technology, and in particular to a humanoid robot intelligent interaction method and system based on the open-source HarmonyOS. Background Technology

[0002] As humanoid robot technology rapidly penetrates the service sector, welcoming scenarios, due to their high frequency and high interaction requirements, have become a key breakthrough for commercialization. Traditional welcoming robots mostly rely on pre-set scripts and fixed processes. For example, in government service halls or corporate showrooms, they complete guidance tasks through voice broadcasts and basic Q&A. However, these methods are rigid in dynamic environments: when faced with sudden surges in customers or complex inquiries, the robot struggles to flexibly adjust its response strategy, resulting in a fragmented user experience. More importantly, early solutions relied on single-modal input (such as pure voice or touchscreen interaction). In noisy or multi-user concurrent scenarios, recognition accuracy drops significantly, and the lack of deep understanding of user emotions and intentions easily leads to misjudgments or service interruptions.

[0003] In recent years, embodied intelligence technology has brought groundbreaking progress to visitor interaction. By integrating multimodal sensors (such as vision, speech, and force) with generative AI models, robots can analyze environmental context in real time and generate human-like behavioral feedback. For example, in museum guided tours, robots can dynamically adjust the pace of their explanations based on changes in visitors' facial expressions and tone of voice, and even proactively initiate personalized recommendations. However, existing methods still face core challenges in visitor scenarios: First, traditional architectures struggle to balance the accuracy of behavioral agents with the flexibility of emotional agents, easily leading to response delays or logical conflicts; second, in dense crowds or noisy environments, robots lack the ability to parse unstructured instructions (such as dialects or vague expressions), often causing functional failures.

[0004] The current technological bottleneck stems from structural flaws in the interaction paradigm, with the unique nature of welcoming scenarios exacerbating the technical challenges. For instance, in large exhibitions, robots need to complete identity recognition, path planning, and multi-turn dialogue within a short timeframe, while traditional solutions, relying on high-precision positioning and pre-set maps, struggle to adapt to dynamic layout changes. Some advanced attempts have focused on enhancing flexibility through smartphone-assisted interaction, but these methods remain limited to point-to-point communication and cannot achieve collaborative group services.

[0005] Therefore, the existing technology still suffers from low welcoming efficiency and poor humanized interaction and continuous optimization capabilities, and the technology needs further improvement. Summary of the Invention

[0006] The main objective of this invention is to provide a humanoid robot intelligent interaction method and system based on the open-source HarmonyOS, aiming to solve the problems of low greeting efficiency and poor humanized interaction and continuous optimization capabilities in the existing technology.

[0007] To achieve the above objectives, this invention provides a humanoid robot intelligent interaction method based on the open-source HarmonyOS, which includes the following steps: When a target object is detected entering the welcoming area, the identity information and sensing parameters of the target object are obtained; The identity of the target object is identified based on the identity information, and a corresponding welcoming message is played according to the identified identity. Based on the identity information, obtain the target object's historical records or preference information, and analyze the target object's intent and needs by combining the sensing parameters, the historical records, or the preference information; Based on the stated intent and needs, the target object is navigated to the corresponding welcoming area, and designated activities or facilities are recommended to the target object.

[0008] Optionally, the step of acquiring the identity information and sensing parameters of the target object when it is sensed to enter the welcoming area includes: Wireless communication connections are established with multiple sensors via the BLE communication protocol; wherein the multiple sensors include any combination of temperature and humidity sensors, infrared sensors, LED light sensors, NFC sensors, blood oxygen and heart rate sensors, human body sensors, and OLED sensors. The human body sensor detects the entry status of the target object in real time.

[0009] Optionally, when a target object is sensed to enter the welcoming area, acquiring the target object's identity information and sensing parameters specifically includes: When the target object is detected entering the welcoming area, proceed to the welcoming area to greet the target object; The system acquires facial images and human body video streams of the target object through its own camera, and obtains the ID card information of the target object through the NFC sensor. The identity information is obtained by combining the ID card information, the facial image, and the human body video stream; The sensing parameters are obtained by acquiring corresponding sensor data through temperature and humidity sensors, infrared sensors, LED light sensors, blood oxygen and heart rate sensors, and OLED sensors.

[0010] Optionally, the step of identifying the target object's identity based on the identity information and playing a corresponding welcoming message based on the identified identity specifically includes: Analyze the gender and behavior type of the target object based on the identity information; Based on the gender and behavior type of the target object, the corresponding welcoming words are matched from the preset welcoming database and played in conjunction with the corresponding display window and welcoming screen.

[0011] Optionally, the step of obtaining the target object's historical records or preference information based on the identity information, and analyzing the target object's intent and needs in conjunction with the sensing parameters, the historical records, or the preference information, specifically includes: Based on the identity information, historical access data, historical behavior data, and historical feedback data of the target object are obtained to obtain the historical record of the target object; Based on the identity information, the explicit and implicit preference data of the target object are obtained to obtain the preference information of the target object; Based on a multi-dimensional intent analysis model, the intent and needs of the target object are analyzed by combining the sensing parameters, historical records, or preference information.

[0012] Optionally, the step of analyzing the target object's intent and needs based on a multi-dimensional intent analysis model, combined with the sensing parameters, historical records, or preference information, specifically includes: Analyze the current environmental factors based on the sensor parameters; Based on the analysis of the historical records or preference information, and combined with the target object's current action trajectory and status, the target object's future intentions and immediate needs are determined; Based on current environmental factors, the stated future intentions, and the stated immediate needs, a collaborative analysis method based on group characteristics is used to analyze and obtain the final intentions and needs of the target object.

[0013] Optionally, the step of navigating the target object to the corresponding welcoming area according to the intent and needs, and recommending specified activities or facilities to the target object, specifically includes: Navigate the target object to the reserved / recommended meeting room or to the corresponding manager's office according to the stated intent and needs; Recommend specified activities or facilities to the target audience and provide in-depth explanations of the target audience.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a humanoid robot intelligent interaction system based on the open-source HarmonyOS, wherein the humanoid robot intelligent interaction system based on the open-source HarmonyOS includes: The information and parameter acquisition module is used to acquire the identity information and sensing parameters of the target object when it senses that the target object has entered the welcoming area. The identity recognition module is used to identify the identity of the target object based on the identity information, and play the corresponding welcoming words according to the identified identity; The intent and demand analysis module is used to obtain the historical records or preference information of the target object based on the identity information, and to analyze the intent and demand of the target object in combination with the sensing parameters, the historical records or the preference information; The welcome guidance module is used to navigate the target object to the corresponding welcome area according to the intent and needs, and recommend specified activities or facilities to the target object.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a humanoid robot intelligent interaction program based on open source HarmonyOS stored in the memory and executable on the processor. When the humanoid robot intelligent interaction program based on open source HarmonyOS is executed by the processor, it implements the steps of the humanoid robot intelligent interaction method based on open source HarmonyOS as described above.

[0016] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a humanoid robot intelligent interaction program based on the open-source HarmonyOS, and when the humanoid robot intelligent interaction program based on the open-source HarmonyOS is executed by a processor, it implements the steps of the humanoid robot intelligent interaction method based on the open-source HarmonyOS as described above.

[0017] In this invention, a human body sensor detects the entry status of a target object in real time. When the target object enters the welcoming area, its identity information and sensor parameters are acquired. Based on the identity information, the target object's identity is identified, and a corresponding welcoming message is played, providing a personalized welcoming experience. Furthermore, based on the identity information, the target object's historical records or preference information are obtained. Combining the sensor parameters, historical records, or preference information, the target object's intentions and needs are analyzed, accurately understanding the guest's needs and providing timely and precise service. Based on the intentions and needs, the target object is navigated to the corresponding welcoming area, and designated activities or facilities are recommended, providing more accurate decision support for the welcoming service. This invention improves the welcoming efficiency of humanoid robots and enhances their humanized interaction and continuous optimization capabilities. Attached Figure Description

[0018] Figure 1 This is a flowchart of a preferred embodiment of the humanoid robot intelligent interaction method based on the open-source HarmonyOS of the present invention; Figure 2 This is a schematic diagram of a scenario for the humanoid robot intelligent interaction system based on the open-source HarmonyOS of this invention; Figure 3 This is a schematic diagram of an intelligent welcoming scene using a humanoid robot based on the open-source HarmonyOS. Figure 4 This is a structural diagram of a preferred embodiment of the humanoid robot intelligent interaction system based on the open-source HarmonyOS of the present invention; Figure 5 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] In recent years, embodied intelligence technology has brought groundbreaking progress to visitor interaction. By integrating multimodal sensors (such as vision, speech, and force) with generative AI models, robots can analyze environmental context in real time and generate human-like behavioral feedback. For example, in museum guided tours, robots can dynamically adjust the pace of their explanations based on changes in visitors' facial expressions and tone of voice, and even proactively initiate personalized recommendations. However, existing methods still face core challenges in visitor scenarios: First, traditional architectures struggle to balance the accuracy of behavioral agents with the flexibility of emotional agents, easily leading to response delays or logical conflicts; second, in dense crowds or noisy environments, robots lack the ability to parse unstructured instructions (such as dialects or vague expressions), often causing functional failures.

[0021] The current technological bottleneck stems from structural flaws in the interaction paradigm, with the unique nature of welcoming scenarios exacerbating the technical challenges. For instance, in large exhibitions, robots need to complete identity recognition, path planning, and multi-turn dialogue within a short timeframe. Traditional solutions, relying on high-precision positioning and pre-set maps, struggle to adapt to dynamic layout changes. Some advanced attempts have focused on enhancing flexibility through smartphone-assisted interaction, but these methods remain limited to point-to-point communication and cannot achieve collaborative group services. Therefore, existing technologies still suffer from low welcoming efficiency and poor capabilities in providing humanized interaction and continuous optimization, requiring further improvement.

[0022] To address the issues of low greeting efficiency and poor humanized interaction and continuous optimization capabilities in existing technologies, this invention provides a humanoid robot intelligent interaction method based on the open-source HarmonyOS. This method, upon sensing a target object entering the greeting area, acquires the target object's identity information and sensor parameters; identifies the target object's identity based on the identity information and plays a corresponding greeting message; acquires the target object's historical records or preference information based on the identity information, and analyzes the target object's intentions and needs by combining the sensor parameters, historical records, or preference information; navigates the target object to the corresponding greeting area based on the intentions and needs, and recommends specified activities or facilities to the target object. This invention improves the greeting efficiency of humanoid robots and enhances their humanized interaction and continuous optimization capabilities.

[0023] The preferred embodiment of the present invention describes a humanoid robot intelligent interaction method based on open-source HarmonyOS, such as... Figure 1 As shown, the humanoid robot intelligent interaction method based on open-source HarmonyOS includes the following steps: Step S10: When the target object is sensed to enter the welcoming area, the identity information and sensing parameters of the target object are obtained.

[0024] It should be noted that in this application, a humanoid robot based on the open-source HarmonyOS system is used to realize intelligent interaction in the welcoming scene. This humanoid robot stands out in the welcoming scene with its efficient welcoming, intelligent management, humanized interaction and continuous optimization capabilities. Compared with other welcoming robots, it shows more significant advantages in human body sensing accuracy, personalized service provision, environmental monitoring and data analysis and overall user experience, which can bring enterprises a higher level of brand image and customer satisfaction.

[0025] As an example, the scenario of the entire system in this application is illustrated as follows: Figure 2 As shown, the humanoid robot used in this application is model RK3568A, which runs on the KaihongOS operating system and has BLE (Bluetooth Low Energy) communication capabilities. It needs to interact with 7 types of ESP32 sensors.

[0026] As an assistive sensor for humanoid robots, this application employs ESP32 sensors, specifically seven different types, including: temperature and humidity sensors, infrared sensors, LED light sensors, NFC sensors, blood oxygen and heart rate sensors, human body sensing sensors, and OLED sensors. Each sensor has BLE communication capabilities. Sensors may be in a network-configured or unconfigured state, but all must remain in a network-ready state to communicate with the robot.

[0027] In this application, communication between the robot and the sensor is achieved through a BLE soft bus; as a virtual bus technology, the BLE soft bus enables wireless communication between devices. In the application scenario of this application, the BLE soft bus is used to connect the robot RK3568A and the Category 7 ESP32 sensor.

[0028] As an example, the intelligent greeting scenario of the humanoid robot in this application is illustrated as follows: Figure 3 As shown, the following is a description of the specific management process for this robot in this scenario: 1. Welcome ceremony begins: Personnel detection: The robot uses wireless human body sensors (such as infrared sensors, ultrasonic sensors, or more advanced WiFi radar human body sensors) to sense the movement of people in the reception area in real time. When the sensor detects that someone has entered the reception area, the reception process is immediately triggered.

[0029] 2. Welcome service execution: Greeting and reception: Upon receiving a sensor signal, the robot automatically starts and heads to the reception area to greet guests. It uses machine vision technology to identify the guest's gender and plays a corresponding welcome message, such as "Welcome, madam / sir!"

[0030] Guiding to the rest area: After recognizing the guest's gender and playing a welcome message, the robot will guide them to the rest area according to a preset navigation path or dynamic path planning. During the guidance process, the robot may provide additional information or guidance through voice interaction or screen display.

[0031] 3. Environmental Management and Data Display: Activating wireless lighting and temperature / humidity sensing: After arriving at the rest area, the robot activates the wireless lighting panel via remote control to provide a suitable lighting environment. Simultaneously, the robot or its connected temperature and humidity sensors begin to sense and monitor the temperature and humidity data of the rest area in real time.

[0032] Display Screen Sensor Data Display: The robot's equipped display screen, or a connected display screen, will show the temperature and humidity data of the rest area in real time. Guests can intuitively understand the current environmental conditions through the display screen, and thus adjust their state or make corresponding requests as needed.

[0033] Based on the aforementioned intelligent welcoming scenario, this application needs to establish a wireless communication connection between the humanoid robot and all sensors in the scenario before performing personnel sensing.

[0034] Furthermore, in this application, the step of obtaining the identity information and sensing parameters of the target object when it is sensed to enter the welcoming area includes: establishing a wireless communication connection with multiple sensors via the BLE communication protocol; wherein the multiple sensors include any combination of temperature and humidity sensors, infrared sensors, LED light sensors, NFC sensors, blood oxygen and heart rate sensors, human body sensors, and OLED sensors; and sensing the entry status of the target object in real time through the human body sensors.

[0035] As an example, in this application, the process of establishing wireless communication connections with multiple sensors via the BLE communication protocol is as follows: 1. BLE Discovery and Networking: After powering on, the RK3568A robot actively scans for surrounding ESP32 sensors via the BLE soft bus. Upon detecting a sensor, the robot pairs and connects with it via BLE, establishing a stable communication network. The robot needs to be able to identify and distinguish seven different types of sensors for subsequent data interaction.

[0036] 2. Data Interaction: The RK3568A robot sends data requests or control commands to the ESP32 sensor via the BLE soft bus. Upon receiving the request or command, the sensor processes the data or executes the corresponding action, and then sends the result data back to the robot via the BLE soft bus. The robot analyzes and processes the received data, achieving intelligent interaction with the sensor.

[0037] 3. Memory Reconnection Mechanism: The RK3568A robot needs to maintain a list of connected sensors, recording the BLE address and related information for each sensor. When the robot is powered off or disconnected from the network and then reconnected / powered on, it automatically iterates through the list of connected sensors. For each sensor in the list, the robot attempts to reconnect via the BLE soft bus. If the reconnection is successful, data interaction with the sensor continues; if the reconnection fails, the error message is recorded and other recovery measures are attempted.

[0038] This application establishes a wireless communication connection between a humanoid robot and multiple sensors using the BLE communication protocol. Security measures such as encryption and authentication can be employed to prevent data leakage and unauthorized access, ensure the security of data transmission, and ensure the stability and reliability of the connection, thereby reducing the risk of communication failure and data loss.

[0039] After establishing a wireless communication connection between the humanoid robot and multiple sensors, this application uses a human body sensing sensor in the sensing system to sense the dynamics of people in a designated area (e.g., the welcoming area at the entrance of the lobby) in real time.

[0040] Furthermore, in this application, the step of obtaining the identity information and sensing parameters of the target object when it is sensed to enter the welcoming area specifically includes: when the target object is sensed to enter the welcoming area, going to the welcoming area to greet the target object; obtaining the target object's facial image and body video stream through its own camera, and obtaining the target object's ID card information through the NFC sensor; combining the ID card information, the facial image, and the body video stream to obtain the identity information; and obtaining the corresponding sensor data through temperature and humidity sensors, infrared sensors, LED light sensors, blood oxygen and heart rate sensors, and OLED sensors to obtain the sensing parameters.

[0041] Specifically, in this application, when a target object is sensed entering the welcoming area, the robot autonomously navigates to the preset coordinates of the "welcoming area" using LiDAR and SLAM technology. It pauses 1.5 meters from the target object, adjusts its posture to face the user, and prepares for interaction. Then, a preset sequence of welcoming actions is generated: the robot tilts its body slightly forward (approximately 15 degrees) to indicate welcome. A smiling animation is displayed on an OLED screen.

[0042] After the robot completes its greeting action, high-definition facial images and human body video streams are captured using the robot's own high-definition camera (e.g., a 4K / 60fps wide-angle camera). The high-definition facial images are primarily used for face recognition, while the human body video stream is mainly used for subsequent trajectory and motion recognition. The MTCNN algorithm can be used to detect faces in the video stream in real time. Key points (eyes, nose, corners of the mouth) are located on the detected faces to extract standardized facial regions.

[0043] In this application, while acquiring high-definition facial images and human body video streams, the target's ID card information can also be obtained through an NFC sensor; subsequently, the target's identity is determined through a multi-source information fusion strategy, as follows: Data source credibility weights: NFC card information 0.6 (highest), facial recognition result 0.3, human morphology analysis 0.1. When NFC information matches facial information, identity is directly confirmed with a confidence level of 0.95. When only NFC information is available, identity is confirmed, but the confidence level drops to 0.75.

[0044] In this application, while obtaining identity information, corresponding sensor data can also be obtained through temperature and humidity sensors, infrared sensors, LED light sensors, blood oxygen and heart rate sensors, and OLED sensors to obtain the sensing parameters. These parameters can be used for subsequent target demand analysis and on-site environment settings. For example, the state and position of the target can be analyzed based on the temperature and humidity sensors, infrared sensors, and blood oxygen and heart rate sensors, the LED light sensors can be used to arrange ambient lighting on site, and the corresponding video content or images can be displayed through the OLED sensors.

[0045] Step S20: Identify the identity of the target object based on the identity information, and play the corresponding welcoming words according to the identified identity.

[0046] Understandably, the humanoid robot of this application can use visual technology to identify the gender of guests and then play corresponding welcoming words or copywriting content. These targeted welcoming words or copywriting content can be customized according to the identified target type, or generated by big data analysis according to the identified target type.

[0047] Furthermore, in this application, the step of identifying the identity of the target object based on the identity information and playing the corresponding welcoming words according to the identified identity specifically includes: analyzing the gender and behavior type of the target object based on the identity information; matching the corresponding welcoming words from a preset welcoming database based on the gender and behavior type of the target object, and playing the corresponding welcoming words in conjunction with the corresponding display window and welcoming screen.

[0048] Specifically, in this application, based on the acquired high-definition facial images and video streams, the OpenCV model is used to perform facial contour feature point analysis, hairstyle style classification and recognition, and common accessory feature extraction. Then, the final gender and behavior type recognition results are obtained by using a preset confidence level.

[0049] The preset welcome database can be configured in the following ways:

[0050] In this application, based on the gender and behavior type of the target object, a corresponding welcoming message is matched from the aforementioned preset welcoming database, and played in conjunction with a corresponding display window and welcoming screen. The display window can be an on-site OLED display or a display screen integrated into the humanoid robot. The welcoming screen can be configured in the following ways: Male business style: Dark blue color scheme, simple geometric shapes, and a design that emphasizes efficiency; Business-style for women: Light purple tones, elegant lines, and a professional design; Social media-friendly: warm colors, dynamic effects, and a welcoming atmosphere.

[0051] Based on the existing intelligent greeting system, this application adds precise identification of the gender and behavior type of the target audience to further enhance the level of personalized service, and calls up matching greeting words and visual content accordingly, thus constructing an end-to-end intelligent greeting experience.

[0052] Step S30: Obtain the target object's historical records or preference information based on the identity information, and analyze the target object's intent and needs in conjunction with the sensing parameters, the historical records, or the preference information.

[0053] It is understood that in this application, historical records or preference information are obtained based on the target's identity information, and combined with previously obtained sensor parameters, historical records or preference information, a multi-dimensional intent analysis model is used to analyze and obtain the target's intent and needs for entering the current scene; wherein, the intent includes: business negotiation, visit and study, etc., and the needs include: on-site environment needs, activity details information needs, rest or other experience needs, etc.

[0054] Furthermore, in this application, the step of obtaining the target object's historical records or preference information based on the identity information, and analyzing the target object's intent and needs in conjunction with the sensing parameters, the historical records, or the preference information, specifically includes: obtaining the target object's historical access data, historical behavior data, and historical feedback data based on the identity information to obtain the target object's historical records; obtaining the target object's explicit preference data and implicit preference data based on the identity information to obtain the target object's preference information; and analyzing the target object's intent and needs based on a multi-dimensional intent analysis model, in conjunction with the sensing parameters, the historical records, or the preference information.

[0055] In this application, the process of obtaining the target object's historical records or preference information, and analyzing the target object's intent and needs, is as follows: I. Data Collection and Information Acquisition: 1) Historical records, sourced from CRM system, appointment system, access logs, and consumption records.

[0056] Key data is: Historical visit data: visit time, frequency, duration of stay, and areas / facilities visited.

[0057] Behavioral history data: past activities participated in and questions consulted.

[0058] Feedback history data: suggestions, complaints, and positive reviews.

[0059] 2) The method for obtaining preference information is: Explicit preferences: User-defined preferences (e.g., preferred language, Do Not Disturb mode, interest tags).

[0060] Implicit preferences: These are discovered by analyzing historical records (e.g., frequent visits to coffee shops may indicate a "love of coffee").

[0061] 3) Environmental parameters: time (weekday / weekend, morning / afternoon), weather, season, special holidays.

[0062] 4) User status parameters: Behavior: direction of travel, speed, whether to hesitate or wander.

[0063] Physiological (under compliant conditions): body temperature, heart rate (which can be initially determined by camera).

[0064] Social parameters: whether they are traveling together, number of people traveling together, and relationship with those traveling together (determined through interaction).

[0065] II. Multi-dimensional Intent Analysis Engine: This is the core processing layer, which transforms the collected data into an understanding of intent.

[0066] 1) Predictive analysis based on historical patterns: Logical reasoning: A user's past behavior is the best predictor of their future intentions.

[0067] Example: User A (Sales Director), historical records show that he comes to the company every Wednesday morning, and in 90% of cases he goes directly to the conference room on the 3rd floor.

[0068] Analysis: The probability that this visit (on Wednesday morning) was intended for a "business meeting" is extremely high.

[0069] Requirement: Prepare the designated meeting room in advance and send meeting information.

[0070] 2) Dynamic analysis based on real-time behavior: Logic: The user's current action trajectory and status reflect their immediate needs.

[0071] Example: User B lingered in front of a new product in the showroom for more than 2 minutes, and observed it while leaning forward.

[0072] Analysis: By comparing their preference information (preference for "technical exchange"), it can be inferred that they currently have a strong interest in this product.

[0073] Requirement: Automatically dispatch nearby tour guides to provide explanations.

[0074] 3) Context-based association analysis: Logic: By incorporating environmental factors, the intent analysis becomes more relevant to the actual scenario.

[0075] Example: Time context: Users entering the building during lunchtime (11:30-13:00).

[0076] Analysis: The likelihood of the intention being "to dine" or "to rest after a meal" has greatly increased.

[0077] Request: Recommend today's restaurant menu or available rest area.

[0078] 4) Collaborative analysis based on group characteristics: Logic: If users come in groups, their common needs as a group need to be considered.

[0079] Example: Sensor parameters: Detected a group of 5 people wearing matching sportswear.

[0080] Analysis: Based on the historical pattern of "team visits", the intention may be "group activities" or "team building".

[0081] Requirements: Recommend an activity room or team experience facility that can accommodate multiple people.

[0082] III. Demand Inference and Decision-Making Level: After analyzing the intent, it needs to be transformed into specific, executable service instructions.

[0083] Intent-demand mapping model (example): The analyzed intent is: business negotiation.

[0084] The inferred specific needs are: a need for a private and professional meeting space, which may require projection equipment, a whiteboard, and tea service.

[0085] System response / service action: 1. Navigate to the reserved / recommended meeting room.

[0086] 2. Ask if the equipment needs to be debugged.

[0087] 3. Send coupons for nearby cafes.

[0088] This application integrates a complete process of data analysis, behavioral modeling, and context awareness to achieve precise personalized services.

[0089] Furthermore, in this application, the step of analyzing the target object's intent and needs based on a multi-dimensional intent analysis model, combined with the sensing parameters, historical records, or preference information, specifically includes: analyzing current environmental factors based on the sensing parameters; analyzing the target object's future intent and immediate needs based on the historical records or preference information, combined with the target object's current action trajectory and state reflection; and using a collaborative analysis method based on group characteristics to analyze and obtain the target object's final intent and needs based on the current environmental factors, the future intent, and the immediate needs.

[0090] In this application, the collaborative analysis method based on group characteristics analyzes and obtains the final intent and needs of the target object as follows: 1) Input: Target object's identity information, real-time sensing parameters, historical records and preference information, current movement trajectory and status.

[0091] 2) Processing: The environmental analysis module outputs current environmental factors. The behavior analysis module outputs future intentions and immediate needs.

[0092] 3) Integration: The group feature collaborative analysis layer receives all the above information.

[0093] First, calibrate the influence of environmental factors on individual behavioral judgments (for example, walking unsteadily due to hot weather should not be misjudged as physical discomfort).

[0094] Then, using collaborative analysis methods, the final intent and needs of the target object are output.

[0095] 4) Output and Execution: Based on the final analysis results, the system drives the display window to present a personalized welcome screen, plays the corresponding welcome message through speech synthesis, and prepares to start relevant service resources.

[0096] This application upgrades the service model from "responsive service" to "predictive service" by predicting "future intentions". It can not only understand the visitor's "implied meaning" but also gain insight into their "unmet needs". Therefore, in the application scenarios of this application, it can provide a matching intelligent response whether facing familiar regular customers, unfamiliar individual visitors or complex visiting groups.

[0097] Step S40: Navigate the target object to the corresponding welcoming area according to the intent and needs, and recommend the specified activities or facilities to the target object.

[0098] Furthermore, in this application, the step of navigating the target object to the corresponding welcoming area according to the intent and needs, and recommending designated activities or facilities to the target object, specifically includes: navigating the target object to a reserved / recommended meeting room, or navigating the target object to the corresponding manager's office, according to the intent and needs; recommending designated activities or facilities to the target object, and providing an in-depth explanation of the target object.

[0099] In this application, the specific method for navigating and recommending designated activities or facilities based on the stated intent and needs is as follows: 1) Visit and study: Intent and needs: Guided tours, information access, and in-depth explanations are required.

[0100] 1. Ask if a guided tour is needed.

[0101] 2. Send out electronic brochures / technical white papers.

[0102] 2) Emergency matters: Intent and needs: Requires fast track, priority processing, and contact with a specific person in charge.

[0103] 1. Initiate the emergency response procedure.

[0104] 2. Navigate directly to the person in charge's office.

[0105] 3) Leisure and social activities: Intent and needs: A relaxed environment, catering services, and entertainment facilities are required.

[0106] 1. Navigate to the rest area or coffee shop.

[0107] 2. Recommend current featured drinks or activities.

[0108] 4) Exploring the Unknown: Intent and needs: Basic information guidance, overall introduction, and help with location are required.

[0109] 1. Play a welcome message and display a navigation map.

[0110] 2. Recommend the most popular areas or facilities.

[0111] The following is a summary of the implementation process of the above-mentioned humanoid robot intelligent interaction method based on the open-source HarmonyOS.

[0112] 1) Trigger: The sensor detects that the target object has entered the welcoming area.

[0113] 2) Identification: Verify the user's identity (User ID) through biometrics or credentials.

[0114] 3) Query: Retrieve user history and preference information from the database based on the user ID.

[0115] 4) Fusion: Input historical preferences and real-time sensing parameters (time, behavior, peers, etc.) into the intent analysis engine.

[0116] 5) Analysis: The engine outputs the most likely intent through rule models and machine learning algorithms.

[0117] 6) Inference: Based on the intent and the specific context, infer the specific needs.

[0118] 7) Execution: Trigger the corresponding service chain (navigation, recommendation, resource preparation) according to the needs.

[0119] In this invention, a human body sensor detects the entry status of a target object in real time. When the target object enters the welcoming area, its identity information and sensor parameters are acquired. Based on the identity information, the target object's identity is identified, and a corresponding welcoming message is played, providing a personalized welcoming experience. Furthermore, based on the identity information, the target object's historical records or preference information are obtained. Combining the sensor parameters, historical records, or preference information, the target object's intentions and needs are analyzed, accurately understanding the guest's needs and providing timely and precise service. Based on the intentions and needs, the target object is navigated to the corresponding welcoming area, and designated activities or facilities are recommended, providing more accurate decision support for the welcoming service. This invention improves the welcoming efficiency of humanoid robots and enhances their humanized interaction and continuous optimization capabilities.

[0120] Furthermore, such as Figure 4 As shown, based on the above-mentioned humanoid robot intelligent interaction method based on open-source HarmonyOS, the present invention also provides a humanoid robot intelligent interaction system based on open-source HarmonyOS, wherein the humanoid robot intelligent interaction system based on open-source HarmonyOS includes: Information and parameter acquisition module 21 is used to acquire the identity information and sensing parameters of the target object when the target object enters the welcoming area; The identity recognition module 22 is used to identify the identity of the target object based on the identity information, and play the corresponding welcoming words according to the identified identity; The intent and demand analysis module 23 is used to obtain the historical records or preference information of the target object based on the identity information, and analyze the intent and demand of the target object in combination with the sensing parameters, the historical records or the preference information; The welcome guidance module 24 is used to navigate the target object to the corresponding welcome area according to the intent and needs, and recommend the specified activities or facilities to the target object.

[0121] Furthermore, in this application, the system further includes: The wireless connection module is used to establish wireless communication connections with multiple sensors via the BLE communication protocol; wherein the multiple sensors include any combination of: temperature and humidity sensor, infrared sensor, LED light sensor, NFC sensor, blood oxygen and heart rate sensor, human body sensor and OLED sensor. The real-time sensing module is used to sense the entry status of the target object in real time through the human body sensing sensor.

[0122] Furthermore, in this application, the information and parameter acquisition module 21 specifically includes: A welcoming activation unit is used to go to the welcoming area to greet the target object when the target object is sensed to enter the welcoming area. The card information acquisition unit is used to acquire the facial image and human body video stream of the target object through its own camera, and to acquire the ID card information of the target object through the NFC sensor. An identity information acquisition unit is used to combine the ID card information, the facial image, and the human body video stream to obtain the identity information; The sensing parameter acquisition unit is used to acquire corresponding sensor data through temperature and humidity sensors, infrared sensors, LED light sensors, blood oxygen and heart rate sensors, and OLED sensors to obtain the sensing parameters.

[0123] Furthermore, in this application, the identity recognition module 22 specifically includes: An identity information analysis unit is used to analyze the gender and behavior type of the target object based on the identity information. The welcoming message playback unit is used to match the corresponding welcoming message from a preset welcoming database based on the gender and behavior type of the target object, and play the corresponding welcoming message in conjunction with the corresponding display window and welcoming screen.

[0124] Furthermore, in this application, the intent and requirements analysis module 23 specifically includes: The historical record acquisition unit is used to acquire the target object's historical access data, historical behavior data, and historical feedback data based on the identity information, thereby obtaining the target object's historical record. The preference information acquisition unit is used to acquire explicit preference data and implicit preference data of the target object based on the identity information, so as to obtain the preference information of the target object; The intent and demand analysis unit is used to analyze the intent and demand of the target object based on a multi-dimensional intent analysis model, combined with the sensing parameters, the historical records, or the preference information.

[0125] Furthermore, in this application, the intent and requirement analysis unit specifically includes: An environmental factor analysis subunit is used to analyze current environmental factors based on the sensor parameters. The real-time demand analysis subunit is used to analyze the historical records or preference information, and combine them with the target object's current action trajectory and state reflection to determine the target object's future intentions and real-time needs. The group characteristic analysis subunit is used to analyze and obtain the final intention and needs of the target object based on the current environmental factors, the future intention, and the immediate needs using a collaborative analysis method based on group characteristics.

[0126] Furthermore, in this application, the welcoming guidance module 24 specifically includes: Navigate the target object to the reserved / recommended meeting room or to the corresponding manager's office according to the stated intent and needs; Recommend specified activities or facilities to the target audience and provide in-depth explanations of the target audience.

[0127] Furthermore, such as Figure 5 As shown, based on the above-mentioned humanoid robot intelligent interaction method and system based on open-source HarmonyOS, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 5 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0128] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard drive or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a humanoid robot intelligent interaction program 40 based on the open-source HarmonyOS, which can be executed by the processor 10 to implement the humanoid robot intelligent interaction method based on the open-source HarmonyOS described in this application.

[0129] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the humanoid robot intelligent interaction method based on the open-source HarmonyOS.

[0130] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0131] In one embodiment, when the processor 10 executes the humanoid robot intelligent interaction program 40 based on the open-source HarmonyOS stored in the memory 20, the following steps are performed: When a target object is detected entering the welcoming area, the identity information and sensing parameters of the target object are obtained; The identity of the target object is identified based on the identity information, and a corresponding welcoming message is played according to the identified identity. Based on the identity information, obtain the target object's historical records or preference information, and analyze the target object's intent and needs by combining the sensing parameters, the historical records, or the preference information; Based on the stated intent and needs, the target object is navigated to the corresponding welcoming area, and designated activities or facilities are recommended to the target object.

[0132] Optionally, the step of acquiring the identity information and sensing parameters of the target object when it is sensed to enter the welcoming area includes: Wireless communication connections are established with multiple sensors via the BLE communication protocol; wherein the multiple sensors include any combination of temperature and humidity sensors, infrared sensors, LED light sensors, NFC sensors, blood oxygen and heart rate sensors, human body sensors, and OLED sensors. The human body sensor detects the entry status of the target object in real time.

[0133] Optionally, when a target object is sensed to enter the welcoming area, acquiring the target object's identity information and sensing parameters specifically includes: When the target object is detected entering the welcoming area, proceed to the welcoming area to greet the target object; The system acquires facial images and human body video streams of the target object through its own camera, and obtains the ID card information of the target object through the NFC sensor. The identity information is obtained by combining the ID card information, the facial image, and the human body video stream; The sensing parameters are obtained by acquiring corresponding sensor data through temperature and humidity sensors, infrared sensors, LED light sensors, blood oxygen and heart rate sensors, and OLED sensors.

[0134] Optionally, the step of identifying the target object's identity based on the identity information and playing a corresponding welcoming message based on the identified identity specifically includes: Analyze the gender and behavior type of the target object based on the identity information; Based on the gender and behavior type of the target object, the corresponding welcoming words are matched from the preset welcoming database and played in conjunction with the corresponding display window and welcoming screen.

[0135] Optionally, the step of obtaining the target object's historical records or preference information based on the identity information, and analyzing the target object's intent and needs in conjunction with the sensing parameters, the historical records, or the preference information, specifically includes: Based on the identity information, historical access data, historical behavior data, and historical feedback data of the target object are obtained to obtain the historical record of the target object; Based on the identity information, the explicit and implicit preference data of the target object are obtained to obtain the preference information of the target object; Based on a multi-dimensional intent analysis model, the intent and needs of the target object are analyzed by combining the sensing parameters, historical records, or preference information.

[0136] Optionally, the step of analyzing the target object's intent and needs based on a multi-dimensional intent analysis model, combined with the sensing parameters, historical records, or preference information, specifically includes: Analyze the current environmental factors based on the sensor parameters; Based on the analysis of the historical records or preference information, and combined with the target object's current action trajectory and status, the target object's future intentions and immediate needs are determined; Based on current environmental factors, the stated future intentions, and the stated immediate needs, a collaborative analysis method based on group characteristics is used to analyze and obtain the final intentions and needs of the target object.

[0137] Optionally, the step of navigating the target object to the corresponding welcoming area according to the intent and needs, and recommending specified activities or facilities to the target object, specifically includes: Navigate the target object to the reserved / recommended meeting room or to the corresponding manager's office according to the stated intent and needs; Recommend specified activities or facilities to the target audience and provide in-depth explanations of the target audience.

[0138] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a humanoid robot intelligent interaction program based on the open-source HarmonyOS, and the humanoid robot intelligent interaction program based on the open-source HarmonyOS, when executed by a processor, implements the steps of the humanoid robot intelligent interaction method based on the open-source HarmonyOS as described above.

[0139] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0140] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0141] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A humanoid robot intelligent interaction method based on open-source HarmonyOS, characterized in that, The humanoid robot intelligent interaction method based on open-source HarmonyOS includes: When a target object is detected entering the welcoming area, the identity information and sensing parameters of the target object are obtained; The identity of the target object is identified based on the identity information, and a corresponding welcoming message is played according to the identified identity. Based on the identity information, obtain the target object's historical records or preference information, and analyze the target object's intent and needs by combining the sensing parameters, the historical records, or the preference information; Based on the stated intent and needs, the target object is navigated to the corresponding welcoming area, and designated activities or facilities are recommended to the target object.

2. The humanoid robot intelligent interaction method based on open-source HarmonyOS according to claim 1, characterized in that, When a target object is sensed entering the welcoming area, the process of acquiring the target object's identity information and sensing parameters includes: Wireless communication connections are established with multiple sensors via the BLE communication protocol; wherein the multiple sensors include any combination of temperature and humidity sensors, infrared sensors, LED light sensors, NFC sensors, blood oxygen and heart rate sensors, human body sensors, and OLED sensors. The human body sensor detects the entry status of the target object in real time.

3. The humanoid robot intelligent interaction method based on open-source HarmonyOS according to claim 2, characterized in that, When a target object is sensed entering the welcoming area, the acquisition of the target object's identity information and sensing parameters specifically includes: When the target object is detected entering the welcoming area, proceed to the welcoming area to greet the target object; The system acquires facial images and human body video streams of the target object through its own camera, and obtains the ID card information of the target object through the NFC sensor. The identity information is obtained by combining the ID card information, the facial image, and the human body video stream; The sensing parameters are obtained by acquiring corresponding sensor data through temperature and humidity sensors, infrared sensors, LED light sensors, blood oxygen and heart rate sensors, and OLED sensors.

4. The humanoid robot intelligent interaction method based on open-source HarmonyOS according to claim 1, characterized in that, The step of identifying the target object's identity based on the identity information and playing a corresponding welcoming message based on the identified identity specifically includes: Analyze the gender and behavior type of the target object based on the identity information; Based on the gender and behavior type of the target object, the corresponding welcoming words are matched from the preset welcoming database and played in conjunction with the corresponding display window and welcoming screen.

5. The humanoid robot intelligent interaction method based on open-source HarmonyOS according to claim 1, characterized in that, The step of obtaining the target object's historical records or preference information based on the identity information, and analyzing the target object's intent and needs in conjunction with the sensing parameters, the historical records, or the preference information, specifically includes: Based on the identity information, historical access data, historical behavior data, and historical feedback data of the target object are obtained to obtain the historical record of the target object; Based on the identity information, the explicit and implicit preference data of the target object are obtained to obtain the preference information of the target object; Based on a multi-dimensional intent analysis model, the intent and needs of the target object are analyzed by combining the sensing parameters, historical records, or preference information.

6. The humanoid robot intelligent interaction method based on open-source HarmonyOS according to claim 5, characterized in that, The multi-dimensional intent analysis model, which combines the sensing parameters, historical records, or preference information to analyze the intent and needs of the target object, specifically includes: Analyze the current environmental factors based on the sensor parameters; Based on the analysis of the historical records or preference information, and combined with the target object's current action trajectory and status, the target object's future intentions and immediate needs are determined; Based on current environmental factors, the stated future intentions, and the stated immediate needs, a collaborative analysis method based on group characteristics is used to analyze and obtain the final intentions and needs of the target object.

7. The humanoid robot intelligent interaction method based on open-source HarmonyOS according to claim 1, characterized in that, The step of navigating the target object to the corresponding welcoming area according to the intent and needs, and recommending specified activities or facilities to the target object, specifically includes: Navigate the target object to the reserved / recommended meeting room or to the corresponding manager's office according to the stated intent and needs; Recommend specified activities or facilities to the target audience and provide in-depth explanations of the target audience.

8. A humanoid robot intelligent interaction system based on the open-source HarmonyOS, characterized in that, The humanoid robot intelligent interaction system based on the open-source HarmonyOS includes: The information and parameter acquisition module is used to acquire the identity information and sensing parameters of the target object when it senses that the target object has entered the welcoming area. The identity recognition module is used to identify the identity of the target object based on the identity information, and play the corresponding welcoming words according to the identified identity; The intent and demand analysis module is used to obtain the historical records or preference information of the target object based on the identity information, and to analyze the intent and demand of the target object in combination with the sensing parameters, the historical records or the preference information; The welcome guidance module is used to navigate the target object to the corresponding welcome area according to the intent and needs, and recommend specified activities or facilities to the target object.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a humanoid robot intelligent interaction program based on open source HarmonyOS stored in the memory and executable on the processor. When the humanoid robot intelligent interaction program based on open source HarmonyOS is executed by the processor, it implements the steps of the humanoid robot intelligent interaction method based on open source HarmonyOS as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a humanoid robot intelligent interaction program based on the open-source HarmonyOS. When the humanoid robot intelligent interaction program based on the open-source HarmonyOS is executed by a processor, it implements the steps of the humanoid robot intelligent interaction method based on the open-source HarmonyOS as described in any one of claims 1-7.

Images