A mobile robot active service method and a mobile robot

Abstract

The application relates to the technical field of robots, in particular to a mobile robot active service method and a mobile robot. The method comprises the following steps: determining a target active service to be executed by the robot currently based on a configured service time axis; controlling the robot to reach a target service location of the target active service; executing a corresponding pre-response action according to a service type of the target active service; determining whether to start the target active service according to a preset service starting rule; generating at least one active response action according to obtained interaction scene information and target service content of the target active service during execution of the target active service; and adjusting a component pose and a display state of the robot according to the at least one active response action. The application can effectively solve the problems of low utilization rate and poor user experience of the robot in the prior art.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a mobile robot proactive service method and a mobile robot. Background Technology

[0002] Existing technologies result in low levels of intelligence and limited service types for service robots, leading to low usage frequency, inability to meet the needs of diverse scenarios, poor user experience, and failure to effectively fulfill the daily auxiliary functions of intelligent service robots in home settings. Summary of the Invention

[0003] In view of this, the embodiments of this application provide a mobile robot proactive service method and a mobile robot, which can effectively solve the problems of low robot utilization and poor user experience in the prior art.

[0004] In a first aspect, embodiments of this application provide a mobile robot proactive service method, the method comprising:

[0005] The target proactive service that the robot needs to perform is determined based on the configured service timeline.

[0006] Control the robot to reach the target service location where the target service is actively provided;

[0007] Execute the corresponding pre-response action based on the service type of the target proactive service;

[0008] Determine whether to start the target active service based on the preset service startup rules;

[0009] If the target proactive service is initiated, at least one proactive response action is generated based on the acquired interaction context information and the target service content of the target proactive service.

[0010] The robot's component poses and display status are adjusted according to at least one of the active response actions.

[0011] Secondly, embodiments of this application provide a mobile robot, which includes a service agent and a touch display screen. The mobile robot provides proactive services using a mobile robot proactive service method provided in the first aspect of this application.

[0012] The embodiments of this application have the following beneficial effects:

[0013] This application determines the target proactive service to be executed by the robot based on a configured service timeline; controls the robot to reach the target service location; executes corresponding pre-response actions according to the service type of the target proactive service; determines whether to start the target proactive service according to preset service initiation rules; if the target proactive service is started, generates at least one proactive response action based on the acquired interaction scenario information and the target service content; and adjusts the robot's component poses and display state according to at least one proactive response action. This application proactively starts services based on the service timeline, improving robot utilization. Furthermore, this application includes pre-response actions and proactive response actions, allowing for the output of services in different scenarios and adaptive adjustment of response actions based on the interaction scenario, thus improving intelligence and effectively solving problems such as low robot utilization and poor user experience in existing technologies. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A schematic diagram of the structure of a robot according to an embodiment of this application is shown;

[0016] Figure 2 A flowchart of a mobile robot proactive service method according to an embodiment of this application is shown;

[0017] Figure 3 A flowchart illustrating the execution of a pre-response action in a mobile robot proactive service method according to an embodiment of this application is shown.

[0018] Figure 4 This paper illustrates a flowchart of the target active service determination process in the mobile robot active service method according to an embodiment of this application.

[0019] Figure 5 A schematic diagram of a mobile robot active service device according to an embodiment of this application is shown.

[0020] Explanation of key component symbols:

[0021] 110 - Robot mobile chassis; 120 - First support rod; 130 - Second support rod; 140 - Touch screen; 150 - Camera; 310 - Target service determination module; 320 - Motion control module; 330 - Pre-response module; 340 - Start-up judgment module; 350 - Active response module; 360 - Execution module. Detailed Implementation

[0022] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0023] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0024] In the following text, the terms "comprising," "having," and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more combinations thereof. Furthermore, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0025] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0026] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] This application first provides a mobile robot, exemplary in that the mobile robot employs the mobile robot proactive service method of the embodiments of this application to provide proactive services. The mobile robot includes a service agent and a touch display screen. Specifically, the mobile robot proactive service method of the embodiments of this application is implemented through the service agent to control the mobile robot to provide proactive services.

[0028] This mobile robot includes, but is not limited to, home service robots, and can be applied in professional service scenarios such as elderly care institutions and hospitals.

[0029] The mobile robot also includes a robot chassis 110, a first support rod 120, a second support rod 130, a touch screen 140 (hereinafter referred to as the screen), and a camera 150. One end of the first support rod 120 is fixedly connected to the robot chassis 110. The other end of the first support rod 120 is rotatably connected to one end of the second support rod 130. The other end of the second support rod 130 is connected to a rotatable mounting plate of the touch screen 140. The touch screen 140 can be rotated in three degrees of freedom by a motor, such as... Figure 1 As shown (X, Y, and Z in the figure represent the directions of rotation around the X-axis, Y-axis, and Z-axis).

[0030] As an example, the service agent includes a large language model and a visual-language model. The large language model is used to understand interaction information with the user and obtain user needs. The visual-language model is used to understand user intent and obtain user needs by acquiring user body language, eye contact, etc.

[0031] The main ideas of this application include: constructing a personalized service model for proactive services in home scenarios; understanding proactive services through a service agent based on multi-level scenarios; and providing different dynamic responses to different interaction scenarios during the service process. As a result, the robot can proactively provide personalized intelligent services to different users in the home at different times and locations, improving utilization and user experience.

[0032] The following describes the mobile robot's proactive service method using specific embodiments.

[0033] Figure 2 A flowchart of a mobile robot proactive service method according to an embodiment of this application is shown. Exemplarily, the mobile robot proactive service method includes the following steps:

[0034] S100 determines the target proactive service that the robot needs to perform based on the configured service timeline.

[0035] Exemplary, embodiments of this application include a personalized service model. This personalized service model includes a service timeline. The service timeline is the robot's daily service execution model, encompassing both preset and predicted services, and represents the robot's work schedule. In other words, the service timeline includes multiple proactive service items arranged according to time. The service timeline creates a "schedule" for the robot's various service arrangements. Proactive services are services automatically provided by the robot to at least one user based on service information. For example, the robot might provide proactive services to household users based on the service timeline's information on daily life. During the daily schedule, the service agent will plan the recipients, locations, times, and content of the services provided based on the services on the service timeline. Simultaneously, other time rules included in the service time information will also be used by the service agent to determine whether to activate the service at that time on that day; for example, the "outfit suggestion" service might not be executed on weekends because users only need that service on weekdays.

[0036] In one implementation, to improve the robot's intelligence level, step S100 involves determining the target proactive service that the robot currently needs to perform based on the configured service timeline, including:

[0037] S110 determines the target proactive service to be executed based on the preset service priority, service timeline, and current time period.

[0038] The service types on the service timeline include preset services and predicted services. Preset services are those manually set by the user, while predicted services are those automatically generated by the robot based on service usage data.

[0039] Each service's service information (information included in proactive services) includes the service topic, service content, target user (who will use it), service time (when it starts, when it ends, and the specific date), service location (where it is), and service type (predictive service or preset service). The service time includes the start time (earliest start time), end time, and specific date. In this application's embodiments, the service location includes multiple alternative options.

[0040] In addition, the prediction service also includes a priority confidence score (between 0 and 1). The higher the priority confidence score, the higher the service priority. When the priority confidence score is equal to 1, its service priority is the same as the preset service.

[0041] The prediction service also includes a service scenario description, which describes the "user's current activity" information.

[0042] As an example, service hours can be specified not only by the exact time, but also by dates such as "Monday", "Weekday", "Birthday", "Valentine's Day".

[0043] Furthermore, service priority includes the priority of preset services over predicted services. Understandably, the service types of the target proactive service also include preset services and predicted services.

[0044] In step S110, the target proactive service to be executed is determined based on the preset service priority, service timeline, and current time period, including:

[0045] S111: Determine the target service node to be executed on the service timeline based on the current time period. If only the prediction service exists at the target service node, execute the prediction service; otherwise, execute the preset service.

[0046] Understandably, if both the predictive service and the preset service exist at the target service node, the preset service will be executed. If only the predictive service exists at the target service node, the predictive service will be executed. If only the preset service exists at the target service node, the preset service will be executed.

[0047] Furthermore, in order to improve the robot's initiative, the method in this application embodiment also includes: a method for generating preset services and predicted services.

[0048] The method for generating preset services includes: receiving preset service information configured by the user, obtaining the preset service, and adding the preset service to the service timeline; the preset service information includes service topic, service content, user identifier (identifying the target user based on the user identifier), service time, service location, and service type (preset service). User-customized services will be added to the service timeline as preset services, and there is no overlap between preset services on the service timeline.

[0049] The method in this application embodiment further includes: when adding a preset service to the service timeline, detecting whether the service time of the preset service overlaps with the service time of other preset services; if they overlap, outputting conflict information. In other words, overlapping service times of two preset services are not allowed in the service timeline, and the robot will automatically check and alert for conflicts when saving services.

[0050] The method for generating predictive services includes: based on a preset update cycle, continuously learning from the acquired service usage data using a large language model to generate predictive services and add them to the service timeline.

[0051] As an example, based on a preset update cycle, service usage data within a preset time period and the current service timeline are input into a large language model for inference (determining whether new data has been added or whether the latest information has been updated) to obtain predicted services.

[0052] Predicted services are services learned by the robot from historical service usage data and automatically added to the service timeline. The service time of a predicted service may overlap with the service times of other services. If the service location is temporarily uncertain, the robot also allows predicted services to overlap with other services. The service time (start time) is typically a time period (e.g., "9-10 AM"), and multiple candidate locations may be provided for the service location. The service content of the predicted service is the robot's inferred demand, and the service content is further refined based on the interaction scenario when actively providing the target service.

[0053] Since prediction services can overlap, this embodiment of the application also sets service priorities for prediction services to facilitate the determination of the target proactive service to be executed. Exemplarily, the prediction service in this embodiment includes a priority confidence level (range 0-1), and the target proactive service is determined based on the priority confidence levels of multiple prediction services within the same (overlapping) time period. That is, the higher the priority confidence level, the higher the service priority, and the corresponding prediction service is the target proactive service.

[0054] As an example, the preset update cycle is triggered at the end of each service session or at a fixed interval (e.g., every 3 hours / day). The specific timing depends on the implementation, and this application embodiment does not impose any limitations.

[0055] Recorded service usage data is input into the service agent to obtain predicted services. Specifically, the input includes the latest service usage data (who, when, and what was done) and the current service timeline. The service agent infers the predicted service based on this input data, obtaining the predicted service fields. If the predicted service is already on the service timeline, it is updated. Otherwise, it is added to the service timeline. If the inferred predicted service is already on the timeline, the time period, priority confidence, predicted service content, and predicted service itself are updated to the latest version; otherwise, a new entry is added.

[0056] In this embodiment, both the preset service and the predicted service are stored in the service timeline using JSON data. For example, the JSON data written to the service timeline by the service agent when a predicted service is generated includes: service unique identifier, user identifier, service topic, service content, user identifier, service time, priority confidence, service location, update time, activity description, and service type. Understandably, the JSON data of the preset service includes: service unique identifier, user identifier, service topic, service content, user identifier, service time, service location, update time, activity description, and service type. Exemplarily, the predicted service in JSON structure includes the following fields:

[0057] {

[0058] "Service ID":"Pred_2025-05-07-0900-Mom-Outfit",

[0059] ServiceType: "Pred"

[0060] User ID: "UID_Mom"

[0061] "ThemeLevel1": "Personalized Recommendations"

[0062] "ThemeLevel2": "Outfit Suggestions"

[0063] "Service TimeSlot": {

[0064] "start": "09:00",

[0065] "end": "10:00",

[0066] "rule": "weekday"

[0067] },

[0068] "Activity Description (UserActivity): "Selecting and matching clothes in the wardrobe".

[0069] "Location Options": ["Walk-in Closet", "Master Bedroom"],

[0070] Priority Confidence: 0.87

[0071] / * --- Service agent-driven service process description--- * /

[0072] "InteractionSpec": {

[0073] ServiceFlow: [

[0074] "① Greet the user and briefly confirm the occasion and dress code for today's outing."

[0075] "② Access the wardrobe database and today's weather data to filter clothing suitable for the occasion (color scheme, style, length, etc. should take into account the user's long-term preferences)."

[0076] "③ Generate 2-3 complete outfits, render the try-on effect on the screen in real time, and highlight key features."

[0077] "④ Retrieve historical images of the same or similar items from the three most recent similar outfit records and present them to the user in a side-by-side comparison."

[0078] "⑤ Invite users to select or propose modifications; after the user selects, record the confirmation result and end the service."

[0079] "ContextSources":["WardrobeDB","WeatherAPI","PastSessionLog"]

[0080] },

[0081] LastUpdated: 2025-05-06T15:32:45Z

[0082] }

[0083] Regarding the explanations of the above fields, `InteractionSpec.ServiceFlow` is used to describe the entire service flow in natural language steps, which the service agent can directly refer to during execution; no fixed template is required, emphasizing a "human-like" service. `InteractionSpec.ContextSources` is used to list the data sources that need to be queried during service execution, helping the scheduler to fetch or cache them in advance. `UserActivity`: A real-time activity description inferred by the robot through vision / speech / sensor fusion, which can directly help the service agent determine whether the predicted service should be executed immediately or adjusted.

[0084] During the continuous updating of the prediction service, if the service agent generates a prediction service for the same user, the same topic, and with overlapping time periods, it only needs to update the Confidence (priority confidence), TimeSlot (service time), or LocationOptions (service location options) fields and refresh LastUpdated (update date).

[0085] Furthermore, the method in this application embodiment also includes: a method for recording service usage data.

[0086] During the proactive service delivery process, basic service information, service topics, and service content logs are recorded to obtain service usage data.

[0087] Basic service information includes a unique service identifier, user identifier, service time (service start time, service end time), service location, and service type; service topics include multi-level topics (e.g., second-level topics); service content logs include multimodal interaction logs, robot response logs, environmental parameters, third-party software usage information, and exception information, as detailed in Table 1:

[0088] Table 1 Service Usage Data

[0089]

[0090] Furthermore, the extraction of service topics and the understanding of service content are achieved by a service agent driven by a large language model. Exemplarily, the above-mentioned methods for generating multi-level service topics include:

[0091] (a) Merge the multimodal interaction logs and robot response logs into a plain text sequence in chronological order, and perform text cleaning (removing stop words, UI component identifiers, etc.). For example, merge UserInputLog and SystemResponseLog into a plain text sequence in chronological order, removing stop words, UI component identifiers, and other irrelevant symbols. Specifically, remove UI identifiers such as id="submit". Remove stop words such as "the", "for", "on", etc. (customizable according to actual needs).

[0092] (b) The vector or cleaned plain text sequence is used as context concatenation in the specified prompt words (input to the large language model). The vector is obtained by pre-processing the plain text sequence. For example, when the plain text sequence is too long, embedding is used. The same pre-trained large language model as the service agent is used to perform embedding on the service robot side to obtain the vector.

[0093] (c) Based on the concatenated prompt, select the primary and secondary themes from the preset theme categories and store them in the corresponding fields of the service usage data. For example, concatenate a vector or corresponding plain text sequence as context into the Prompt: "Based on the following service robot interaction log content, please return the most suitable service theme label in the 'primary-secondary' format, limited to the preset theme category 'taxonomy'." This ultimately generates a two-level service theme in JSON format. For example, the robot can record various theme combinations in the service usage data, such as {"ThemeLevel1":"Health Management","ThemeLevel2":"Morning Hydration Reminder"}, {"ThemeLevel1":"Learning Supervision","ThemeLevel2":"Learning Companionship and Supervision in Cooperation with Parents"}, {"ThemeLevel1":"Life Care","ThemeLevel2":"Medication Reminder"}, etc. Through these diverse records, the robot's ability to provide differentiated proactive services to different family members at different times and locations can be reflected, and these records are stored in the corresponding fields of the service usage data.

[0094] S200 controls the robot to reach the target service location for proactive service.

[0095] Before performing an active service, the robot needs to be moved to the target service location. The target active service includes multiple alternative target service locations. If the first target service location reached does not meet the service initiation rules, the robot is moved to the next alternative target service location, and the service initiation rules are re-evaluated.

[0096] Further, in step S200, controlling the robot to reach the target service location for the target proactive service includes:

[0097] Determine the destination service location based on a pre-built map and visual features of the current environment.

[0098] Each time a proactive service is performed, the robot identifies the current location, as follows:

[0099] Before the robot begins its service, users can use a map-building tool to have the robot scan the environment and generate a map, allowing users to divide the map into regions and add semantic labels.

[0100] Real-time localization based on the constructed map involves the robot using the built map and acquired visual SLAM to match the current location, obtain the location ID, and retrieve its semantic tags. The semantic tag system includes main tags and attribute tags. Main tags include fixed functional areas such as kitchen, living room, bedroom, and balcony. Attribute tags include fine-grained descriptions that can be supplemented by the user or the system, such as "the area where the son does his homework," "an area where pets are not allowed," "the area where Grandma sunbathes after breakfast every day," "the area where Mom does yoga," "the area where the daughter does crafts," and "the area where Dad practices guitar."

[0101] The method in this application embodiment also includes dynamic semantic completion of tags and updating tags. Specifically, when the location lacks semantic tags or the attributes are incomplete, the robot triggers the learning of tags and completes the semantic tags in the process of "continuously learning user service preferences".

[0102] S300: Execute the corresponding pre-response action based on the service type of the target proactive service.

[0103] In this embodiment, the dynamic response output by the robot includes two types of responses: pre-service and in-service. Specifically, this embodiment provides pre-service response actions. Pre-service response actions are actions executed before service initiation. To better provide the service, pre-service response actions are defined as actions that initially adjust the state parameters of robot components, preparing for proactive response actions. This embodiment provides different pre-service response actions for different service types. During the service, multiple proactive response actions are provided. Proactive response actions are actions that adaptively adjust the output based on the interaction scenario during the current service.

[0104] In one implementation, such as Figure 3 As shown, the service types of the target proactive service include preset services and predicted services. To improve user experience, it is understandable that corresponding pre-response actions are executed based on the service type of the target proactive service, including:

[0105] S310, if the service type is a preset service, then enter the pre-start response mode; in the pre-start response mode, adjust the robot's position, robot orientation, and the orientation of the touch display screen 140 according to the acquired user service preferences, in order to wait for the preset service to start. In other words, the service intelligence will adjust the robot's position and orientation, screen orientation, etc., according to the user service preferences, in order to wait for the service to start.

[0106] S320, if the service type is predictive service, then enter suggestion response mode. In suggestion response mode, the robot's position, robot orientation, and the orientation of the touch display screen 140 are adjusted according to the acquired user service preferences. Personalized service content is generated and displayed on the screen based on the service type and user service preferences to guide the user to initiate the service. In other words, the service agent adjusts the robot's position and orientation, and the screen orientation, according to the user's service preferences. The service agent also generates personalized on-screen suggestion content based on the service type and user service preferences to guide the user to initiate the service. Personalized service content includes the text style and interface style of the suggestion content.

[0107] Understandably, in order to improve intelligence, the embodiments of this application also include:

[0108] User service preferences are learned from the acquired service usage data through a large language model; user service preferences include service content preferences and preferences for interaction methods with the robot.

[0109] Furthermore, user service preferences include service content preferences and preferences for interaction methods with the robot; the target proactive service includes service topics.

[0110] User service preferences are learned from the acquired service usage data using a large language model, including:

[0111] (a) Among the multiple target proactive services that have been executed, the frequency of service topics is counted to obtain the target proactive service with the highest frequency; based on the service content logs in the target proactive service with the highest frequency, a service summary is generated and saved to obtain the service content preference; wherein, the service summary includes: user ID, secondary service topic tag, summary content, trust value, and update date.

[0112] (b) Generate interaction preference based on the frequency of use of interaction methods in the same interaction scenario; the fields of interaction preference include: user identifier, interaction scenario label, interaction modality weight (voice, touch, gesture), comfort distance, robot orientation, robot screen tilt angle, and update date.

[0113] The robot continuously learns the user's preferences for using the robot's services from the user's service usage data, including specific service content preferences in specific services and preferences for the robot's interaction methods (interaction modalities, the position and orientation of the robot, usage distance, screen orientation). Different from inferring predictive services from service usage data, the user service preferences are stored separately. The long-term preferences of the user for service content and interaction methods are independently saved for the service agent to read at any time. The user service preferences and service usage data are associated through fields such as the user identifier UserID and the secondary theme ThemeLevel2. The service content preferences are recorded in text form. The service agent summarizes the service content logs of the same theme in the past n times and outputs natural language memory. For example, <preference identifier PrefID, user identifier UserID, secondary theme ThemeLevel2, service summary TextSummary, confidence Confidence, latest update date LastUpdated>. Example, <PREF_1024, UID_Mom, dressing suggestions, "prefer warm colors + simple workplace style; exclude high heels", 0.92, 2025-05-06>.

[0114] Demonstratively, the interaction method preferences store data in Json format. The interaction method preferences include fields such as user identifier, interaction scenario label, interaction modality weight assignment, comfortable distance, robot orientation, screen tilt angle, update date, etc. For example, the fields of the interaction method preferences are as follows:

[0115] json{

[0116] "UserID": "UID_Mom",

[0117] "ContextTag": "watching movies on the sofa", / / interaction scenario label

[0118] "ModalityRatio": {"speech": 0.8, "touch": 0.1, "gesture": 0.1},

[0119] "ComfortDistance_m": 2.0,

[0120] "RobotFacing": "the screen faces the user's left side",

[0121] "ScreenTilt_deg": -10,

[0122] "Updated": "2025 05 06"}

[0123] User service preferences and prediction services are updated together, for example, after a service session ends, or in batches at regular intervals each day. The service agent reads the latest multimodal interaction logs and secondary topic tags within the secondary topics (such as "user is lying on the sofa watching TV") from the service usage data, summarizes or statistically analyzes service content preferences and interaction parameters (corresponding to interaction method preferences), and writes them into the corresponding interaction scenario tag ContextTag.

[0124] S400 determines whether to start the target active service based on the preset service startup rules.

[0125] Proactive services include pre-set services and predictive services.

[0126] Understandably, to improve user experience, the method in this application embodiment further includes: configuring service startup rules for proactive services, specifically including: configuring the startup time of the proactive service to start a specified proactive service at a preset time, target user, target service location, and preset location, and adding the proactive service to the service timeline. Exemplarily, the service startup rule includes: starting a specified proactive service at a preset time and preset location, and adding the proactive service to the service timeline. Users manually define service startup rules. This application embodiment allows users to customize the robot's service startup rules, that is, the robot's service can be customized by the user, automatically starting a specified service at a preset time and preset location.

[0127] In one implementation, the service initiation rule is constructed based on the identified current scenario, current user, and target user and target service location included in the target proactive service. Understandably, step S300, determining whether to initiate the target proactive service according to the preset service initiation rule, includes:

[0128] Determine whether to initiate the target proactive service based on the identified current scenario, current user, target user, and target service location.

[0129] This application embodiment utilizes a large language model and a vision-language model to understand the scene and determine whether to execute the service. The robot will travel to the service location in advance and enter a pre-start response mode according to the service information customized by the user, and the service intelligence will continuously identify and understand the current scene.

[0130] In one implementation, such as Figure 4 As shown, based on the identified current scenario, current user, target user, and target service location, it is determined whether to initiate the target proactive service, including:

[0131] S410, if the robot identifies the current scene location as the target service location and the current user is the target user, then it proactively initiates a service request to the target user. Understandably, scene understanding in this embodiment includes: identifying whether the target user is at the target service location; if the current scene meets the service initiation requirements (target user at the service location), then the service is initiated. If no target user is detected at the current location, then the service initiation conditions are not met.

[0132] In another implementation, if the current user is the target user and it is detected that the target user's time period at the target service location is longer than a preset stay duration, then the target active service is initiated.

[0133] For example, for a preset service, if the robot's current location is the target service location and the current user is the target user, and it is confirmed that the target user is continuously active in the space where the target service location is located (the target user can be detected for a period of time), then the robot will actively initiate a service request to the target user; otherwise, it will wait until the conditions for ending the wait are met.

[0134] S420: If the current scene location does not match the target service location, and there are other active services besides the target active service in the current time period, then control the robot to go to the target service location of the other active service. For predicted services, if the current scene does not meet the conditions and there are other active services in the current time period, then go to the next target service location according to the next active service on the service timeline.

[0135] S430, if the current scenario does not match the target service location, and the current user does not match the target user, then wait until the conditions for ending the wait are met. The conditions for ending the wait include meeting the requirements for starting the target active service, being requested by the user to perform other services, and having other preset services that need to be planned on the service timeline.

[0136] For prediction services, if neither the current scenario nor the current user meets the conditions, the service will wait until the conditions for ending the wait are met.

[0137] Furthermore, the method in this application embodiment also provides a target user identification method, specifically including: confirming the identity of the target user to be served based on at least one of a first confidence level, a second confidence level, and a third confidence level. The first confidence level is obtained by performing facial recognition on the detected current user; the second confidence level is obtained by performing voiceprint recognition on the current user; and the third confidence level is obtained by verifying the terminal device carried by the current user based on near-field communication.

[0138] Each time a target proactive service is performed, the robot identifies the identity of the currently interacting user. In this embodiment, the user identification method employs "multimodal step-by-step confirmation + service timeline prior + low-confidence interaction registration." Multimodal step-by-step confirmation includes multiple levels of confirmation such as facial recognition, voiceprint compensation, and near-field verification. Exemplarily, the identity of the target user to be served is confirmed based on at least one of a first confidence level, a second confidence level, and a third confidence level, including:

[0139] (a) Face recognition: The robot first uses the camera 150 to acquire the face image of the user and compares the faces to obtain the first confidence level.

[0140] (b) Voiceprint recognition: If the first confidence level is lower than the set threshold, voiceprint recognition is performed on the voice segments of the synchronously received interactive users to obtain the second confidence level. Among them, face recognition and voiceprint recognition can be performed locally or in the cloud according to deployment requirements.

[0141] (c) Near-field verification: Verifying the identity of the user by using near-field communication to verify the information of the terminal device carried by the user. For example, when both the first and second confidence levels are below a set threshold, the robot detects the Wi-Fi signal of the user's mobile phone, wristband, or other terminal devices. The third confidence level can be obtained by using Fi / BLE near-field communication characteristics; or by using the recognition results of a home IoT camera.

[0142] (d) Multi-source fusion: The first confidence level, the second confidence level, the third confidence level and the target user prior corresponding to the "preset service" or "predicted service" in the service time axis are input into the fusion model, and the final identity and fusion confidence level are output.

[0143] (e) Interactive Confirmation and Registration: If the fusion confidence level is still below the set threshold, the robot will output prompts via voice / screen to ask the user to confirm their identity. For example, for registered users, the user's corresponding ID can be selected from the robot's screen, and the robot will confirm the service recipient. For unregistered users, the robot will guide them to complete the registration of their name, family identity, face, and voiceprint, or select visitor mode (providing only basic services and not recording service data).

[0144] If the current service has multiple target users, the same method as above should be used for confirmation. If user information cannot be confirmed during the execution of the current service, service usage data will not be recorded.

[0145] S500, if the target proactive service is initiated, at least one proactive response action is generated based on the acquired interaction context information and the target service content of the target proactive service. In other words, during the execution of the target proactive service, multiple proactive response actions are generated based on the acquired interaction context information and the target service content of the target proactive service.

[0146] The interaction context information includes user input request information, user gesture information recognized by the service agent, and intent information. In this embodiment, different proactive response actions are provided based on different interaction context information.

[0147] During the execution of the target proactive service, that is, during the interaction with the user, the service agent (including large language model and vision-language model) identifies the robot's current scene, generates interaction requirements, plans dynamic response interactions, and generates interaction content (multiple main response actions).

[0148] Exemplary examples show that the main response action includes, but is not limited to, screen rotation control, which is part of the dynamic response and is controlled by interactive content generated by the service agent.

[0149] In one implementation, if a target proactive service is initiated, a proactive response action is generated based on the acquired interaction context information and the target service content of the target proactive service, including:

[0150] (a) Input the multimodal perception information included in the acquired interaction context information into the large language model for reasoning to obtain the interaction requirements included in the target service content, and output active response actions based on the interaction requirements; the multimodal perception information includes: natural language information input by the user, visual information output by the user, and acquired external device data.

[0151] For example, the robot will simultaneously receive user's natural language dialogue, visual images (people, objects, gestures), and IoT / external device data (such as human body sensors, temperature and humidity sensors, light sensors, door and window magnetic sensors, wardrobe door magnetic sensors, smart refrigerators, smart rice cookers, smart light bulbs, wearable devices, weather APIs, etc.) as multimodal perception information. The service agent reads the above multimodal perception information, converts it into a unified text prompt, and then infers and summarizes "what the current user wants" and "what robot actions are needed".

[0152] (b) Generate proactive response actions based on current interaction needs, user service preferences, scene context, and current perception information. Proactive response actions include adjusting the robot's position, adjusting the displayed UI style and content, and outputting voice prompts; the scene context includes real-time activity tags and environmental parameters. The real-time activity tags are inferred by the service agent based on the detected actions of the current user.

[0153] As an example, the first step is to obtain the user's current interaction needs. This involves the service agent extracting immediate requests from the service content logs, such as "Help me put together my commute outfit for today."

[0154] Secondly, user service preferences are obtained. Relevant user service preference entries are read from the user service preference memory based on user identifiers and interaction scenario tags, such as "likes warm colors and minimalist style" or "prefers list-style browsing".

[0155] Next, the scene context is obtained. This includes obtaining real-time activity tags (such as "picking clothes in the cloakroom") and environmental parameters (weather, schedule), etc.

[0156] Finally, output the active response action.

[0157] For example, the robot detects that a mother is standing in the dressing room and says via voice, "Help me pick out an outfit for today's office."

[0158] The service agent determines the need as "commuting outfit suggestions", taking into account user preferences (warm colors, minimalist) and weather (cloudy, 18℃).

[0159] The service agent plans proactive response actions. Proactive response actions include:

[0160] Controlling the robot's movement: 1.5 meters directly facing the wardrobe; screen angle: -15° downwards;

[0161] UI: Generate a "list view" that displays 3 complete outfits, each with a small thumbnail and key tags; provide a "click to view details" interaction instead of card swiping.

[0162] Voice prompts provide brief explanations and hints that can be viewed via touch.

[0163] Furthermore, the method also includes:

[0164] If the robot is performing a target proactive service, it will not proactively plan and perform other services outside of the target proactive service.

[0165] Furthermore, in order not to affect the provision of proactive services, the method in this application embodiment determines the robot's charging time according to the service timeline to ensure that it does not conflict with the proactive services on the service timeline.

[0166] S600, adjust the robot's component pose and display state according to at least one active response action. Control the robot to execute the aforementioned active response action.

[0167] This application's robot can continuously optimize the service timeline and user service preferences based on user-configured service content and service usage data learned during interaction. This allows it to proactively switch service locations, continuously providing personalized services to different users, significantly increasing usage frequency and overall service efficiency. The robot can autonomously plan and execute service tasks, enhancing its intelligence and optimizing user experience. In particular, based on user service preferences, the robot can interact with screen content through its own movement and screen rotation, delivering a rich, integrated hardware and software interactive experience. This application's service agent can plan and guide the interaction process based on communication with users and understanding of the scenario, making application types, content, and interaction methods diverse, personalized, and highly flexible.

[0168] Figure 5 A schematic diagram of a mobile robot proactive service device according to an embodiment of this application is shown. Exemplarily, the mobile robot proactive service device includes: a target service determination module 310, a movement control module 320, a pre-response module 330, a start-up judgment module 340, a proactive response module 350, and an execution module 360.

[0169] The target service determination module 310 is used to determine the target proactive service that the robot is currently to execute based on the configured service timeline.

[0170] The mobile control module 320 is used to control the robot to reach the target service location for proactive service.

[0171] The pre-response module 330 is used to execute corresponding pre-response actions based on the service type of the target proactive service;

[0172] The startup judgment module 340 is used to determine whether to start the target active service based on the preset service startup rules.

[0173] The proactive response module 350 is used to generate at least one proactive response action based on the acquired interaction context information and the target service content of the target proactive service if the target proactive service is initiated.

[0174] The execution module 360 ​​is used to adjust the component poses and display status of the robot based on at least one active response action.

[0175] It is understood that the device in this embodiment corresponds to the mobile robot proactive service method in the above embodiments, and the options in the above embodiments are also applicable to this embodiment, so they will not be described again here.

[0176] This application also provides a terminal device, exemplary of which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to enable the terminal device to perform the functions of the various modules in the above-described mobile robot active service method or the above-described mobile robot active service device.

[0177] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0178] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving execution instructions.

[0179] This application also provides a computer-readable storage medium for storing the computer program used in the aforementioned terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0181] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0182] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0183] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A mobile robot proactive service method, characterized in that, The method includes: The target proactive service that the robot needs to perform is determined based on the configured service timeline. Control the robot to reach the target service location where the target service is actively provided; Execute the corresponding pre-response action based on the service type of the target proactive service; Determine whether to start the target active service based on the preset service startup rules; If the target proactive service is initiated, at least one proactive response action is generated based on the acquired interaction context information and the target service content of the target proactive service. The robot's component poses and display status are adjusted according to at least one of the active response actions; The service startup rule is constructed based on the identified current scenario, current user, and target user and target service location included in the target proactive service; determining whether to start the target proactive service according to the preset service startup rule includes: Determining whether to activate the target proactive service based on the current scenario, the current user, the target user, and the target service location specifically includes: if the current user is the target user, and the time period of the target user at the target service location is detected to be longer than a preset stay duration, then the target proactive service is activated; if the location of the current scenario does not match the target service location, and there are other proactive services besides the target proactive service in the current time period, then the robot is controlled to go to the target service location of the other proactive service; if the current scenario does not match the target service location, and the current user does not match the target user, then waiting is performed until the end-waiting conditions are met; the end-waiting conditions include meeting the requirements for activating the target proactive service, being requested by the user to perform other services, and having other preset services that need to be planned on the service timeline.

2. The mobile robot proactive service method according to claim 1, characterized in that, The configuration-based service timeline determines the target proactive service that the robot currently needs to execute, including: The target proactive service to be executed is determined based on the preset service priority, service timeline, and current time period.

3. The mobile robot proactive service method according to claim 2, characterized in that, The target proactive service includes preset services and predicted services; the service priority includes the preset service having a higher priority than the predicted service. The process of determining the target proactive service to be executed based on preset service priorities, service timelines, and the current time period includes: The target service node to be executed on the service timeline is determined based on the current time period. If only the predicted service exists at the target service node, the predicted service is executed; otherwise, the preset service is executed.

4. The mobile robot proactive service method according to claim 1, characterized in that, The target proactive service includes preset services and predicted services; the robot is equipped with a touch display screen. The step of executing the corresponding pre-response action based on the service type of the target proactive service includes: If the service type is a preset service, then enter the pre-start response mode; in the pre-start response mode, adjust the position of the robot, the orientation of the robot, and the orientation of the touch display screen according to the acquired user service preferences, in order to wait for the preset service to be started; If the service type is a predictive service, then the system enters the suggestion response mode. In the suggestion response mode, the position of the robot, the orientation of the robot, and the orientation of the touch display screen are adjusted according to the acquired user service preferences. Personalized service content is generated and displayed on the screen based on the service type and user service preferences to guide the user to start the service.

5. The mobile robot proactive service method according to claim 1, characterized in that, The method further includes: The system receives user-configured preset service information, obtains the preset service, and adds the preset service to the service timeline. The preset service information includes service topic, service content, user identifier, service time, service location, and service type. And / or, based on a preset update cycle, continuously learn from the acquired service usage data using a large language model to generate predictive services and add them to the service timeline; And / or, user service preferences are learned from the acquired service usage data through a large language model; the user service preferences include service content preferences and interaction method preferences with the robot.

6. The mobile robot proactive service method according to claim 5, characterized in that, The user service preferences include service content preferences and preferences for interaction methods with the robot; The target proactive service includes service topics; The process of learning user service preferences from acquired service usage data using a large language model includes: Among the multiple target proactive services that have been executed, the frequency of the service topic is counted to obtain the target proactive service with the highest frequency; based on the service content logs within the target proactive service with the highest frequency, a service summary is generated and saved to obtain the service content preference; The interaction preference is generated based on the frequency of use of the interaction method in the same interaction scenario.

7. The mobile robot proactive service method according to claim 1, characterized in that, The method further includes: If the robot is performing the target proactive service, it will not proactively plan and perform other services besides the target proactive service; And / or, determine the robot's charging time based on the service timeline to ensure it does not conflict with active services on the service timeline.

8. The mobile robot proactive service method according to claim 1, characterized in that, The method further includes: During the execution of the target proactive service, basic service information, service topic, and service content logs are recorded to obtain service usage data; The basic service information includes a unique service identifier, user identifier, service time, service location, and service type; the service topic includes multi-level topics; and the service content log includes multimodal interaction information, robot response log environment parameters, third-party software usage information, and anomaly information.

9. The mobile robot proactive service method according to claim 1, characterized in that, The method further includes at least one of the following: The first item: Confirm the identity of the target user to be served based on at least one of the first confidence level, the second confidence level, and the third confidence level; wherein, the first confidence level is obtained by performing facial recognition on the detected current user; the second confidence level is obtained by performing voiceprint recognition on the current user; and the third confidence level is obtained by verifying the terminal device carried by the current user based on near-field communication. The second item: controlling the robot to reach the target service location for the active service includes: The destination service location is determined based on a pre-built map and visual features of the current environment.

10. The mobile robot proactive service method according to any one of claims 1-9, characterized in that, The method further includes: Configure the service startup rules for the proactive service, specifically including: configuring the service time, target user, and target service location of the proactive service, and adding the proactive service to the service timeline.

11. The mobile robot proactive service method according to any one of claims 1-9, characterized in that, If the target proactive service is initiated, at least one proactive response action is generated based on the acquired interaction context information and the target service content of the target proactive service, including: The multimodal perception information included in the acquired interaction scenario information is input into a large language model for reasoning to obtain the interaction requirements included in the target service content, and at least one active response action is output based on the interaction requirements; the multimodal perception information includes: natural language information input by the user, visual information output by the user, and acquired external device data; Based on current interaction needs, user service preferences, scene context, and current perception information, proactive response actions are generated; the proactive response actions include adjusting the robot's position, adjusting the displayed UI style and content, and outputting voice prompts; the scene context includes real-time activity tags and environmental parameters.

12. A mobile robot, characterized in that, The mobile robot includes a service agent and a touch display screen, wherein the mobile robot is used to implement the mobile robot proactive service method as described in any one of claims 1-11.

Images