Body data acquisition planning method and device, electronic equipment and storage medium

Abstract

The invention provides a body data acquisition planning method and device, electronic equipment and a storage medium, and the method comprises the steps: constructing a body ability map, enabling nodes to be used for representing the executable ability of a robot and target elements in an environment, and enabling edges to be used for representing an association relationship between the nodes; body data are collected in real time, and the current coverage rate of the body data is calculated; determining one or more candidate data acquisition tasks based on the current coverage rate, and evaluating the expected gain of the coverage rate for executing each candidate task; according to the expected gain, a data collection task is selected from the candidate tasks, and a data collection action sequence is generated. The executable ability of the robot, the environmental target elements and the quantitative coverage target are fused by constructing the atlas, so that the data collection planning process becomes explainable and intervening; moreover, according to the coverage rate expected gain, a data acquisition action sequence is generated, blind repetition is avoided, and the effects of improving the data coverage rate and reducing redundancy are achieved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for embodied data acquisition planning. Background Technology

[0002] In recent years, with the rapid development of autonomous mobile robots and embodied intelligence technologies, robots are widely expected to autonomously perform data acquisition tasks in various complex environments, such as environmental monitoring, map building, and dataset collection. A core requirement for these tasks is that the data acquired by the robot must be highly comprehensive and representative to ensure the generalization ability of subsequent modeling, analysis, or machine learning models. Traditional pre-programmed approaches are poorly adaptable and inefficient in dynamic environments. While data-driven planning methods, such as reinforcement learning, improve environmental adaptability, their training costs are high, and the decision-making process is like a black box, difficult to understand and intervene in. On the other hand, methods using Large Language Models (LLM) for task planning face challenges in real-time robot deployment, including high computational demands, unstable decision-making, and difficulty in accurately integrating with embodied capabilities, leading to a disconnect between the planning and specific task objectives and physical constraints. In robot data acquisition, traditional solutions often employ exhaustive or random exploration methods to obtain data. To maximize data diversity, existing large-scale data collection attempts typically involve having robots perform various operations in different environments, attempting to cover as many scenario combinations as possible. This exploratory collection, without a clear optimization objective, is prone to data redundancy and omission of key information due to the lack of quantitative assessment of data value and coverage progress. Summary of the Invention

[0003] This invention provides an embodied data acquisition planning method, device, electronic device, and storage medium to address the shortcomings of embodied data acquisition methods, such as difficulty in understanding and intervention, high computing power requirements, unstable decision-making, and lack of quantitative assessment of data value and coverage progress in real-time robot deployment.

[0004] This invention provides an embodied data acquisition planning method, comprising: Construct an embodied capability graph, wherein the nodes of the graph are used to represent the robot's executable capabilities and target elements in the environment, and the edges of the graph are used to represent the relationships between the nodes; Real-time acquisition of embodied data, and calculation of the current coverage of the embodied data; Based on the current coverage, the embodied capability map is queried to determine one or more candidate data acquisition tasks, and the expected gain of performing each candidate task on the coverage is evaluated. Based on the expected gain, a data acquisition task is selected from the candidate tasks, and a data acquisition action sequence is generated based on the selected data acquisition task.

[0005] According to the embodied data acquisition planning method provided by the present invention, the construction of the embodied capability map includes: Based on the robot's preset capability list and prior environmental information, initial capability nodes, environment nodes, and initial edges representing the initial association between the capability nodes and the environment nodes are created to generate an initial embodied capability graph. The robot receives feedback information during task execution and updates the initial embodied capability map based on the feedback information to obtain the embodied capability map.

[0006] According to the embodied data acquisition planning method provided by the present invention, the real-time acquisition of embodied data and the calculation of the current coverage of the embodied data include: Construct an embodied dataset based on the real-time collected embodied data; The coverage metric is calculated based on the embodied dataset. The coverage metric includes the number of collected samples, the diversity entropy, and the confidence interval. The number of collected samples is determined based on the ratio of the number of data samples acquired to the expected total number of samples. The diversity entropy is determined based on the distribution entropy values ​​of the embodied dataset under different categories, locations, or conditions. The confidence interval is determined based on the performance confidence interval of the model trained on the embodied dataset on the validation set. Based on the coverage metric, the current coverage of the embodied data is obtained.

[0007] According to the embodied data acquisition planning method provided by the present invention, the step of querying the embodied capability map based on the current coverage to determine one or more candidate data acquisition tasks, and evaluating the expected gain of executing each candidate task on the coverage, includes: Based on the current coverage, query the task nodes associated with environmental element nodes and / or robot capability nodes that are not currently fully covered in the embodied capability map, and determine the tasks corresponding to the task nodes as the candidate data acquisition tasks. For each candidate data collection task, based on the node information associated with it in the embodied capability map, predict the new embodied data that will be added after the candidate data collection task is executed; Based on the newly added embodied data, assess the expected gain of the candidate data collection task on the current coverage.

[0008] According to the embodied data acquisition planning method provided by the present invention, the step of predicting the additional embodied data after executing the candidate data acquisition task, based on the node information associated with the candidate task in the embodied capability map, includes: Analyze at least one environmental element node and at least one capability node that are directly associated with the candidate data acquisition task in the embodied capability map; Each environmental element node is matched with a preset environment-data mapping rule to determine the data modal that can be collected under the current environmental element; each capability node is matched with a preset capability-operation mapping rule to determine the specific data collection operation supported by the current capability. Based on the collectable data modality and the specific data collection operation, determine the predicted data modality; Based on the spatial range, object category, or event type parameters in the attribute information of the environmental element nodes, and in conjunction with the parameters of the specific data collection operation, the predicted range of the collected content is determined. Based on the data modality and the content range, the newly added embodied data after executing the candidate data collection task is predicted.

[0009] According to the embodied data acquisition planning method provided by the present invention, the step of evaluating the expected gain of the candidate data acquisition task on the current coverage based on the newly added embodied data includes: The newly added embodied data is merged into the currently collected embodied dataset to form a simulated dataset; Calculate the simulation coverage corresponding to the simulation dataset; The difference between the simulated coverage and the current coverage is used as the expected gain of the candidate data acquisition task on the current coverage.

[0010] According to the embodied data acquisition planning method provided by the present invention, when multiple robots exist, the step of generating a data acquisition action sequence based on the selected data acquisition task further includes: Based on the executable capabilities represented by each robot in the embodied capability map, different selected data acquisition tasks are assigned to different robots. Based on the geographical location information represented by the environmental element nodes in the embodied capability map, the execution path is planned for each robot assigned a task; Based on the assigned tasks and execution paths, a data acquisition action sequence is generated for each robot. The present invention also provides a embodied data acquisition planning device, comprising: A construction module is used to construct an embodied capability graph, wherein the nodes of the graph represent the robot's executable capabilities and target elements in the environment, and the edges of the graph represent the relationships between the nodes. The data acquisition module is used to collect embodied data in real time and calculate the current coverage of the embodied data; An evaluation module is used to query the embodied capability map based on the current coverage to determine one or more candidate data acquisition tasks, and to evaluate the expected gain of performing each candidate task on the coverage. The generation module is used to select a data acquisition task from the candidate tasks according to the expected gain, and generate a data acquisition action sequence according to the selected data acquisition task.

[0011] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the embodied data acquisition planning method as described in any of the preceding claims.

[0012] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the embodied data acquisition planning method described in any of the preceding claims.

[0013] The embodied data acquisition planning method, apparatus, electronic device, and storage medium provided by this invention construct an embodied capability graph, where nodes represent the robot's executable capabilities and target elements in the environment, and edges represent the relationships between the nodes. The invention involves real-time acquisition of embodied data and calculation of its current coverage. Based on the current coverage, the embodied capability graph is queried to determine one or more candidate data acquisition tasks, and the expected gain of executing each candidate task on the coverage is evaluated. According to the expected gain, a data acquisition task is selected from the candidate tasks, and a data acquisition action sequence is generated based on the selected task. This invention, by constructing a graph reflecting the relationship between robot capabilities and environmental elements, deeply integrates the robot's executable capabilities, environmental target elements, and quantified coverage targets, making the data acquisition planning process transparent, interpretable, and operable, overcoming the shortcomings of traditional black-box models such as unstable and difficult-to-understand decision-making. Furthermore, generating a data acquisition action sequence based on the expected gain of coverage avoids blind repetition, achieving the effects of improving data coverage and reducing redundancy. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the specific data acquisition planning method provided in the embodiments of the present invention; Figure 2This is a functional structure diagram of the data acquisition and planning device provided in the embodiments of the present invention; Figure 3 This is a functional structure diagram of the electronic device provided in the embodiments of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0017] Figure 1 A flowchart of the specific data acquisition planning method provided in the embodiments of the present invention is shown below. Figure 1 As shown, the specific data acquisition planning method provided in this embodiment of the invention includes: Step 101: Construct an embodied capability graph, wherein the nodes of the graph are used to represent the robot's executable capabilities and target elements in the environment, and the edges of the graph are used to represent the relationships between the nodes; Step 102: Collect embodied data in real time and calculate the current coverage of the embodied data; In this embodiment of the invention, after collecting embodied data, edge computing nodes can be connected to perform real-time preprocessing of the embodied data to improve data effectiveness.

[0018] Step 103: Based on the current coverage, query the embodied capability map to determine one or more candidate data acquisition tasks, and evaluate the expected gain of performing each candidate task on the coverage. Step 104: Select a data acquisition task from the candidate tasks based on the expected gain, and generate a data acquisition action sequence based on the selected data acquisition task.

[0019] In this embodiment of the invention, after each round of the data acquisition action sequence is completed, the updated data coverage is compared with a preset coverage threshold; when the data coverage has reached the threshold, the generation of new acquisition tasks is stopped and an acquisition completion instruction is sent to terminate the further data acquisition process; if the data coverage has not reached the threshold, the task planning and data acquisition action sequence are repeated until the coverage requirement is met.

[0020] Traditional pre-programmed approaches are poorly adaptable and inefficient in dynamic environments. Data-driven planning methods, such as reinforcement learning, suffer from high training costs, and their decision-making processes are often like black boxes, making them difficult to understand and intervene in. Methods utilizing large language models for task planning face challenges in real-time robot deployment, including high computational demands, unstable decision-making, and difficulty in accurately aligning with embodied capabilities, leading to a disconnect between the planned data and specific task objectives and physical constraints. Furthermore, the lack of quantitative assessment of data value and coverage progress easily results in data redundancy and the omission of critical information.

[0021] The embodied data acquisition planning method provided in this invention constructs an embodied capability graph, where nodes represent the robot's executable capabilities and target elements in the environment, and edges represent the relationships between nodes. It then acquires embodied data in real time and calculates the current coverage rate. Based on the current coverage rate, it queries the embodied capability graph to determine one or more candidate data acquisition tasks and evaluates the expected gain of each candidate task on the coverage rate. According to the expected gain, it selects a data acquisition task from the candidate tasks and generates a data acquisition action sequence based on the selected task. This invention, by constructing a graph reflecting the relationship between robot capabilities and environmental elements, deeply integrates the robot's executable capabilities, environmental target elements, and quantified coverage targets, making the data acquisition planning process transparent, interpretable, and operable. This overcomes the shortcomings of traditional black-box models, such as unstable and difficult-to-understand decision-making. Furthermore, by generating a data acquisition action sequence based on the expected gain of the coverage rate, it avoids blind repetition, thereby improving data coverage and reducing redundancy.

[0022] Based on any of the above embodiments, the construction of the embodied capability map includes: Step 201: Based on the robot's preset capability list and environmental prior information, create initial capability nodes, environmental nodes, and initial edges representing the initial association between the capability nodes and the environmental nodes to generate an initial embodied capability graph. Step 202: Receive feedback information from the robot during the task execution process, and update the initial embodied capability map based on the feedback information to obtain an embodied capability map.

[0023] During the system initialization phase, an "embodied capability graph" is constructed offline or online based on the robot's known capabilities and prior knowledge of the target environment. This graph represents the robot's executable capabilities in the environment, key elements in the environment (locations, objects, events, etc.), and the relationships between them in the form of nodes and edges.

[0024] In this embodiment of the invention, nodes include, but are not limited to: robot capability nodes (e.g., navigation, obstacle avoidance, recognition, data acquisition, etc.) and environmental element nodes (e.g., rooms, areas, object categories, sensor observation points, etc.). Edges represent the relationships between nodes, such as the applicability of a capability to a certain environmental element, the prerequisite skills required to perform a certain task, and the dependency order between tasks. Through this graph, the system obtains a comprehensive representation of "what the robot can do," "what information can be collected from the environment," and "how to apply capabilities to the environment." Existing technologies rely solely on preset scripts; this embodiment of the invention, by constructing and updating an embodied capability graph, structurally integrates robot skills and environmental knowledge, providing a reasonable knowledge foundation for subsequent planning.

[0025] Based on any of the above embodiments, the real-time acquisition of embodied data and the calculation of the current coverage of the embodied data include: Step 301: Construct an embodied dataset based on the real-time collected embodied data; Step 302: Calculate the coverage index based on the embodied dataset. The coverage index includes the number of collected samples, the diversity entropy, and the confidence interval. The number of collected samples is determined based on the ratio of the number of acquired data samples to the expected total number of samples. The diversity entropy is determined based on the distribution entropy value of the embodied dataset under different categories, locations, or conditions. The confidence interval is determined based on the performance confidence interval of the model trained on the embodied dataset on the validation set. Step 303: Obtain the current coverage rate of the specific data based on the coverage rate index.

[0026] In this embodiment of the invention, a coverage metric is defined in the knowledge base to measure the completeness of data collection, based on specific application requirements. The coverage metric can be a single numerical value or a vector form, taking into account the following factors: a) Number of samples collected: The proportion of the data samples obtained relative to the expected total number of samples; b) Diversity entropy: The distribution entropy value of the collected data under different categories, locations, or conditions, used to quantify data diversity; c) Confidence interval: The confidence interval of the model's performance on the validation set based on the current data, reflecting whether the data sufficiently supports reliable model inference.

[0027] During system initialization, the coverage assessment module calculates the above indicators to obtain the initial coverage value. During task execution, the module dynamically updates the coverage indicators to provide feedback for planning decisions, enabling the system to have quantitative assessment capabilities, objectively judge the current data coverage level, and guide the next data collection decision.

[0028] Based on any of the above embodiments, the step of querying the embodied capability map based on the current coverage to determine one or more candidate data acquisition tasks, and evaluating the expected gain of performing each candidate task on the coverage, includes: Step 401: Based on the current coverage rate, query the task nodes associated with the environmental element nodes and / or robot capability nodes that are not currently fully covered in the embodied capability map, and determine the tasks corresponding to the task nodes as the candidate data acquisition tasks; Step 402: For each candidate data collection task, predict the new embodied data after executing the candidate data collection task based on the node information associated with it in the embodied capability map. Step 403: Based on the newly added embodied data, evaluate the expected gain of the candidate data collection task on the current coverage.

[0029] In this embodiment of the invention, predicting the additional embodied data after executing the candidate data collection task, based on the node information associated with it in the embodied capability map, for each candidate data collection task includes: Step 501: Analyze at least one environmental element node and at least one capability node that are directly associated with the candidate data acquisition task in the embodied capability map; Step 502: Match each environmental element node with the preset environment-data mapping rules to determine the data modal that can be collected under the current environmental element; match each capability node with the preset capability-operation mapping rules to determine the specific data collection operation supported by the current capability; Step 503: Determine the predicted data mode based on the collectable data mode and the specific data acquisition operation; Step 504: Based on the spatial range, object category, or event type parameters in the attribute information of the environmental element nodes, and in conjunction with the parameters of the specific data collection operation, determine the predicted range of the collected content. Step 505: Based on the data modality and the content range, predict the new embodied data to be added after the candidate data collection task is executed.

[0030] This invention uses an embodied capability map and current coverage assessment results as input to generate a data acquisition plan for the next step. The planning process employs heuristic search or optimization algorithms to find the action sequence that maximizes coverage within the task space represented by the map, specifically including: Based on the map, list the currently executable candidate data acquisition tasks (e.g., "Robot A goes to region X to collect sensor data", "Robot B interacts with object Y to obtain visual information", etc.). Remove completed or inapplicable nodes, and focus on map regions that are not yet covered or have low coverage.

[0031] For each candidate task, predict its gain on the coverage metric. Gains can be obtained through graph inference—for example, a task corresponding to a previously unexplored environmental node will significantly increase diversity entropy; or the data collected by a task can reduce the uncertainty of the model output, thereby improving the confidence interval. The task planning module combines the cost of task execution (such as required time and resource consumption) to calculate the benefit / cost ratio.

[0032] Based on any of the above embodiments, evaluating the expected gain of the candidate data collection task on the current coverage based on the newly added embodied data includes: Step 601: Merge the newly added embodied data into the currently collected embodied dataset to form a simulated dataset; Step 602: Calculate the simulation coverage corresponding to the simulation dataset; Step 603: Use the difference between the simulated coverage and the current coverage as the expected gain of the candidate data acquisition task on the current coverage.

[0033] In this embodiment of the invention, the task with the highest gain is selected from the candidates and included in the execution plan. If the robot discovers new environmental elements (e.g., previously unknown rooms or objects) or learns new skills / constraints (e.g., a path is impassable), the embodied capability graph is dynamically updated, adding new nodes or modifying relational edges. This dynamic updating of the graph ensures that the system's understanding of the environment and its own capabilities is always up-to-date. After each task or planning cycle is completed, the coverage assessment module recalculates the coverage index based on the accumulated dataset to determine the degree of completion of the current coverage target. If the coverage still does not reach the preset threshold, the system will trigger the next round of planning, forming a closed-loop control cycle. Conversely, if the coverage index has reached or exceeded predetermined requirements (e.g., the diversity entropy growth tends to stabilize and the confidence interval converges within the ideal range), the data acquisition task is considered essentially complete.

[0034] When the coverage rate reaches a preset threshold or the system determines that the marginal benefit has decreased to a certain extent, the task planning module issues a stop command to terminate further data collection, thereby saving resources and preventing invalid data collection. This closed-loop execution-evaluation-replanning mechanism ensures the system's adaptability: even in dynamic environments, the robot can continuously adjust its strategy to gradually approach the full coverage target.

[0035] Based on any of the above embodiments, generating a data acquisition action sequence according to the selected data acquisition task includes: Step 701: Analyze the robot capability nodes and environmental element nodes associated with the selected data acquisition task in the embodied capability map; Step 702: Based on the executable capabilities represented by the robot capability nodes, determine one or more atomic operations required to complete the data acquisition task; Step 703: Determine the execution parameters or spatial target of each atomic operation based on the location or attribute information represented by the environmental element nodes; Step 704: Arrange the one or more atomic operations in logical order to generate the data acquisition action sequence.

[0036] In some embodiments of the present invention, when multiple robots are present, the step of generating a data acquisition action sequence according to the selected data acquisition task further includes: Step 801: Based on the executable capabilities represented by each robot in the embodied capability map, assign the selected data acquisition tasks to different robots. Step 802: Based on the geographical location information represented by the environmental element nodes in the embodied capability map, plan the execution path for each robot assigned a task; Step 803: Based on the assigned task and the execution path, generate a data acquisition action sequence for each robot.

[0037] In this embodiment of the invention, at least one autonomous mobile robot comprises multiple autonomous mobile robots with different functions. When generating the data acquisition action sequence, different tasks are assigned to different robots for execution. Based on the matching of capability nodes of each robot in the embodied capability map and the geographical location of the task, an optimized scheduling strategy is adopted to enable multiple robots to work in parallel and collaboratively, thereby avoiding redundant data acquisition and improving the overall efficiency of data acquisition. When the multiple robots are heterogeneous and each has different sensors or operational capabilities, the embodied capability map further records the capability attributes of each robot and associates task nodes with corresponding robot nodes possessing that capability. Each data acquisition task is assigned to a target robot with the capability required to perform the task, ensuring that the task is undertaken by the most suitable robot, thereby improving the success rate and quality of data acquisition.

[0038] For example, strategies such as auction bidding and parallel task decomposition can be used to assign tasks to the most suitable robots based on geographical location or capability requirements, thereby improving overall efficiency. Compared to manual assignment or random selection, this module enables autonomous task decision-making, ensuring that each action moves towards improving coverage. At least one robot performs data collection tasks based on the planning results. In the case of a single robot, sub-tasks are executed sequentially according to the planned route; in the case of multiple robots, each robot executes its assigned tasks in parallel and maintains coordination through communication (e.g., avoiding duplicate collection of the same area). During execution, the robots provide real-time feedback of newly acquired data and environmental change information to the embodied capability map.

[0039] This invention implements advanced decision-making functions through an internal knowledge graph and customized planning algorithms, eliminating the need to call large models such as LLM / VLM in the external cloud for inference each time, thus avoiding the huge computing power and potential communication latency / instability required by the latter. It can run locally in a lightweight manner and update knowledge at any time, offering advantages such as low cost, fast response, and privacy security for practical deployment. Furthermore, it is easily expandable according to different application scenarios. For example, it can further introduce rich graph content such as semantic maps and task ontology, or combine reinforcement learning algorithms to improve planning strategies. In some embodiments of this invention, the coverage metric is expanded to multi-objective optimization (considering both accuracy and cost), human-computer interaction is introduced to obtain additional prior knowledge, and incremental learning mechanisms are introduced for scenarios with frequent dynamic changes.

[0040] To better illustrate the present invention, a detailed description of specific embodiments will be provided first.

[0041] Example 1: A single-robot environment acquisition system driven by embodied capability map.

[0042] This embodiment of the system includes an autonomous mobile robot and a back-end planning and control unit. The robot is equipped with a mobile chassis, environmental sensors (cameras, LiDAR, etc.), and a computing unit, enabling it to navigate and perceive in indoor environments. The planning and control unit can be deployed on the robot's onboard computer or a remote server, and includes a knowledge base module, a coverage assessment module, and a task planning module.

[0043] Initially, an embodied capability graph was constructed based on the indoor map and the robot's functional specifications. For example, nodes include location nodes for each room (living room, kitchen, etc.), sensor capability nodes for the robot (camera capture, temperature and humidity data collection, gas detection, etc.), and task nodes (capturing a panoramic view of the room, reading temperature sensor data, etc.). Relationship edges indicate that a task requires the robot to enter the corresponding room and activate the corresponding sensor capabilities to complete. The graph also records dependencies between tasks (e.g., "turn on the lights before taking a picture") and the impact of tasks on environmental changes.

[0044] In actual operation, the robot starts from its initial position and autonomously executes instructions from the planning and control unit to collect environmental data. The coverage assessment module sets metrics including: "the proportion of covered rooms to the total number of rooms," "the diversity entropy of acquired image samples (considering different lighting conditions)," and "the spatial coverage of temperature and humidity readings (the width of the interpolated confidence interval)." Initially, the robot only covered the starting room, resulting in low coverage. Based on this, the task planning module searches for uncovered room nodes in the map and plans the next navigation task: "go to the unvisited kitchen and take images." During this task, if the robot discovers multiple cubicles within the kitchen, it adds this new structure to the map as a refined environmental node.

[0045] Upon completion of the task, the coverage assessment module updates the metrics: room coverage increases, and diversity entropy increases slightly. The task planning module then selects the next task, such as "collecting temperature and humidity data in the living room." This process repeats, with the robot gradually traversing all areas of the environment and performing various sensing tasks. When the coverage assessment module detects that room coverage equals 100% and the rate of change of various data metrics approaches zero (i.e., the newly added data has no significant impact on diversity entropy and model confidence intervals), it determines that the coverage target has been achieved. The system terminates further tasks and issues a return command. Ultimately, the robot successfully acquires a complete environmental dataset, including images and environmental parameters for each room, with a distribution sufficient to support environmental modeling or agent training.

[0046] This invention verifies that in a single-robot scenario, it guides the robot to explore systematically, avoiding the omission of any region or data type. Simultaneously, through dynamic map expansion, new environmental discoveries (such as additional compartments) are also incorporated into the plan, ensuring the integrity of the final data. Compared to traditional methods of manually pre-programming routes, this system exhibits stronger adaptability and higher data utilization efficiency. By introducing coverage as an optimization objective, the system can specifically compensate for uncovered data areas or categories, ensuring the comprehensiveness of the collected results. Experimental results show that the dataset planned using this scheme significantly improves coverage in key dimensions, avoiding "blind spots" that traditional methods might miss, and increasing the robot's task success rate in new environments from less than 3% to over 77%. Furthermore, by optimizing coverage, the acquired data is sufficient to support robust model learning across various situations, significantly improving the performance of downstream applications.

[0047] Example 2: Multi-robot collaborative data acquisition system driven by embodied capability map.

[0048] This invention extends to multi-robot scenarios, including two mobile robots with different functions: Robot A is equipped with a high-definition camera for visual data acquisition; Robot B is equipped with an environmental sensor array (temperature, humidity, air quality detection, etc.) for environmental parameter acquisition. The two robots are connected to a central planning and control unit via a wireless network to collaboratively collect data from an office floor.

[0049] In this scenario, the embodied capability graph stored in the knowledge base module contains information about the robot's heterogeneous capabilities: for example, nodes are categorized as "visual acquisition capability (only A possesses it)," "environmental sensing acquisition capability (only B possesses it)," and nodes for each room and corridor on the floor. Task nodes include "acquiring image data from room X," "acquiring temperature and humidity data from room X," and "air quality detection in corridor Y," etc. Nodes and edges clearly map which tasks require which robot to perform. For example, the "room X - image acquisition" task node in the graph is connected to robot A's visual capability node, while the "room X - environmental parameter acquisition" task is connected to robot B's sensing capability node.

[0050] After data collection begins, the coverage assessment module tracks the coverage of visual and environmental data, such as the percentage of rooms already photographed and the geographical coverage of various environmental sensor readings. The task planning module comprehensively evaluates the coverage of both types of data and allocates tasks based on the map: prioritizing robots A and B to work in parallel in different uncollected rooms without interfering with each other. For example, at a certain moment, robot A is sent to the conference room to take photos, while robot B is simultaneously sent to the server room to record environmental indicators. If the coverage progress of a certain type of data lags behind (for example, many room photos have not yet been collected while environmental data is nearly complete), the planning module will prioritize scheduling relevant robots to accelerate that task.

[0051] During the collaborative process, if robot A discovers an inaccessible room, it notifies the planning unit via communication. The planning unit updates the map (marking the room as requiring special access) and temporarily suspends or reschedules the task for last. Simultaneously, if robot B discovers an unusually long and narrow corridor during data collection, causing slow movement and falling behind, the planning unit adjusts its strategy accordingly, allowing robot A to complete its own task before supporting environmental data collection in that corridor (e.g., robot A carrying robot B's sensors into the corridor). All this coordination is completed automatically through map and real-time information updates, requiring no human intervention.

[0052] Ultimately, the multi-robot team completed the collection of imagery and environmental data for all areas of the floor in a short period of time. The coverage assessment module confirmed that both types of data were covered in all rooms, and based on the data distribution, the various diversity and confidence indicators met expectations.

[0053] Compared to sequential execution by a single robot, the total data acquisition time in this embodiment of the invention is reduced by approximately 50%, significantly improving efficiency. When any robot encounters difficulties, the system can dynamically adjust for support from other robots, thus enhancing robustness. This verifies the effectiveness of the method provided in multi-robot scenarios. Through task allocation driven by capability graphs, robots with different capabilities can utilize their strengths and complement each other when needed, ensuring the successful completion of the overall task.

[0054] This invention is applicable to multi-robot scenarios. By using a unified map to represent the capabilities and task requirements of each robot, and combining this with optimization algorithms to allocate tasks, efficient collaboration among team robots is achieved. Each robot can both divide tasks and work in parallel to increase speed, and dynamically relay and complement each other as needed, maximizing overall performance. This enables high-coverage data acquisition tasks to be completed over a larger area in a shorter time.

[0055] Example 3: Application of coverage metrics in active learning.

[0056] This invention illustrates an application scenario for constructing a robot active learning dataset. Assume a robot needs to collect videos of various human behaviors in a room to train a gesture recognition model. The embodied capability atlas includes human behavior classification nodes (such as sitting, walking, picking up objects, etc.), robot sensing capability nodes (camera recording, microphone recording, etc.), and environmental region nodes. The coverage metric is primarily defined as "behavior sample coverage," which is the proportion of behavior types covered in the recorded video clips to the total list of behaviors, supplemented by the entropy value of the number of samples for each behavior to measure the uniformity of distribution.

[0057] The system determines which behavioral scenario the robot should record next based on the current coverage. When there are insufficient samples for certain behaviors, the system guides the robot to acquire more data on that type of behavior through human interaction or scene guidance. For example, if the geographic map and coverage assessment indicate that there are too few samples of the "picking up an item" behavior, the robot will move to a busy kitchen area (the geographic map indicates that picking up items is more likely to occur here) and prompt a person to pick up a cup, thus recording the video. Conversely, if there are already many samples of the "sitting down" behavior and they are sufficiently distributed, the system avoids collecting data on similar scenarios. In this way, the robot proactively balances the amount of data for various behaviors.

[0058] Throughout the active learning data acquisition process, the coverage metric provides clear guidance for the learning objectives, and the embodied capability graph ensures that the robot can map high-level semantic goals (behavioral categories) to specific executable actions. Ultimately, the behavioral video dataset collected by the robot achieves comprehensive and relatively balanced category coverage, significantly improving the performance of the subsequent gesture recognition model and validating the value of this invention in improving training data quality. While ensuring comprehensive coverage, redundant acquisition is minimized. The knowledge graph guides the robot to "only perform useful work," avoiding ineffective repetition. For example, when data for a certain area is sufficient, the system will no longer send the robot to collect data, thus saving time and resources. Compared to unplanned traversal methods, experimental results show that this invention reduces data acquisition time and energy consumption, achieving efficient and high-quality exploration, improving the robustness and flexibility of the robot system in unknown environments, and enabling it to safely and efficiently complete complex and varied tasks.

[0059] This invention overcomes many shortcomings of traditional robot data acquisition schemes by combining "embodied capability mapping" with "coverage-driven planning," significantly improving the intelligence level and result quality of data acquisition tasks. Its advantages, such as high coverage, high efficiency, adaptability, and collaboration, make it promising for applications in autonomous mobile robots, multi-agent systems, and robot active learning.

[0060] The embodied data acquisition planning method provided by this invention uses an embodied capability map to associate robot-executable actions with environmental knowledge, enabling planning decisions to be based on a comprehensive understanding of the environment and capabilities. This improves the intelligence and rationality of task planning and avoids blind exploration and ineffective operations. For example, when the map shows that a certain area is not yet covered, the system will prioritize dispatching robots to collect data, thereby expanding the coverage area in a targeted manner. By quantifying the collection progress through coverage indicators, the planning module can optimize with the goal of maximizing data coverage, improving the comprehensiveness and efficiency of data collection. Robots will not repeatedly collect saturated information but will concentrate their efforts on uncovered areas or areas with high information gain. According to statistics, this solution can obtain a richer data distribution with the same resource investment, reducing a large amount of redundant collection compared to traditional methods without indicator guidance. It supports multiple robots working collaboratively, leveraging the parallelism and complementarity of team robots and accelerating the coverage process. For example, when deploying robots with different sensors in an environment, the system can assign visual acquisition tasks to robots equipped with cameras and environmental monitoring tasks to robots equipped with gas sensors, each performing its own task simultaneously, significantly shortening the time required for full coverage. The embodied capability map is dynamically updated, allowing new knowledge acquired during robot exploration to be promptly incorporated into the map, enhancing the system's adaptability and robustness in the face of unknown scenarios and environmental changes. When new situations arise beyond the scope of the original map, the system can learn and adjust subsequent plans in real time, ensuring the continuity and integrity of the task. For example, if an unrecorded room is discovered during exploration, the system will automatically include that room in the subsequent coverage plan after the map is updated, obtaining high-coverage, high-quality environmental data, providing a foundation for downstream data analysis or model training.

[0061] The present invention provides a method for planning embodied data acquisition. The method described below can be referred to in correspondence with the method described above.

[0062] Figure 2 A functional structure diagram of the data acquisition and planning device provided in the embodiments of the present invention is shown below. Figure 2 As shown, the embodied data acquisition and planning device provided in this embodiment of the invention includes: The construction module 201 is used to construct an embodied capability graph, wherein the nodes of the graph are used to represent the robot's executable capabilities and target elements in the environment, and the edges of the graph are used to represent the relationships between the nodes. The acquisition module 202 is used to acquire embodied data in real time and calculate the current coverage of the embodied data; Evaluation module 203 is used to query the embodied capability map based on the current coverage to determine one or more candidate data acquisition tasks, and to evaluate the expected gain of performing each candidate task on the coverage. The generation module 204 is used to select a data acquisition task from the candidate tasks according to the expected gain, and generate a data acquisition action sequence according to the selected data acquisition task.

[0063] This device can be applied to at least one autonomous mobile robot and a planning and control unit that communicates with the robot. The planning and control unit contains core components such as a knowledge base module (storing embodied capability maps), a coverage assessment module, and a task planning module.

[0064] The embodied data acquisition planning device provided in this invention constructs an embodied capability graph, where nodes represent the robot's executable capabilities and target elements in the environment, and edges represent the relationships between nodes. It collects embodied data in real time and calculates the current coverage rate of the embodied data. Based on the current coverage rate, it queries the embodied capability graph to determine one or more candidate data acquisition tasks and evaluates the expected gain of executing each candidate task on the coverage rate. According to the expected gain, it selects a data acquisition task from the candidate tasks and generates a data acquisition action sequence based on the selected task. This invention, by constructing a graph reflecting the relationship between robot capabilities and environmental elements, deeply integrates the robot's executable capabilities, environmental target elements, and quantified coverage targets, making the data acquisition planning process transparent, interpretable, and operable. This overcomes the shortcomings of traditional black-box models, such as unstable and difficult-to-understand decision-making. Furthermore, by generating a data acquisition action sequence based on the expected gain of the coverage rate, it avoids blind repetition, thereby improving data coverage and reducing redundancy.

[0065] Figure 3 An example is a schematic diagram of the physical structure of a communication device, such as... Figure 3As shown, the communication device may include a processor 310, a communication interface 320, a memory 330, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The memory 330 includes computer programs, an operating system, and acquired data. The processor 310 can call logical instructions in the memory 330 to execute an embodied data acquisition planning method. This method includes: constructing an embodied capability graph, where nodes represent the robot's executable capabilities and target elements in the environment, and edges represent the relationships between nodes; acquiring embodied data in real time and calculating the current coverage of the embodied data; based on the current coverage, querying the embodied capability graph to determine one or more candidate data acquisition tasks and evaluating the expected gain of executing each candidate task on the coverage; selecting a data acquisition task from the candidate tasks according to the expected gain, and generating a data acquisition action sequence based on the selected data acquisition task.

[0066] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the embodied data acquisition planning method provided by the methods described above. The method includes: constructing an embodied capability graph, wherein nodes of the graph characterize the robot's executable capabilities and target elements in the environment, and edges of the graph characterize the relationships between the nodes; acquiring embodied data in real time and calculating the current coverage of the embodied data; based on the current coverage, querying the embodied capability graph to determine one or more candidate data acquisition tasks and evaluating the expected gain of executing each candidate task on the coverage; selecting a data acquisition task from the candidate tasks according to the expected gain, and generating a data acquisition action sequence according to the selected data acquisition task.

[0068] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0069] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An embodied data acquisition planning method, characterized in that, include: Construct an embodied capability graph, wherein the nodes of the graph are used to represent the robot's executable capabilities and target elements in the environment, and the edges of the graph are used to represent the relationships between the nodes; Real-time acquisition of embodied data, and calculation of the current coverage of the embodied data; Based on the current coverage, the embodied capability map is queried to determine one or more candidate data acquisition tasks, and the expected gain of performing each candidate task on the coverage is evaluated. Based on the expected gain, a data acquisition task is selected from the candidate tasks, and a data acquisition action sequence is generated based on the selected data acquisition task.

2. The embodied data acquisition planning method according to claim 1, characterized in that, The construction of the embodied capability graph includes: Based on the robot's preset capability list and prior environmental information, initial capability nodes, environment nodes, and initial edges representing the initial association between the capability nodes and the environment nodes are created to generate an initial embodied capability graph. The robot receives feedback information during task execution and updates the initial embodied capability map based on the feedback information to obtain the embodied capability map.

3. The embodied data acquisition planning method according to claim 1, characterized in that, The real-time acquisition of embodied data and the calculation of the current coverage of the embodied data include: Construct an embodied dataset based on the real-time collected embodied data; The coverage metric is calculated based on the embodied dataset. The coverage metric includes the number of collected samples, the diversity entropy, and the confidence interval. The number of collected samples is determined based on the ratio of the number of data samples acquired to the expected total number of samples. The diversity entropy is determined based on the distribution entropy values ​​of the embodied dataset under different categories, locations, or conditions. The confidence interval is determined based on the performance confidence interval of the model trained on the embodied dataset on the validation set. Based on the coverage metric, the current coverage of the embodied data is obtained.

4. The embodied data acquisition planning method according to claim 1, characterized in that, The step of querying the embodied capability map based on the current coverage to determine one or more candidate data acquisition tasks, and evaluating the expected gain of performing each candidate task on the coverage, includes: Based on the current coverage, query the task nodes associated with environmental element nodes and / or robot capability nodes that are not currently fully covered in the embodied capability map, and determine the tasks corresponding to the task nodes as the candidate data acquisition tasks. For each candidate data collection task, based on the node information associated with it in the embodied capability map, predict the new embodied data that will be added after the candidate data collection task is executed; Based on the newly added embodied data, assess the expected gain of the candidate data collection task on the current coverage.

5. The embodied data acquisition planning method according to claim 4, characterized in that, For each candidate data collection task, based on its associated node information in the embodied capability map, the predicted additional embodied data after executing the candidate data collection task includes: Analyze at least one environmental element node and at least one capability node that are directly associated with the candidate data acquisition task in the embodied capability map; Each environmental element node is matched with a preset environment-data mapping rule to determine the data modal that can be collected under the current environmental element; each capability node is matched with a preset capability-operation mapping rule to determine the specific data collection operation supported by the current capability. Based on the collectable data modality and the specific data collection operation, determine the predicted data modality; Based on the spatial range, object category, or event type parameters in the attribute information of the environmental element nodes, and in conjunction with the parameters of the specific data collection operation, the predicted range of the collected content is determined. Based on the data modality and the content range, the newly added embodied data after executing the candidate data collection task is predicted.

6. The embodied data acquisition planning method according to claim 4, characterized in that, The step of evaluating the expected gain of the candidate data collection task on the current coverage based on the newly added embodied data includes: The newly added embodied data is merged into the currently collected embodied dataset to form a simulated dataset; Calculate the simulation coverage corresponding to the simulation dataset; The difference between the simulated coverage and the current coverage is used as the expected gain of the candidate data acquisition task on the current coverage.

7. The embodied data acquisition planning method according to claim 1, characterized in that, When multiple robots are present, the step of generating a data acquisition action sequence based on the selected data acquisition task further includes: Based on the executable capabilities represented by each robot in the embodied capability map, different selected data acquisition tasks are assigned to different robots. Based on the geographical location information represented by the environmental element nodes in the embodied capability map, the execution path is planned for each robot assigned a task; Based on the assigned task and the execution path, a data acquisition action sequence is generated for each robot.

8. A personalized data acquisition and planning device, characterized in that, include: A construction module is used to construct an embodied capability graph, wherein the nodes of the graph represent the robot's executable capabilities and target elements in the environment, and the edges of the graph represent the relationships between the nodes. The data acquisition module is used to collect embodied data in real time and calculate the current coverage of the embodied data; An evaluation module is used to query the embodied capability map based on the current coverage to determine one or more candidate data acquisition tasks, and to evaluate the expected gain of performing each candidate task on the coverage. The generation module is used to select a data acquisition task from the candidate tasks according to the expected gain, and generate a data acquisition action sequence according to the selected data acquisition task.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the embodied data acquisition planning method as described in any one of claims 1 to 7.

10. A non-transitory readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the embodied data acquisition planning method as described in any one of claims 1 to 7.

Images