Scene restoration method and device, medium and program product
By generating a dynamic difference point cloud representing the current scene relative to the target scene, the intelligent agent can effectively detect and process complex scene changes, improving detection accuracy and real-time performance.
Patent Information
- Application Number
- CN202511700680.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-19
- Publication Date
- 2026-02-24
AI Technical Summary
In existing technologies, intelligent agents cannot effectively address the complexity of scenarios when dealing with changes in displacement, shape, and existence, especially in terms of methods for detecting and processing scene changes.
By generating a dynamic difference point cloud representing the current scene, a scene restoration method is developed that significantly improves the detection accuracy and real-time performance of scene changes.
This method employs a scene reconstruction approach based on dynamic difference point clouds. By generating representations of changes in the current scene, it enables dynamic updates of the agent, solving the problem of scene change detection and processing in existing technologies and improving the detection accuracy and real-time performance of existing technologies.
Smart Images
Figure CN121564211A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of embodied intelligence technology, and in particular to a scene restoration method, apparatus, medium, and program product. Background Technology
[0002] In real-world environments, the scene changes faced by intelligent agents are far more diverse and complex than just displacement changes. Specifically, common object changes can be categorized into displacement changes, shape changes, and existence changes. Existing methods primarily focus on handling displacement changes, while neglecting shape and existence changes, resulting in agents' inability to effectively cope with complex real-world environments. Furthermore, most existing methods rely on predefined policies, lacking flexibility and failing to address dynamic and complex scene changes. Summary of the Invention
[0003] To address the shortcomings of existing technologies, this invention proposes a scene restoration method based on dynamic difference point clouds, which significantly improves the detection accuracy and real-time performance of scene changes.
[0004] This invention provides a scene restoration method based on dynamic difference point clouds, comprising:
[0005] The intelligent agent acquires depth observation data of the target scene and depth observation data of the current scene, generates a difference point cloud representing the changes of objects in the current scene relative to the target scene, and dynamically updates the difference point cloud during the interaction process.
[0006] After the agent performs an interactive action in response to changes in objects in the difference point cloud, it evaluates the effectiveness of the interactive action in reducing the difference between the current scene and the target scene by calculating the amount of change in the difference point cloud before and after the interactive action is performed.
[0007] The agent prioritizes candidate interactive actions applied to the differential point cloud, selects the candidate interactive action with the highest score for execution, and adjusts the action strategy based on the effectiveness evaluation results to gradually restore the scene.
[0008] In one embodiment of the present invention, generating the difference point cloud includes:
[0009] During the roaming phase, the intelligent agent observes the target scene, acquires environmental depth observation data of the target scene, generates the first global point cloud of the target scene, and stores it as memory information;
[0010] During the restoration phase, the agent observes the current scene, acquires depth observation data of the current scene, and generates a second global point cloud of the current scene. By comparing the second global point cloud with the first global point cloud, the difference point cloud is extracted.
[0011] In one embodiment of the present invention, a local observation point cloud centered on the perspective of the agent is generated based on the environmental depth observation data of the target scene. The local observation point cloud is then converted into a point cloud in the global coordinate system by combining the agent's own pose information and recorded as the first global point cloud.
[0012] In one embodiment of the present invention, during the recovery phase, the agent alternately operates in exploration mode and interaction mode;
[0013] In the exploration mode, the depth observation data of the current scene is converted into a point cloud in the global coordinate system and recorded as the second global point cloud;
[0014] The current observation position of the second global point cloud is matched with the pose information of the first global point cloud. The second global point cloud and the first global point cloud from the corresponding viewpoint are compared to extract the disappearing point cloud clusters and the newly added point cloud clusters to form the difference point cloud.
[0015] In interactive mode, after performing the interactive action, the difference point cloud in the current field of view is refreshed instead of being overlaid and updated.
[0016] In one embodiment of the present invention, the quantization of the change in the difference point cloud includes:
[0017] Define the change ,
[0018] in, The difference point cloud before the execution of the interactive action. For the difference point cloud after execution, |⋅| is the norm operator that quantifies the size of the difference point cloud, defined as the number of difference points in the difference point cloud;
[0019] When the change is greater than a preset difference reduction threshold, the interaction action is determined to be a valid action.
[0020] In one embodiment of the present invention, prioritizing candidate interactive actions includes:
[0021] For the newly added point cloud clusters in the difference point cloud, a scoring function is used to score the candidate interactive actions; the scoring function is:
[0022]
[0023] in, , As a weighting factor, Geometric matching factor As an environmental adaptation factor, For the set of moving parts, For current observation information, For the current candidate interaction action, This is for adding a new point cloud cluster.
[0024] In one embodiment of the present invention, the geometric matching factor is calculated as follows: if the newly added point cloud cluster matches a cluster in the set of moving parts, it is determined that the object is the original object and its position has moved or its shape has changed. The geometric matching score of the discard action is set to 0, and the score of other actions is set to 1.
[0025] If the newly added point cloud cluster does not match a cluster in the set of moving parts, the object is determined to be a newly appearing foreign object, the geometric matching score of the discard action is set to 1, and the scores of other actions are set to 0.
[0026] The changes in the object include positional movement, shape change, and changes in the object's existence.
[0027] In one embodiment of the present invention, the calculation method of the environmental adaptation factor includes:
[0028] The newly added point cloud cluster is projected onto the RGB image at the current viewpoint to obtain a two-dimensional bounding box;
[0029] The RoI pooling method is used to extract the local visual features of the region from the global feature map of the RGB image;
[0030] The local visual features are processed by a multilayer perceptron, and a scalar score is output as the score of the environment adaptation factor to represent the degree of adaptation of the candidate interactive action under the current observation conditions.
[0031] In one embodiment of the present invention, the multilayer perceptron is trained using a static dataset, which contains various scenarios and interactive behaviors. The training samples include the local visual features, action categories, and adaptability labels. The adaptability labels are marked as 1 or 0 depending on whether the action is executed successfully or not.
[0032] In one embodiment of the present invention, the candidate interactive action with the highest score is selected for execution, and the result is...
[0033] Adjust action strategies based on effectiveness assessment results, including:
[0034] If the candidate interaction action with the highest evaluation score is effective, the current object change is marked as recovered and the next object change is processed; if it is ineffective, the candidate interaction action with the second highest score is selected for execution and its effectiveness is re-evaluated until the scene is restored or it is determined to be unrecoverable.
[0035] In another aspect, the present invention provides a scene restoration device based on dynamic difference point clouds, comprising:
[0036] The scene change detection module is used by the agent to acquire depth observation data of the target scene and depth observation data of the current scene, generate a difference point cloud representing the changes of objects in the current scene relative to the target scene, and dynamically update the difference point cloud during the interaction process.
[0037] The interaction effect evaluation module is used to evaluate the effectiveness of the interaction action in reducing the difference between the current scene and the target scene by calculating the amount of change in the difference point cloud before and after the interaction action is performed after the agent performs the interaction action in response to the changes in the object in the difference point cloud.
[0038] The trial-and-error module is used by the agent to prioritize candidate interactive actions acting on the difference point cloud, select the interactive action with the highest score for execution, and adjust the action strategy based on the effectiveness evaluation results to gradually restore the scene.
[0039] In another aspect, the present invention provides a readable storage medium storing a program or instructions which, when executed by a processor, implement the steps of the method described above.
[0040] In another aspect, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.
[0041] As can be seen from the above solutions, the advantages of the present invention are:
[0042] This invention discloses a scene restoration method based on dynamic difference point clouds. First, an intelligent agent acquires depth observation data of the target scene and the current scene, generating a difference point cloud, which is dynamically updated during interaction. Then, after the agent performs an interactive action in response to changes in objects within the difference point cloud, the effectiveness of the interactive action in reducing the difference between the current and target scenes is evaluated by calculating the change in the difference point cloud before and after the action. Furthermore, the agent prioritizes candidate interactive actions acting on the difference point cloud, selects the highest-scoring candidate action for execution, and adjusts the action strategy based on the effectiveness evaluation results to gradually achieve scene restoration. This method uses dynamic difference point cloud technology to accurately detect subtle changes in the environment in real time, without relying on semantic labels or category information, significantly improving the detection accuracy and real-time performance of scene changes. It is particularly suitable for complex, unknown environments and objects, and effectively improves the accuracy and efficiency of environmental state restoration in variable scenes. Attached Figure Description
[0043] Figure 1 A schematic diagram of the overall process of a scene restoration method based on dynamic difference point cloud provided by an embodiment of the present invention is shown;
[0044] Figure 2 A schematic diagram of the structure of a scene restoration device based on dynamic difference point cloud provided in an embodiment of the present invention is shown.
[0045] The attached figures are labeled as follows:
[0046] 200: Scene restoration device based on dynamic difference point cloud;
[0047] 210: Scene change detection module;
[0048] 220: Interaction effect evaluation module;
[0049] 230: Trial and error module. Detailed Implementation
[0050] It should be noted that, in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.
[0051] In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0052] See Figure 1 As shown, Figure 1 The diagram shows the overall flow of a scene restoration method based on dynamic difference point clouds provided in an embodiment of the present invention.
[0053] A scene restoration method based on dynamic difference point clouds includes the following steps:
[0054] Step S1: The agent acquires depth observation data of the target scene and depth observation data of the current scene, generates a difference point cloud representing the changes of objects in the current scene relative to the target scene, and dynamically updates the difference point cloud during the interaction process.
[0055] This invention utilizes dynamic differential point clouds to depict various changes in objects within a scene with finer granularity. Because point cloud data can accurately capture the geometric shape, spatial location, and scale information of objects, it is highly applicable in representing changes in object position, shape (such as opening and closing states), and presence changes such as addition or disappearance. Furthermore, point cloud data exhibits strong viewpoint robustness, maintaining high change detection accuracy even under different viewing angles and partial occlusion.
[0056] Specifically, the construction process of the difference point cloud not only considers information from the static scene but also dynamically updates it during agent interaction, reflecting the interaction effects between the agent and the environment in real time (e.g., moving furniture or changing the opening and closing state of containers). Compared to static construction methods, this dynamic mechanism can quickly capture environmental change information, improving the real-time performance and accuracy of scene change detection. It should be noted that the "intelligent agent" mentioned in this invention refers to an intelligent robot capable of perceiving its environment and performing actions.
[0057] In one specific implementation, during the roaming phase, the agent only explores the environment without interacting with it. The agent observes the target scene, acquires environmental depth observation data D1 of the target scene through its own RGB-D camera, and generates the first global point cloud of the target scene. And store it as memory information. Specifically, based on the environmental depth observation data D1 of the target scene, generate a local observation point cloud centered on the agent's perspective. Combined with the agent's own pose information The local observation point cloud Convert to a point cloud in the global coordinate system and record it as the first global point cloud. and corresponding pose information They are recorded together in the agent's memory module.
[0058] During the restoration phase, the agent observes the current scene, acquires the depth observation data D2 of the current scene, and generates the second global point cloud of the current scene. By comparing the second global point cloud With the first global point cloud The difference point cloud is extracted. Specifically, during the restoration phase, the agent alternately runs exploration mode and interaction mode. In exploration mode, the agent observes the current scene, acquires environmental depth observation data D2 of the current scene through its own RGB-D camera, and transforms the depth observation data D2 of the current scene into a point cloud in the global coordinate system, recording it as the second global point cloud. Match the current observation position of the second global point cloud. With the self-pose information in the first global point cloud If a match is successful, meaning the viewpoints roughly overlap, then compare with the second global point cloud. With the first global point cloud To extract vanishing point cloud clusters and newly added point cloud clusters relative to the target scene, which includes the point set portion of the current scene that has been displaced, shaped, or has added objects relative to the target scene, thus constituting the difference point cloud.
[0059] In interactive mode, after the interactive action is performed, the difference point cloud in the current field of view is refreshed in real time, instead of being continuously superimposed on the previous one, so as to quickly and accurately reflect the immediate change effect brought about by the action.
[0060] Step S2: After the agent performs an interactive action in response to the changes in objects in the difference point cloud, it evaluates the effectiveness of the interactive action in reducing the difference between the current scene and the target scene by calculating the amount of change in the difference point cloud before and after the interactive action is performed.
[0061] After the agent constructs the difference point cloud, it enters the restoration phase. The agent performs interactive actions in response to changes in objects within the difference point cloud. By calculating the changes in the difference point cloud before and after the interactive action, it is determined whether the agent's current interactive action can effectively reduce the difference between the current scene state and the target scene state. By quantifying the specific impact of each interactive action on the difference point cloud, it is determined whether the interactive action contributes to scene restoration.
[0062] Specifically, the effectiveness of an interactive action is evaluated by calculating the changes in the difference point cloud before and after the action is performed. Assuming a time step... The difference point cloud over time is represented as This includes all point cloud information where the current scene differs from the target scene. When the agent performs an interaction action... Subsequently, the scene changes accordingly, and the updated difference point cloud is denoted as... .
[0063] Define the change ,in The difference point cloud before the execution of the interactive action The resulting difference point cloud, |⋅|, is a norm operator that quantifies the size of the difference point cloud. It is defined as the number of difference points in the difference point cloud. This norm directly reflects the degree of difference between the current scene and the target scene: the more point points, the greater the difference; a decrease in the number means that the agent's interaction has successfully reduced the difference between the current scene and the target scene. When the change exceeds a preset difference reduction threshold... When, determine the interaction action. For this to be considered a valid action, the difference reduction threshold is [not specified]. Its function is to distinguish between significantly effective actions and weak or ineffective actions, preventing the agent from misjudging the effect of actions due to observation errors, noise, or small random changes.
[0064] By comprehensively considering the changes in the environmental state before and after the interaction, the actual effect of the agent's actions is quantified, effectively improving the accuracy and efficiency of environmental state recovery.
[0065] Step S3: The agent prioritizes the candidate interactive actions acting on the differential point cloud, selects the interactive action with the highest score for execution, and adjusts the action strategy based on the effectiveness evaluation results to gradually restore the scene.
[0066] For each newly added point cloud cluster in the difference point cloud How to effectively select the appropriate recovery operation to be performed by the main execution? In one embodiment, all possible candidate interaction actions are effectively evaluated and a priority ranking is established.
[0067] In one specific implementation, a scoring function is used to score the candidate interactive actions. The scoring function is defined as follows:
[0068]
[0069] in, , As adjustable weighting factors, the contribution ratios of geometric matching factor and environmental adaptation factor to the final score are controlled separately. Geometric matching factor As an environmental adaptation factor, For the set of moving parts, For current observation information, For the current candidate interaction action, This is for adding a new point cloud cluster.
[0070] Among them, the geometric matching factor New point cloud clusters added based on differences in point clouds With moving part set The matching relationship is analyzed to determine the specific type of object change, thereby guiding action selection. Specifically, if the newly added point cloud cluster matches a cluster in the set of moving parts, it is determined that the object is an existing object, meaning it originally existed in the environment and its position has moved or its shape has changed. In this case, the geometric matching score for the discard action is set to 1, and the scores for other actions are set to 0. If the newly added point cloud cluster does not match a cluster in the set of moving parts, it is determined that the object is a newly appeared foreign object. The geometric matching score for the discard action is set to 1, and the scores for other actions are set to 0. In this way, the agent can initially distinguish the change types of new objects from existing objects from a geometric space perspective.
[0071] The environment adaptation factor analyzes current environmental information, determining the feasibility and applicability of an action based on contextual information such as the object's shape, position, and environmental constraints. This factor introduces richer visual perception and local environment understanding, enabling the agent to assess the feasibility of interactive actions in light of the actual scene. For example, the success of a pickup action depends not only on whether the object's type or shape is graspable, but also heavily on factors such as the object's current spatial location (ease of access), the presence of obstructions, and the agent's current orientation.
[0072] Specifically, the environmental adaptation factor The calculation method includes: adding the new point cloud clusters Projecting the RGB image onto the current viewpoint, a two-dimensional bounding box for the region is obtained. Then, the RoI pooling method is used to extract the local visual features of the region from the global feature map of the RGB image. The local visual features are processed by a multilayer perceptron, and a scalar score is output as the score of the environment adaptation factor to represent the degree of adaptation of the candidate interactive action under the current observation conditions.
[0073] In one embodiment, a static dataset containing various scenarios and interactive behaviors was constructed to train a multilayer perceptron. During data acquisition, the agent was randomly initialized at different positions and angles, attempting to perform a series of interactive actions (such as picking up, placing, dropping, moving, opening, etc.) around various objects, and the success or failure of each interactive action was labeled. The training samples included the local visual features, action category, and adaptability label corresponding to each attempt, thus providing clear signals for supervised training. The adaptability label was marked according to the success or failure of the action execution; for example, the adaptability label was defined as 1 for success and 0 for failure.
[0074] Furthermore, in one embodiment, although priority scoring can provide the agent with preliminary action decision-making basis, there is still a certain risk of misjudgment in real complex scenarios. Therefore, the candidate interaction action with the highest score is further selected for execution, and the action strategy is adjusted in combination with the effectiveness evaluation result determined in step S2 to gradually realize scene restoration. Specifically, the agent first selects the candidate interaction action with the highest score. Execute the effectiveness evaluation of the highest-rated interactive action. If the highest-rated interactive action is evaluated... Effective, meaning it effectively reduced the size of the differential point cloud and successfully recovered the newly added point cloud cluster. The corresponding changes will affect the newly added point cloud clusters. Marked as successfully recovered, the agent then continues processing the next newly added point cloud cluster in the scene; if the candidate interaction action with the highest evaluation score is selected... If invalid, it means that the candidate interaction action with the highest score did not achieve the desired effect. The agent will then try the candidate interaction action with the second-best score and repeat the cycle of action execution and effect evaluation until a valid candidate interaction action is found or it is clear that the newly added point cloud cluster cannot be effectively recovered.
[0075] In this embodiment, static scoring and real-time interactive feedback are dynamically integrated, and the interaction decision-making order of the agent is iteratively optimized based on the actual environmental feedback results, which significantly enhances the generalization ability and adaptability in complex and ever-changing environments.
[0076] In summary, the scene restoration method based on dynamic difference point clouds provided by this invention firstly acquires depth observation data of the target scene and the current scene through an intelligent agent, generates a difference point cloud, and dynamically updates the difference point cloud during interaction. Then, after the intelligent agent performs an interactive action in response to changes in objects in the difference point cloud, the effectiveness of the interactive action in reducing the difference between the current scene and the target scene is evaluated by calculating the change in the difference point cloud before and after the interactive action. Furthermore, the intelligent agent prioritizes candidate interactive actions acting on the difference point cloud, selects the candidate interactive action with the highest score for execution, and adjusts the action strategy based on the effectiveness evaluation results to gradually achieve scene restoration. This method uses dynamic difference point cloud technology to accurately detect subtle changes in the environment in real time, without relying on semantic labels or category information, significantly improving the detection accuracy and real-time performance of scene changes. It is particularly suitable for complex unknown environments and objects, and effectively improves the accuracy and efficiency of environmental state restoration in variable scenes.
[0077] This embodiment corresponds to the device embodiment described above, such as... Figure 2 As shown, Figure 2 Show
[0078] A schematic diagram of a scene restoration device based on dynamic difference point clouds according to an embodiment of the present invention is shown. This device embodiment can be implemented in conjunction with the above-described method embodiment. The relevant technical details mentioned in the above method embodiment remain valid in this device embodiment, and will not be repeated here to avoid repetition.
[0079] A scene restoration device 200 based on dynamic difference point clouds, comprising:
[0080] The scene change detection module 210 is used to enable the agent to acquire depth observation data of the target scene and depth observation data of the current scene, generate a difference point cloud representing the changes of objects in the current scene relative to the target scene, and dynamically update the difference point cloud during the interaction process.
[0081] The interaction effect evaluation module 220 is used to evaluate the effectiveness of the interaction action in reducing the difference between the current scene and the target scene by calculating the amount of change in the difference point cloud before and after the interaction action is performed after the agent performs the interaction action in response to the changes in the object in the difference point cloud.
[0082] The trial-and-error module 230 is used by the agent to prioritize the candidate interactive actions acting on the difference point cloud, select the interactive action with the highest score to execute, and adjust the action strategy based on the effectiveness evaluation results to gradually realize scene restoration.
[0083] This device embodiment can be implemented in conjunction with the implementation methods described above. The relevant technical details mentioned in the implementation methods of the above embodiments remain valid in the implementation methods of this system embodiment, and will not be repeated here to avoid repetition. It should be noted that the scope of the methods and devices in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be applied, omitted, or combined. Furthermore, features described with reference to certain examples may be combined in other examples.
[0084] This invention also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the above-described scene restoration method based on dynamic difference point clouds, and achieve the same technical effect.
[0085] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described scene restoration method based on dynamic difference point clouds, and achieves the same technical effect.
[0086] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software program product is stored in a storage medium and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0088] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.
Claims
1. A scene restoration method based on dynamic difference point clouds, characterized in that, Include: The intelligent agent acquires depth observation data of the target scene and depth observation data of the current scene, generates a difference point cloud representing the changes of objects in the current scene relative to the target scene, and dynamically updates the difference point cloud during the interaction process. After the agent performs an interactive action in response to changes in objects in the difference point cloud, it evaluates the effectiveness of the interactive action in reducing the difference between the current scene and the target scene by calculating the amount of change in the difference point cloud before and after the interactive action is performed. The agent prioritizes candidate interactive actions applied to the differential point cloud, selects the candidate interactive action with the highest score for execution, and adjusts the action strategy based on the effectiveness evaluation results to gradually restore the scene.
2. The method according to claim 1, characterized in that, Generating the difference point cloud includes: During the roaming phase, the intelligent agent observes the target scene, acquires environmental depth observation data of the target scene, generates the first global point cloud of the target scene, and stores it as memory information; During the restoration phase, the agent observes the current scene, acquires depth observation data of the current scene, and generates a second global point cloud of the current scene. By comparing the second global point cloud with the first global point cloud, the difference point cloud is extracted.
3. The method according to claim 2, characterized in that, Based on the environmental depth observation data of the target scene, a local observation point cloud centered on the perspective of the agent is generated. The local observation point cloud is then converted into a point cloud in the global coordinate system by combining the agent's own pose information, and recorded as the first global point cloud.
4. The method according to claim 2, characterized in that, During the recovery phase, the intelligent agent alternates between exploration mode and interaction mode; In the exploration mode, the depth observation data of the current scene is converted into a point cloud in the global coordinate system and recorded as the second global point cloud; The current observation position of the second global point cloud is matched with the pose information of the first global point cloud. The second global point cloud and the first global point cloud from the corresponding viewpoint are compared to extract the disappearing point cloud clusters and the newly added point cloud clusters to form the difference point cloud. In interactive mode, after performing the interactive action, the difference point cloud in the current field of view is refreshed instead of being overlaid and updated.
5. The method according to claim 1, characterized in that, The quantitative difference points Changes in cloud cover include: Define the change in, The difference point cloud before the execution of the interactive action. For the difference point cloud after execution, |⋅| is the norm operator that quantifies the size of the difference point cloud, defined as the number of difference points in the difference point cloud; When the change is greater than a preset difference reduction threshold, the interaction action is determined to be a valid action.
6. The method according to claim 1, characterized in that, Prioritizing candidate interaction actions includes: For the newly added point cloud clusters in the difference point cloud, a scoring function is used to score the candidate interactive actions; the scoring function is: in, , As a weighting factor, Geometric matching factor As an environmental adaptation factor, For the set of moving parts, For current observation information, For the current candidate interaction action, This is for adding a new point cloud cluster.
7. The method according to claim 6, characterized in that, The geometric matching factor is calculated as follows: if the newly added point cloud cluster matches a cluster in the set of moving parts, it is determined that the object is the original object and its position has moved or its shape has changed. The geometric matching score of the discard action is set to 0, and the score of other actions is set to 1. If the newly added point cloud cluster does not match a cluster in the set of moving parts, the object is determined to be a newly appearing foreign object, the geometric matching score of the discard action is set to 1, and the scores of other actions are set to 0. The changes in the object include positional movement, shape change, and changes in the object's existence.
8. The method according to claim 6, characterized in that, The calculation method for the environmental adaptation factor includes: The newly added point cloud cluster is projected onto the RGB image at the current viewpoint to obtain a two-dimensional bounding box; The RoI pooling method is used to extract the local visual features of the region from the global feature map of the RGB image; The local visual features are processed by a multilayer perceptron, and a scalar score is output as the score of the environment adaptation factor to represent the degree of adaptation of the candidate interactive action under the current observation conditions.
9. The method according to claim 8, characterized in that, The multilayer perceptron is trained using a static dataset containing various scenarios and interactive behaviors. The training samples include local visual features, action categories, and adaptability labels. The adaptability labels are marked as 1 or 0 depending on whether the action was executed successfully or failed.
10. The method according to claim 1 or 8, characterized in that, Choose the highest rating The execution of the candidate interactive actions is high, and the action strategy is adjusted based on the effectiveness evaluation results, including: If the candidate interaction action with the highest evaluation score is effective, the current object change is marked as recovered and the next object change is processed; if it is ineffective, the candidate interaction action with the second highest score is selected for execution and its effectiveness is re-evaluated until the scene is restored or it is determined to be unrecoverable.
11. A scene restoration device based on dynamic difference point clouds, characterized in that, Include: The scene change detection module is used by the agent to acquire depth observation data of the target scene and depth observation data of the current scene, generate a difference point cloud representing the changes of objects in the current scene relative to the target scene, and dynamically update the difference point cloud during the interaction process. The interaction effect evaluation module is used to evaluate the effectiveness of the interaction action in reducing the difference between the current scene and the target scene by calculating the amount of change in the difference point cloud before and after the interaction action is performed after the agent performs the interaction action in response to the changes in the object in the difference point cloud. The trial-and-error module is used by the agent to prioritize candidate interactive actions acting on the difference point cloud, select the candidate interactive action with the highest score for execution, and adjust the action strategy based on the effectiveness evaluation results to gradually realize scene restoration.
12. A readable storage medium storing a program or instructions, characterized in that, When the program or instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-10.
13. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-10.