Scene deformation risk early warning method and device based on mobile robot

Abstract

The invention relates to a scene deformation risk early warning method and device based on a mobile robot, and the method comprises the steps: obtaining perception information uploaded by at least one mobile robot, the perception information comprises real-time three-dimensional point cloud data collected by a 3D perception sensor carried by the mobile robot and a high-precision pose of the mobile robot calculated in real time based on the same frame of point cloud data; according to the pose of each mobile robot, converting the real-time three-dimensional point cloud data of at least one mobile robot into a unified global coordinate system, and performing splicing and fusion through a point cloud registration and fusion algorithm so as to incrementally construct and dynamically update a current three-dimensional morphology model; comparing the current three-dimensional shape model with a historical reference three-dimensional shape model, and calculating the deformation quantity of the target structure body in the operation scene; and when the deformation quantity exceeds a preset safety threshold value, risk early warning information is generated and output, so that the monitoring efficiency can be improved, and real-time early warning can be realized.

Description

Technical Field

[0001] This invention relates to the field of mobile robot and industrial safety monitoring technology, and in particular to a method and apparatus for monitoring the structural health of a scene using daily operational data of a mobile robot, thereby achieving real-time early warning of deformation risks. Background Technology

[0002] In modern manufacturing, logistics, and warehousing, the structural stability of tall or complex rigid structures such as factory buildings, shelving, automated storage and retrieval systems (AS / RS), piping systems, and stacked goods (pallets) is the cornerstone of safe production. These structures may experience slow but continuous tilting, twisting, or localized deformation due to factors such as foundation settlement, long-term load, equipment impact, material fatigue, or environmental changes. Once the accumulated deformation exceeds a critical point, it can easily lead to catastrophic safety accidents such as structural instability and collapse, causing significant casualties and property damage.

[0003] Currently, safety monitoring of the aforementioned structures mainly relies on periodic manual inspections. Technicians can conduct periodic on-site measurements using specialized tools such as total stations, inclinometers, and crack detectors. However, this method is inefficient, costly, and has a long measurement cycle (usually monthly or quarterly), making it impossible to achieve real-time or near-real-time early warnings. Furthermore, inspections of high-altitude and hazardous areas themselves pose personal safety risks, and manual readings are prone to introducing errors. Summary of the Invention

[0004] (a) Technical problems to be solved In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention aims to provide a method and device for scene deformation risk early warning based on mobile robots. The core technical problem to be solved is: how to achieve high-precision, normalized, real-time (or near real-time) automated monitoring and early warning of the deformation of key structures in the entire operation scene by utilizing the data stream generated by the existing navigation sensors and algorithms of the mobile robot without adding extra dedicated hardware or interfering with its normal operation, thereby overcoming the drawbacks of low efficiency, high cost, poor real-time performance and limited coverage of the existing monitoring methods.

[0005] (II) Technical Solution To achieve the above objectives, the main technical solutions adopted by the present invention include: In a first aspect, embodiments of the present invention provide a method for scene deformation risk warning based on mobile robots, the method being executed by an edge server deployed locally in the work scene; the method includes: acquiring perception information uploaded by each of at least one mobile robot; wherein, the perception information includes real-time three-dimensional point cloud data collected by 3D perception sensors mounted on the mobile robot, and high-precision pose of the mobile robot calculated in real time using a SLAM algorithm based on the same frame of point cloud data, the pose and the three-dimensional point cloud data being encapsulated after being timestamped; based on the pose of each mobile robot, converting the real-time three-dimensional point cloud data of at least one mobile robot to a unified global coordinate system, and stitching and fusing it through a point cloud registration and fusion algorithm to incrementally construct and dynamically update the current three-dimensional topography model of the work scene; comparing the current three-dimensional topography model with a pre-stored historical baseline three-dimensional topography model established in a healthy state, and calculating the deformation of the target structure in the work scene through a deformation analysis algorithm; when the deformation exceeds a preset safety threshold, generating and outputting risk warning information.

[0006] Optionally, the real-time 3D point cloud data and its corresponding pose originate from the synchronous parallel processing flow of the same frame of sensor raw data by the local computing unit of the mobile robot. This frame of data is simultaneously input into the SLAM module of the mobile robot for real-time positioning, navigation and map updates, and is cached or copied to generate uploaded perception information.

[0007] Optionally, the process of calculating deformation includes: extracting a subset of point clouds of the same target structure from the current 3D topography model and the historical benchmark 3D topography model based on a preset monitoring target or semantic segmentation; performing fine registration on the extracted subset of point clouds to eliminate the difference in the comparison benchmark; and then calculating the change in the key geometric features of the structure represented by the subset of point clouds through a geometric model fitting algorithm; wherein, the deformation is at least one of the following: the overall tilt angle of the structure, the 3D displacement of key feature points, the deflection angle of the normal vector of the bearing plane, or the change in the surface curvature distribution.

[0008] Optionally, the target structure is a rack, automated warehouse support, stacking, piping system, or factory building structure; the key geometric features are the central axis of the column, the equation of the bearing plane, the main skeleton line of the structure, or the preset reference corner point.

[0009] Optionally, the current 3D topography model can be constructed and updated using voxel mesh fusion or surface reconstruction algorithms, with the output being a dense triangular mesh model or a high-precision point cloud model.

[0010] Optionally, the risk warning information includes multiple items such as the warning level, risk target identifier, specific deformation data, visual deformation diagram, and historical deformation trend analysis report, and is automatically pushed to the central monitoring platform or operation and maintenance system.

[0011] Secondly, embodiments of the present invention provide a scene deformation risk warning device based on a mobile robot, the device being deployed on an edge server; the device includes: The acquisition module is used to acquire the perception information uploaded by each mobile robot in at least one mobile robot; wherein, the perception information includes real-time three-dimensional point cloud data collected by the 3D perception sensor on the mobile robot, and the high-precision pose of the mobile robot calculated in real time by the SLAM algorithm based on the same frame of point cloud data, and the pose and the three-dimensional point cloud data are encapsulated after being aligned with the timestamp. The module is used to convert the real-time 3D point cloud data of at least one mobile robot to a unified global coordinate system based on the pose of each mobile robot, and to stitch and fuse the data through point cloud registration and fusion algorithms to incrementally build and dynamically update the current 3D topography model of the work scene. The comparison module is used to compare the current 3D topography model with the pre-stored historical baseline 3D topography model established in a healthy state, and calculate the deformation of the target structure in the work scene through deformation analysis algorithm; The output generation module is used to generate and output risk warning information when the deformation exceeds a preset safety threshold.

[0012] Optionally, the real-time 3D point cloud data and its corresponding pose originate from the synchronous parallel processing flow of the same frame of sensor raw data by the local computing unit of the mobile robot. This frame of data is simultaneously input into the SLAM module for real-time positioning, navigation and map updates, and is cached or copied to generate the perception information to be uploaded.

[0013] Thirdly, embodiments of this application provide a storage medium for computer-readable storage, the storage medium storing one or more programs, which can be executed by one or more processors to implement the scene deformation risk warning method based on a mobile robot according to some embodiments of the first aspect of this application.

[0014] Fourthly, embodiments of this application provide an electronic device, which includes a memory, a processor, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the scene deformation risk warning method based on a mobile robot, as described in some embodiments of the first aspect of this application.

[0015] (III) Beneficial Effects The beneficial effects of this invention are: Zero hardware additions, significant cost-effectiveness: This invention creatively reuses existing 3D perception and SLAM systems used for navigation in mobile robots as a high-precision 3D scanning and monitoring system. No additional dedicated monitoring hardware is required for the robot, achieving "zero hardware additions, significant functional expansion," greatly reducing the complexity of system deployment and the total cost of ownership (TCO).

[0016] Data source consistency ensures monitoring accuracy: Navigation and scene reconstruction are driven synchronously using the same sensor and the same data frame, fundamentally avoiding the complex spatiotemporal calibration problems and data fusion errors between multiple sensors. The high-precision pose provided by the SLAM algorithm provides reliable spatial constraints for point cloud stitching, ensuring a high degree of geometric consistency between the constructed "digital twin" model and the physical world, laying a solid data foundation for millimeter-level deformation analysis.

[0017] Real-time and efficient, enabling unmanned monitoring: Real-time data processing and analysis are performed on-site in the workshop via edge servers, overcoming network latency caused by cloud computing and achieving near real-time deformation calculation and risk warning. Monitoring tasks are seamlessly integrated into the robot's daily operation process, achieving unobtrusive and routine monitoring that is "collecting data as it works and scanning as it moves," without interfering with normal production order.

[0018] Data assetization multiplies value: The low-value data streams generated daily by mobile robot clusters, which originally only had immediate navigation value, are systematically transformed into continuously updated, high-precision "digital twin" assets that can be used for the full life cycle health management of facilities, greatly unlocking and enhancing the data value and return on investment of existing infrastructure.

[0019] Comprehensive coverage and flexible applicability: Leveraging the autonomous movement capabilities of mobile robots, the monitoring range can dynamically cover the entire factory area along the robot's work path, leaving no blind spots. The method is applicable to various visible rigid structures such as shelves, automated storage and retrieval systems, stacks of goods, pipelines, and building structures, demonstrating strong scene adaptability and broad application prospects.

[0020] Proactive safety, prevention is better than cure: Through automated continuous comparison and multi-level threshold early warning, potential hazards can be detected in the early stages of deformation and before an accident occurs, turning passive response into proactive prevention, significantly improving the overall safety management level of the factory, and completely freeing personnel from high-risk, low-efficiency repetitive labor.

[0021] To make the above-mentioned objects, features and advantages to be achieved by the embodiments of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments of this disclosure will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this disclosure and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart of a scene deformation risk warning method based on a mobile robot provided in an embodiment of this application is shown; Figure 2 The diagram shows a structural block diagram of a scene deformation risk warning device based on a mobile robot, according to an embodiment of this application. Detailed Implementation

[0024] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Currently, in addition to manual periodic inspections, the following monitoring methods also exist in existing technologies: Fixed sensor network monitoring: Inclinometers, displacement gauges, strain gauges, etc., are pre-deployed on key structures for continuous data acquisition. While this method provides a continuous data stream, it has significant drawbacks: sensor installation, wiring, and long-term maintenance costs are extremely high; monitoring point locations are fixed, and coverage is limited, making it difficult to flexibly respond to the relocation of risk points or cover all potentially risky structures in a large factory area; the system has poor scalability and flexibility.

[0026] Manual high-precision 3D scanning: This method involves manually operating a terrestrial 3D laser scanner or photogrammetry equipment to acquire a high-precision point cloud model of the scene. Deformations are detected through periodic model comparisons. This method requires expensive equipment, specialized operation, and a cumbersome and time-consuming process. It also cannot achieve routine, unmanned, and non-invasive real-time monitoring, and the scanning operation itself can interfere with normal production activities.

[0027] On the other hand, mobile robots such as Automated Guided Vehicles (AGVs) and Autonomous Mobile Robots (AMRs) have been widely used in the aforementioned industrial scenarios, performing routine tasks such as material handling, production line docking, and inventory inspection. These robots are generally equipped with 3D perception sensors such as LiDAR and depth cameras (e.g., RGB-D cameras), and utilize Simultaneous Localization and Mapping (SLAM) technology to achieve real-time localization, navigation, and obstacle avoidance. However, in the existing technological system, the environmental maps generated by robots through SLAM algorithms (mostly 2D grid maps or sparse 3D occupancy maps optimized for navigation) primarily and solely serve the robot's own movement. These maps are far from sufficient in terms of accuracy, resolution, geometric integrity, and global consistency to support millimeter-level or even centimeter-level precise deformation analysis and health diagnosis of structures such as shelf uprights and building beams.

[0028] Looking deeper, even research attempting to use data collected by mobile robots for environmental modeling often faces the following limitations: either it requires the addition of dedicated high-precision scanning equipment for monitoring tasks, increasing the cost, weight, and complexity of the robots and deviating from the original intention of utilizing existing robot assets; or it involves wirelessly transmitting all the massive amounts of raw sensor data to a remote cloud server for processing, which is limited by the bandwidth, stability, and latency of wireless networks at the factory site, making it difficult to meet the high real-time requirements of safety monitoring. Most importantly, existing technologies have failed to effectively transform the data streams continuously generated by mobile robots in their daily operations—originally used only for navigation and containing rich geometric information—into a sustainable, factory-wide, high-precision "digital twin" monitoring asset. This results in a huge waste of data value and makes the use of widely deployed mobile robot swarms for universal facility health monitoring a technological blind spot.

[0029] Based on this, this application provides a method and apparatus for scene deformation risk early warning based on mobile robots. It acquires perception information uploaded by each of at least one mobile robot. This perception information includes real-time 3D point cloud data collected by 3D perception sensors mounted on the mobile robot, and high-precision pose of the mobile robot calculated in real-time using a SLAM algorithm based on the same frame of point cloud data. The pose and 3D point cloud data are timestamped and encapsulated. Based on the pose of each mobile robot, the real-time 3D point cloud data of at least one mobile robot is converted to a unified global coordinate system and spliced ​​and fused using point cloud registration and fusion algorithms to incrementally build and dynamically update the current 3D topography model of the work scene. The current 3D topography model is compared with a pre-stored historical baseline 3D topography model established in a healthy state. Deformation analysis algorithms are used to calculate the deformation of target structures in the work scene. When the deformation exceeds a preset safety threshold, risk early warning information is generated and output. This not only improves monitoring efficiency but also enables real-time early warning and completely liberates personnel from high-risk, inefficient, repetitive labor.

[0030] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0031] Example 1

[0032] Please see Figure 1 , Figure 1 A flowchart of a scene deformation risk warning method based on a mobile robot, according to an embodiment of this application, is shown. It should be understood that this scene deformation risk warning method can be executed by a scene deformation risk warning device based on a mobile robot, and the specific device can be configured according to actual needs; this embodiment is not limited thereto. For example, the scene deformation risk warning device can be applied to an edge server. Specifically, the scene deformation risk warning method is executed by an edge server; the method includes: Step S110: Obtain the perception information uploaded by each mobile robot in at least one mobile robot. This perception information includes real-time 3D point cloud data collected by the 3D perception sensors mounted on the mobile robot, and high-precision pose of the mobile robot calculated in real-time using a SLAM algorithm based on the same frame of point cloud data. The pose and 3D point cloud data are encapsulated after being timestamped.

[0033] Specifically, when a mobile robot performs routine tasks such as material handling and inspection along a predetermined path within a factory, its onboard 3D perception sensors (such as 3D LiDAR or depth cameras) are activated and begin to work. These sensors scan the surrounding environment at a fixed frequency (such as 10Hz) and continuously output raw data frames containing depth information.

[0034] For each frame of raw sensor data acquired, the mobile robot's local computing unit processes it synchronously and in parallel to form the final uploaded perception information: Serving the robot's own navigation: The raw data of this frame is immediately input into the robot's local synchronous localization and mapping (SLAM) algorithm module. The SLAM algorithm uses this frame data to perform correlation calculations with historical data, calculates and outputs the robot's current precise pose in the global coordinate system in real time (usually including X, Y, Z three-dimensional coordinates and pitch, roll, and yaw angles), and updates the real-time map used for navigation to ensure the continuity and safety of robot operations.

[0035] Serving scene deformation monitoring: The same frame of raw data is completely cached or copied when input into the SLAM module, and undergoes necessary preprocessing (such as denoising and filtering) to generate dense or semi-dense 3D point cloud data for backend reconstruction. This point cloud data preserves the complete geometric details of the scene acquired by the sensor.

[0036] Data Association and Encapsulation: The robot's local system timestamps and logically associates the two outputs mentioned above to ensure that the pose calculated by SLAM strictly corresponds to the same frame of 3D point cloud data on which the pose was generated. Subsequently, this pose-point cloud data pair is encapsulated into a complete perceptual information.

[0037] Data Upload: Finally, the mobile robot uploads this encapsulated sensing information to an edge server deployed locally in the factory via its wireless communication module (such as a 5G module or Wi-Fi module). This process repeats throughout the robot's operation, forming a continuous, stable, and low-latency data stream, providing raw materials for backend modeling.

[0038] Step S120: Based on the pose of each mobile robot, the real-time 3D point cloud data of at least one mobile robot is converted to a unified global coordinate system, and then stitched and fused using point cloud registration and fusion algorithms to incrementally build and dynamically update the current 3D shape model of the work scene.

[0039] Specifically, after receiving the perception information streams from each mobile robot, the edge server activates the distributed high-precision 3D reconstruction engine. The core steps of this process are as follows: Coordinate System 1 and Initial Alignment: Based on the high-precision pose provided by the robot's SLAM algorithm associated with each frame of point cloud data, a rigid coordinate transformation matrix is ​​used to transform these scattered local point clouds, which were originally referenced by their respective sensor coordinate systems, into a predefined, fixed global world coordinate system. This coordinate system is usually consistent with the coordinate system of the factory's CAD drawings or BIM model, facilitating subsequent comparison and analysis.

[0040] Fine-grained point cloud registration: Due to potential cumulative drift or instantaneous errors in SLAM pose estimation, the initially aligned point cloud may exhibit slight misalignment. Therefore, the system invokes point cloud registration algorithms (such as the iterative nearest point algorithm and its variants, and feature-based registration algorithms) to further refine the alignment of point cloud frames with overlapping areas in time and space. This step aims to eliminate pose errors and ensure that data acquired at different times, by different robots, and from different perspectives achieves sub-centimeter or even millimeter-level spatial consistency in the global coordinate system.

[0041] Point cloud fusion and model update: Using aligned point clouds, the system applies point cloud fusion techniques (such as voxel meshing combined with the moving cube algorithm, Poisson surface reconstruction, or a simple weighted average method) to fuse multiple observations of the same physical surface. This not only generates a single dense triangular mesh model or a high-precision point cloud model with clearer structure and lower noise, but also reduces occlusion and shadow areas through fusion. This model is incrementally and dynamically updated: as new sensing information continuously flows in, the system fuses new data into the existing model in a "frame-model" or "subgraph-model" manner, expanding the coverage area, refreshing the details of existing areas, and optimizing the overall accuracy of the model in real time.

[0042] Model Output: The final constructed 3D topographic model is a high-fidelity scene "digital twin" with geometric precision sufficient to reflect details such as the verticality of shelf uprights, the straightness of beams, and the flatness of the ground. It is completely different from the highly simplified 2D grid maps or sparse 3D occupancy maps used in robot navigation for the sake of real-time performance, providing a unique, reliable, and continuously updated geometric benchmark for subsequent precise deformation detection.

[0043] Step S130: Compare the current 3D topography model with the pre-stored historical baseline 3D topography model to calculate the deformation of the target structure in the work scene.

[0044] Specifically, deformation calculation is a refined analytical process: Model Alignment and Target Extraction: First, the current 3D topography model generated in step S120 is coarsely aligned with the historical baseline 3D topography model retrieved from the database, established during the system's initial healthy state or at a certified safe moment (e.g., by selecting fixed feature points in the scene that have not undergone deformation). Then, based on a predefined list of monitoring targets (such as shelf IDs, automated storage and retrieval system support numbers) or through automated semantic / instance segmentation algorithms, a subset of 3D point clouds or a set of triangular faces representing the same physical target structure (such as "Warehouse B-05 Shelf") is accurately extracted from the current model and the historical baseline model, respectively.

[0045] Fine-grained registration and change detection: For the extracted paired point cloud subsets, a high-precision geometric alignment algorithm is applied. This algorithm first performs fine-grained registration, using only assumed undeformed or negligibly deformed portions of the target structure (such as the fixed base at the bottom of a shelf) to optimally align the models from the two periods, thus isolating overall pose differences. Subsequently, the algorithm performs deformation calculations: Feature-based analysis: By using geometric model fitting algorithms (such as RANSAC for fitting planes, cylinders, and lines), changes in key geometric features are calculated and compared. For example, the central axis of a shelf upright is fitted, and its tilt angle is calculated; the plane supporting the pallet on the shelf is fitted, and its normal vector direction is calculated (i.e., plane tilt); the three-dimensional coordinates of the shelf corner points are extracted, and their displacement vectors are calculated.

[0046] Analysis based on distance field: Calculate the distance from the surface of the historical benchmark model to the nearest point of the current model, and generate a displacement field or distance map to intuitively display the magnitude and direction of deformation in each region.

[0047] Deformation Quantization: The above analysis will output one or more quantifiable deformation variables. These quantities may be: Overall deformation: such as overall tilt angle (degrees) and center of gravity offset (millimeters).

[0048] Local deformation: such as the displacement (in millimeters) of key feature points (column base, connection) in the XYZ directions.

[0049] Surface deformation: such as changes in surface curvature and flatness error (millimeters).

[0050] Ultimately, this step generates a structured deformation analysis report that clearly indicates "which structure, which part, what type of deformation, and what the specific value is," thus realizing the transformation from massive point cloud data to intuitive engineering safety indicators.

[0051] Step S140: When the deformation exceeds the preset safety threshold, generate and output risk warning information.

[0052] Specifically, this step is the risk decision-making and response phase: Threshold determination: The system compares the various deformations calculated in step S130 with the multi-level safety thresholds preset for this type of structure. These thresholds are set based on engineering specifications, structural design parameters, and historical safety data, and may include "attention thresholds", "early warning thresholds", and "danger thresholds".

[0053] Warning Information Generation: Once any deformation exceeds the corresponding threshold, the system will immediately and automatically trigger the warning process. First, a structured Level 1 warning message will be generated, which will include at least the following: warning level (e.g., attention, warning, severe), risk target identifier (e.g., the 5th row of shelves in Zone C), specific risk location (east side of the third beam from the top), description of the exceeding deformation ("Eastward tilt angle reaches 2.5°, exceeding the warning threshold of 1.5°"), and timestamp.

[0054] Diagnostic Report Generation: Simultaneously, the system automatically generates a detailed, multi-dimensional diagnostic report for in-depth analysis and decision support. This report typically includes: Visual comparison chart: The deformation area and direction are highlighted on the 3D model using heat maps, vector arrows, etc.

[0055] Quantitative data table: Lists the deformation measurement values, corresponding thresholds, and the proportion of exceeding limits for all monitoring points.

[0056] Historical trend curve: Shows the trend of key deformation parameters of the structure over time, helping to determine whether the deformation is gradual or sudden.

[0057] Correlation analysis: It may correlate with robot operation records, temperature and humidity sensor data, etc. in the same area during the same period to help analyze the causes of deformation.

[0058] Preliminary handling recommendations: Based on the preset rule base, recommendations are automatically generated (such as "It is recommended to suspend the use of this shelf and conduct manual verification" or "It is recommended to increase the frequency of inspections in this area in the next 24 hours").

[0059] Information Output and Linkage: Finally, the system automatically pushes early warning information and diagnostic reports to the factory's central monitoring system (SCADA / MES), the mobile terminals of relevant management personnel (APP / SMS), and the maintenance work order system through standard interfaces (such as API, message queue). This ensures that risk information is delivered to the responsible parties through multiple channels in the first instance, thereby driving the immediate initiation of subsequent safety measures such as inspection, maintenance, and area lockdown, forming a complete closed loop for risk early warning and handling.

[0060] Therefore, this application embodiment obtains the perception information uploaded by each of at least one mobile robot. The perception information includes real-time 3D point cloud data collected by the 3D perception sensor on the mobile robot, and the high-precision pose of the mobile robot calculated in real time by the SLAM algorithm based on the same frame of point cloud data. The pose and 3D point cloud data are encapsulated after being aligned with timestamps. Based on the pose of each mobile robot, the real-time 3D point cloud data of at least one mobile robot is converted to a unified global coordinate system and spliced ​​and fused by point cloud registration and fusion algorithms to incrementally build and dynamically update the current 3D topography model of the work scene. The current 3D topography model is compared with the pre-stored historical baseline 3D topography model established in a healthy state. The deformation analysis algorithm is used to calculate the deformation of the target structure in the work scene. When the deformation exceeds a preset safety threshold, risk warning information is generated and output. This not only improves monitoring efficiency but also enables real-time warning and completely liberates personnel from high-risk, inefficient, repetitive labor.

[0061] It should be understood that the above-described method for early warning of scene deformation risks based on mobile robots is merely exemplary. Those skilled in the art can make various modifications based on the above method, and the modified solutions also fall within the protection scope of this application.

[0062] Example 2

[0063] Please see Figure 2 , Figure 2 This diagram illustrates a structural block diagram of a scene deformation risk warning device 200 based on a mobile robot, according to an embodiment of this application. It should be understood that the scene deformation risk warning device 200 is capable of executing the steps described in the above method embodiments. The specific functions of the scene deformation risk warning device 200 can be found in the description above; to avoid repetition, detailed descriptions are omitted here. The scene deformation risk warning device 200 includes at least one software function module that can be stored in a memory or embedded in the operating system (OS) of the scene deformation risk warning device 200 in the form of software or firmware. Specifically, the scene deformation risk warning device 200 is applied to an edge server; the scene deformation risk warning device 200 includes: The acquisition module 210 is used to acquire the perception information uploaded by each mobile robot in at least one mobile robot; wherein, the perception information includes real-time three-dimensional point cloud data collected by the 3D perception sensor mounted on the mobile robot, and the high-precision pose of the mobile robot calculated in real time by the SLAM algorithm based on the same frame of point cloud data, and the pose and the three-dimensional point cloud data are encapsulated after being aligned with the timestamp. The construction and update module 220 is used to convert the real-time 3D point cloud data of at least one mobile robot to a unified global coordinate system based on the pose of each mobile robot, and to stitch and fuse the data through point cloud registration and fusion algorithms to incrementally build and dynamically update the current 3D topography model of the work scene. The comparison module 230 is used to compare the current three-dimensional topography model with the pre-stored historical baseline three-dimensional topography model established in a healthy state, and calculate the deformation of the target structure in the work scene through deformation analysis algorithm. The output generation module 240 is used to generate and output risk warning information when the deformation exceeds the preset safety threshold.

[0064] Optionally, the real-time 3D point cloud data and its corresponding pose originate from the synchronous parallel processing flow of the same frame of sensor raw data by the local computing unit of the mobile robot. This frame of data is simultaneously input into the SLAM module for real-time positioning, navigation and map updates, and is cached or copied to generate the perception information to be uploaded.

[0065] Since the apparatus described in the above embodiments of the present invention is an apparatus used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the methods described in the above embodiments of the present invention, and therefore will not be described again here. All apparatuses used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0066] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0068] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0069] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0070] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0071] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A mobile robot-based scene morphing risk early warning method, characterized in that, The method is executed by an edge server deployed locally in a work scene; the method comprises: obtaining perception information uploaded by each of at least one mobile robot; wherein the perception information comprises real-time three-dimensional point cloud data collected by a 3D perception sensor carried by the mobile robot, and a high-precision pose of the mobile robot calculated in real time based on the same frame of point cloud data by a SLAM algorithm, which is encapsulated after the three-dimensional point cloud data and the pose are timestamped; converting the real-time three-dimensional point cloud data of the at least one mobile robot into a unified global coordinate system according to the pose of each mobile robot, and splicing and fusing by a point cloud registration and fusion algorithm to incrementally build and dynamically update a current three-dimensional topography model of the work scene; comparing the current three-dimensional topography model with a historical baseline three-dimensional topography model established under a healthy state, and calculating a deformation amount of a target structure in the work scene by a deformation analysis algorithm; when the deformation amount exceeds a preset safety threshold, generating and outputting risk warning information.

2. The method of claim 1, wherein, The real-time three-dimensional point cloud data and the corresponding pose originate from a synchronous parallel processing flow of a local computing unit of the mobile robot on the same frame of sensor raw data, wherein the frame of data is simultaneously input into a SLAM module of the mobile robot for real-time positioning and navigation and map updating, and is cached or copied for generating the uploaded perception information.

3. The method of claim 1, wherein, The process of calculating the deformation amount comprises: extracting point cloud subsets of the same target structure from the current three-dimensional topography model and the historical baseline three-dimensional topography model based on a preset monitoring target or semantic segmentation; performing fine registration on the extracted point cloud subsets to eliminate differences in comparison reference, and then calculating a change amount of a key geometric feature of a structure represented by the point cloud subsets by a geometric model fitting algorithm; wherein the deformation amount is at least one of a whole structure inclination angle, a three-dimensional displacement of a key feature point, a bearing plane normal vector deflection angle, or a surface curvature distribution change.

4. The method of claim 3, wherein, The target structure is a shelf, a rack support, a stack, a pipeline system, or a plant building structure; and the key geometric feature is a center axis of a column, an equation of a bearing plane, a main skeleton line of a structure, or a preset reference corner point.

5. The method of claim 1, wherein, The current three-dimensional topography model is built and updated by a voxel grid fusion or surface reconstruction algorithm, and is output as a dense triangular mesh model or a high-precision point cloud model.

6. The method of claim 1, wherein, The risk warning information comprises multiple items such as a warning level, a risk target identification, specific deformation data, a visual deformation graph, and a historical deformation trend analysis report, and is automatically pushed to a central monitoring platform or an operation and maintenance system.

7. A mobile robot-based scene morphing risk early warning device, characterized in that, The device is deployed in an edge server; the device comprises: An acquisition module is configured to acquire perception information uploaded by each mobile robot of at least one mobile robot; wherein the perception information comprises real-time three-dimensional point cloud data collected by a 3D perception sensor carried by the mobile robot, and a high-precision pose of the mobile robot calculated in real time based on the same frame of point cloud data by a SLAM algorithm, and the pose is encapsulated after being timestamped with the three-dimensional point cloud data; A construction update module is configured to convert the real-time three-dimensional point cloud data of the at least one mobile robot to a unified global coordinate system according to the pose of each mobile robot, and splice and fuse the real-time three-dimensional point cloud data by a point cloud registration and fusion algorithm, so as to incrementally construct and dynamically update a current three-dimensional topography model of the working scene; A comparison module is configured to compare the current three-dimensional topography model with a historical reference three-dimensional topography model established in a healthy state, and calculate a deformation amount of a target structure in the working scene by a deformation analysis algorithm; An output generation module is configured to generate and output risk early warning information when the deformation amount exceeds a preset safety threshold.

8. The apparatus of claim 7, wherein, The real-time three-dimensional point cloud data and the corresponding pose are derived from a synchronous parallel processing procedure of a local computing unit of the mobile robot on the same frame of sensor original data, wherein the frame of data is simultaneously input into a SLAM module for real-time positioning, navigation and map updating, and is buffered or copied for generating the uploaded perception information.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the mobile robot-based scene deformation risk early warning method according to any one of claims 1 to 6.

10. An electronic device comprising a processor and a memory, the memory storing a computer program, characterized in that, The processor executes the program to implement the mobile robot-based scene deformation risk early warning method according to any one of claims 1 to 6.

Images