An intelligent wall surface fresco system and method based on multi-modal fusion positioning

Abstract

The application relates to the technical field of color drawing systems, and discloses an intelligent wall surface color drawing system and method based on multi-modal fusion positioning, which comprises the following components: module M1, an intelligent base station array; module M2, an AR intelligent operation system; module M3, a cloud management platform; and module M4, a multi-modal fusion positioning model; the method comprises the following steps: step S1, an initialization stage, in which the intelligent base station array scans a wall surface; step S2, a path planning stage, in which the cloud platform analyzes color drawing scheme data; step S3, a real-time operation stage, in which the AR system solves a pose in real time through the multi-modal fusion positioning model; and step S4, a quality control stage.The application realizes real-time high-precision pose solving, and generates a spraying track based on a wall surface point cloud map through a dynamic path planning module; the system can compensate for joint angle errors of a mechanical arm in real time, thereby effectively solving the problem that a spraying track deviates from a complex wall surface.

Description

Technical Field

[0001] This invention relates to the field of mural system technology, specifically to an intelligent wall mural system and method based on multimodal fusion positioning. Background Technology

[0002] Wall painting, as an art form, is widely used in urban promotion and interior decoration. In urban promotion, wall painting can be used to advocate civilized behavior, promote public welfare activities, and encourage healthy exercise, thereby shaping a civilized image of the city. In interior decoration, wall painting can be used to depict creative scenes on the walls according to the homeowner's personal preferences and pursuits, achieving a personalized decorative effect.

[0003] A search revealed Chinese patent CN110228298A, which discloses a wall painting device based on an automated control system. This device comprises casters, a motion device, a material conveying device, a spraying device, a drive device, and a frame. The casters are mounted on the frame at the bottom of the painting machine, allowing for easy movement. The spraying device acts as the actuator, atomizing and spraying pigments onto the wall surface. The motion device drives the spraying device in both horizontal and vertical directions, moving it from point strokes to lines and lines to surfaces, enabling the spraying device to complete the painting of the entire wall. The material conveying device continuously supplies pigments from three bottles containing different colors (the three primary colors) to the spraying device to maintain the spraying operation and complete the painting of the entire wall.

[0004] However, existing systems use RGB pigment bottles for feeding and generate colors through physical mixing. However, due to limitations in the uniformity of mechanical mixing and the precision of algorithmic control, they struggle to meet the high requirements of complex patterns for gradients and transitions. For example, the superposition of dark and light colors easily produces color differences, resulting in a lack of pattern depth. Furthermore, the motion devices of existing equipment rely on a preset coordinate system for horizontal and vertical movement, but lack the ability to dynamically perceive complex wall surfaces such as uneven surfaces, corners, and obstacles. When the wall is uneven or there are obstacles such as doors and windows, the spraying trajectory may deviate from the expected position, causing pattern misalignment or interruption. In addition, there is no compensation mechanism for the equipment's own positioning errors, such as omnidirectional wheel slippage and mechanical vibration, which can easily accumulate positioning deviations over long-term use. Therefore, this invention designs an intelligent wall painting system and method based on multimodal fusion positioning to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent wall painting system and method based on multimodal fusion positioning, which solves the problems of lack of dynamic perception of complex walls and spraying deviation in the background technology.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A smart wall painting system based on multimodal fusion positioning includes the following components:

[0008] Module M1, the intelligent base station array, consists of no fewer than four base stations deployed around the perimeter of the wall, with the spacing between base stations dynamically adjusted to within 10-30 meters based on the wall size. Each base station is equipped with a 2.4GHz / 5GHz dual-band LoRa protocol self-organizing network communication module to achieve distributed clock synchronization and dynamic power adjustment. Module M2, the AR intelligent operation system, integrates an RTK positioning unit, an IMU inertial measurement unit, and a binocular visual positioning unit. It uses a lightweight convolutional neural network built into the edge computing unit to process wall defect identification and repair scheme generation in real time, and supports AR virtual image overlay onto the real wall. Module M3, the cloud management platform, communicates with the intelligent base station array and AR system via a wireless network, stores painting scheme data, and dynamically issues control commands based on the construction progress. Module M4, the multimodal fusion positioning model, fuses RTK positioning data, IMU motion data, and visual feature point data based on extended Kalman filtering and deep learning dynamic weighted algorithms to output high-precision pose results with a positioning error of less than 2 cm.

[0009] Preferably, the intelligent base station array further includes: an environmental perception module, integrating a 16-line LiDAR and an RGB-D camera, constructing a 3D point cloud map of the wall with a 0.5° angular resolution, and marking the coordinates of concave and convex defect areas; a base station self-test program, which optimizes the base station deployment position through a greedy algorithm upon startup, while simultaneously monitoring the communication delay between base stations in real time and dynamically adjusting the LoRa transmission power; a redundant positioning module, which switches to a SLAM-based visual-inertial tightly coupled positioning mode when a single base station signal loss is detected, maintaining positioning accuracy within 5 cm; and a dynamic obstacle avoidance module, which combines LiDAR and visual data to detect obstacles in the construction area in real time, and dynamically adjusts the base station deployment position through an algorithm to avoid signal obstruction and interference.

[0010] Preferably, the AR intelligent operation system specifically includes: a high-altitude operation mode, which automatically activates a redundant data verification mechanism when the detected operation height exceeds 2 meters, increases the positioning frequency to 100Hz and enables dual IMU cross-verification; a dynamic path planning module, which generates a spraying trajectory based on the wall point cloud map and painting scheme data through the A* algorithm, and compensates for the joint angle error of the robotic arm in real time; and a spectral analysis unit, which detects the LAB color difference value between the sprayed color and the target color card in real time through a built-in spectrophotometer, and triggers closed-loop adjustment of the pigment mixing ratio when ΔE>3.

[0011] Preferably, the AR intelligent operation system further includes: a robotic arm adaptive control module, which monitors the force state of the robotic arm in real time through a six-axis torque sensor and dynamically adjusts the spraying pressure and speed in conjunction with a PID controller to ensure uniform coverage of complex walls; and a user interface that allows construction workers to view the construction progress, defect markings, and operation prompts in real time through AR glasses, and to adjust the painting scheme or pause construction through voice commands.

[0012] Preferably, the implementation of the multimodal fusion positioning model includes: a data layer fusion stage, aligning the timestamps of RTK satellite signals, IMU raw acceleration data, and visual feature point coordinates, and using a sliding window method to eliminate temporal deviations of heterogeneous data; a feature layer fusion stage, dynamically allocating fusion weights for RTK, IMU, and visual data through an attention mechanism, wherein the visual feature weight is increased to 0.6-0.8 in dynamic scenarios; a decision layer fusion stage, predicting positioning drift trends based on an LSTM network, outputting six-degree-of-freedom pose estimation results, and feeding them back to the motion control module; an environment adaptive optimization module, dynamically adjusting the data fusion strategy according to the light intensity, temperature, and humidity of the construction environment to ensure the stability of positioning accuracy under extreme conditions; and a real-time error compensation module, post-processing the positioning results through a Kalman filter, and predicting and compensating for positioning drift by combining historical data.

[0013] Preferably, the system further includes: module M5, an intelligent construction monitoring system, which monitors the construction environment in real time through an embedded camera and an infrared thermal imaging module, and uses deep learning algorithms to identify the standardization of construction personnel's actions. When abnormal behavior is detected, it automatically triggers a safety warning and suspends equipment operation; module M6, a pigment management system, which has a built-in high-precision weighing sensor and pigment mixing device, dynamically adjusts the pigment ratio according to the painting scheme, and supports the mixing of 16 basic pigments to ensure color consistency and construction efficiency.

[0014] Preferably, the intelligent construction monitoring system includes: a construction progress management module, which dynamically plans construction tasks based on Gantt charts and automatically adjusts subsequent construction plans in conjunction with real-time progress data to ensure timely project completion; a resource allocation module, which dynamically allocates base stations, robotic arms, and pigment resources according to the area and complexity of the construction area to support collaborative operation of multiple devices; and an energy consumption optimization module, which adjusts equipment power and working modes in real time through deep reinforcement learning algorithms to reduce construction energy consumption and extend equipment life.

[0015] Preferably, a method for an intelligent wall painting system based on multimodal fusion positioning includes the following steps: Step S1, initialization stage: the wall surface is scanned using an intelligent base station array, a 3D map is constructed with a point cloud density of 5cm, the coordinates of doors, windows, cracks, and obstacles are marked, and uploaded to the cloud platform; Step S2, path planning stage: the cloud platform analyzes the painting scheme data, generates a global spraying path based on the Dijkstra algorithm, and simulates the collision probability of the robotic arm movement using the Monte Carlo method; Step S3, real-time operation stage: the AR system calculates the pose in real time using a multimodal fusion positioning model, and automatically triggers the spraying pressure adjustment to the range of 0.3-0.5MPa when a local depression depth >5mm is detected on the wall surface; Step S4, quality control stage: after construction is completed, the wall surface image is acquired through the AR system and compared with the target scheme using SSIM structural similarity; when the difference area exceeds 2%, a repainting instruction is generated.

[0016] Preferably, step S3 includes the following steps: Step a, when the ambient light intensity change exceeds 2000 lux, activate the visual positioning enhancement mode and increase the sampling rate of the binocular camera to 30fps; Step b, for the gradient color area, generate the transition color spraying instruction through the HSV color space interpolation algorithm, and the single color gradient band resolution is up to 256 levels; Step c, when working in the corner area, switch to the high-precision laser radar assisted positioning mode 20cm in advance to ensure that the corner connection error is less than 1cm.

[0017] Preferably, step S4 includes the following steps: Step a, when the overall wall coverage is detected to be less than 98%, the respraying mode is automatically triggered, and the respraying path is replanned based on the point cloud data of the uncovered area; Step b, the uniformity of the sprayed color is detected by the spectral analysis unit, and when the color difference ΔE>1, the spraying pressure and pigment ratio are automatically adjusted; Step c, after the construction is completed, the wall image is collected by the AR system, and a construction quality report is generated by combining the deep learning algorithm. The report includes key indicators such as coverage, color difference value and flatness.

[0018] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0019] 1. This invention addresses the problem of spray trajectory deviation on complex walls caused by existing equipment. It achieves real-time high-precision pose calculation by combining a multimodal fusion positioning model with RTK, IMU, and visual data. The system generates the spray trajectory based on the wall point cloud map through a dynamic path planning module. The system can compensate for the joint angle error of the robotic arm in real time and switch to a high-precision lidar-assisted positioning mode in corner areas, thereby effectively solving the problem of spray trajectory deviation on complex walls.

[0020] 2. This invention addresses the shortcomings of existing systems in generating gradient and transition colors. It supports dynamic adjustment of the proportions of 16 basic pigments through a pigment management system and combines this with a spectral analysis unit to detect the color difference between the sprayed color and the target color chart in real time. When the detected color difference exceeds a threshold, the system automatically adjusts the pigment mixing ratio and spraying parameters, significantly improving the uniformity and consistency of gradient and transition colors, solving the color difference problem caused by the superposition of dark and light colors, and enhancing the layering of the pattern.

[0021] 3. This invention fuses multi-source data using extended Kalman filtering and a deep learning dynamic weighting algorithm to output a high-precision pose result with a positioning error of less than 2 cm. In dynamic scenarios, the system dynamically allocates data fusion weights through an attention mechanism and combines an LSTM network to predict positioning drift trends, ensuring improved positioning stability in complex environments. In addition, the real-time error compensation module uses a Kalman filter to combine historical data to predict and compensate for positioning drift, significantly reducing the cumulative deviation over long-term use. Attached Figure Description

[0022] Figure 1 This is a system module structure diagram of the present invention;

[0023] Figure 2 This is a flowchart illustrating the operation of the intelligent base station array of the present invention.

[0024] Figure 3 This is a flowchart of the AR intelligent operation system of the present invention;

[0025] Figure 4 This is a flowchart illustrating the multimodal fusion localization model of the present invention.

[0026] Figure 5 This is a flowchart of the intelligent construction monitoring system of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Example 1;

[0029] Please see Figures 1-5 In this embodiment of the invention, an intelligent wall painting system based on multimodal fusion positioning includes the following components:

[0030] Module M1, intelligent base station array, consists of no less than 4 base stations, deployed around the perimeter of the wall with the spacing between base stations dynamically adjusted to within 10-30 meters according to the wall size. Each base station is equipped with a 2.4GHz / 5GHz dual-band LoRa protocol self-organizing network communication module to achieve distributed clock synchronization and dynamic power adjustment.

[0031] Module M2, AR intelligent operation system, integrates RTK positioning unit, IMU inertial measurement unit and binocular vision positioning unit. It uses the lightweight convolutional neural network built into the edge computing unit to process wall defect identification and repair scheme generation in real time, and supports AR virtual image overlay onto real wall surface.

[0032] Module M3, a cloud management platform, communicates with the intelligent base station array and AR system via wireless network, stores the painting scheme data, and dynamically issues control commands based on the construction progress;

[0033] Module M4, a multimodal fusion localization model, is based on extended Kalman filtering and deep learning dynamic weighting algorithm to fuse RTK localization data, IMU motion data and visual feature point data, and outputs high-precision pose results with a localization error of less than 2 cm.

[0034] The smart base station array further includes:

[0035] The environmental perception module integrates a 16-line LiDAR and an RGB-D camera to construct a 3D point cloud map of the wall with a 0.5° angular resolution and mark the coordinates of areas with concave and convex defects. The base station self-test program optimizes the base station deployment location using a greedy algorithm upon startup, while simultaneously monitoring the communication delay between base stations and dynamically adjusting the LoRa transmission power. The redundant positioning module switches to a SLAM-based visual-inertial tightly coupled positioning mode when a single base station signal loss is detected, maintaining positioning accuracy within 5 centimeters. The dynamic obstacle avoidance module combines LiDAR and visual data to detect obstacles in the construction area in real time and dynamically adjusts the base station deployment location through algorithms to avoid signal obstruction and interference.

[0036] The AR intelligent operation system specifically includes: a high-altitude operation mode, which automatically activates a redundant data verification mechanism when the detected operation height exceeds 2 meters, increases the positioning frequency to 100Hz and enables dual IMU cross-verification; a dynamic path planning module, which generates a spraying trajectory based on the wall point cloud map and painting scheme data through the A* algorithm and compensates for the joint angle error of the robotic arm in real time; and a spectral analysis unit, which detects the LAB color difference value between the sprayed color and the target color card in real time through a built-in spectrophotometer, and triggers closed-loop adjustment of the pigment mixing ratio when ΔE>3.

[0037] The AR intelligent operation system also includes: a robotic arm adaptive control module, which monitors the force state of the robotic arm in real time through a six-axis torque sensor and dynamically adjusts the spraying pressure and speed in combination with a PID controller to ensure uniform coverage of complex walls; and a user interface, which allows construction workers to view the construction progress, defect markings and operation prompts in real time through AR glasses, and to adjust the painting scheme or pause construction through voice commands.

[0038] The implementation of the multimodal fusion positioning model includes: a data layer fusion stage, aligning the timestamps of RTK satellite signals, IMU raw acceleration data, and visual feature point coordinates, and using a sliding window method to eliminate temporal deviations of heterogeneous data; a feature layer fusion stage, dynamically allocating fusion weights for RTK, IMU, and visual data through an attention mechanism, with the visual feature weight increased to 0.6-0.8 in dynamic scenarios; a decision layer fusion stage, predicting positioning drift trends based on an LSTM network, outputting six-degree-of-freedom pose estimation results, and feeding them back to the motion control module; an environment adaptive optimization module, dynamically adjusting the data fusion strategy according to the light intensity, temperature, and humidity of the construction environment to ensure the stability of positioning accuracy under extreme conditions; and a real-time error compensation module, post-processing the positioning results using a Kalman filter, combining historical data to predict and compensate for positioning drift.

[0039] The system also includes: Module M5, an intelligent construction monitoring system, which monitors the construction environment in real time through an embedded camera and an infrared thermal imaging module, and uses deep learning algorithms to identify the standardization of construction personnel's actions. When abnormal behavior is detected, it automatically triggers a safety warning and suspends equipment operation; Module M6, a pigment management system, which has a built-in high-precision weighing sensor and pigment mixing device. It dynamically adjusts the pigment ratio according to the painting scheme and supports the mixing of 16 basic pigments to ensure color consistency and construction efficiency.

[0040] The intelligent construction monitoring system includes: a construction progress management module, which dynamically plans construction tasks based on Gantt charts and automatically adjusts subsequent construction plans in conjunction with real-time progress data to ensure timely project completion; a resource allocation module, which dynamically allocates base stations, robotic arms, and paint resources according to the area and complexity of the construction zone, supporting collaborative operation of multiple devices; and an energy consumption optimization module, which adjusts equipment power and operating modes in real time through deep reinforcement learning algorithms to reduce construction energy consumption and extend equipment life.

[0041] The working principle of this invention is as follows: An intelligent wall painting system based on multimodal fusion positioning achieves high-precision positioning and automated construction in complex scenarios through the collaborative action of modules. The intelligent base station array (module M1) first deploys no fewer than four base stations around the wall, dynamically adjusting the spacing to 10-30 meters based on the wall size. Each base station is equipped with a dual-band LoRa module to achieve self-organizing network communication, and a greedy algorithm optimizes the deployment location to ensure signal coverage ≥95%. The environmental perception module uses a 16-line LiDAR and an RGB-D camera to construct a 3D point cloud map of the wall with a 0.5° angular resolution, marking the coordinates of uneven and defective areas to provide an accurate environmental model for subsequent construction. When the base station signal is lost, the redundant positioning module switches to a SLAM-based visual-inertial tightly coupled positioning mode, maintaining positioning accuracy within 5 centimeters to ensure construction continuity.

[0042] The AR intelligent operation system (module M2) integrates RTK, IMU, and binocular vision units. Through a lightweight CNN built into the edge computing unit, it processes wall defect identification in real time and generates repair plans. In high-altitude operation mode (height ≥ 3 meters), the system automatically activates a redundant data verification mechanism, increasing the positioning frequency to 100Hz and enabling dual IMU cross-validation to ensure positioning accuracy within ±2mm, superior to traditional total stations (±5mm). The dynamic path planning module generates the spraying trajectory based on the A* algorithm and compensates for robotic arm joint angle errors in real time. Combined with the spectral analysis unit, it detects the LAB color difference between the sprayed color and the target color chart (triggers closed-loop adjustment of pigment mixing ratio when ΔE>3), achieving color consistency control.

[0043] The multimodal fusion localization model (module M4) fuses RTK, IMU, and visual data through extended Kalman filtering and a deep learning dynamic weighting algorithm, outputting high-precision pose results with a localization error of less than 2 cm. In dynamic scenes, the visual feature weights are dynamically increased to 0.6-0.8, and combined with an LSTM network to predict localization drift trends, ensuring an 80% improvement in localization robustness in complex environments. The intelligent construction monitoring system (module M5) monitors the construction environment in real time through an embedded camera and an infrared thermal imaging module. It uses deep learning algorithms to identify the standardization of construction workers' actions, automatically triggering safety warnings and suspending equipment operation when abnormal behavior is detected, ensuring construction safety.

[0044] The pigment management system (module M6) incorporates a high-precision weighing sensor and pigment mixing device, supporting dynamic ratio adjustment of 16 basic pigments. It uses a spectral analysis unit to detect the uniformity of sprayed color in real time (automatically adjusting spraying parameters when ΔE>1) to ensure color consistency and construction efficiency. The cloud management platform (module M3) communicates with each module via a wireless network, stores painting scheme data, and dynamically issues control commands based on the construction progress. It combines Gantt charts to plan construction tasks and automatically adjusts subsequent construction plans to ensure the project is completed on time.

[0045] Example 2;

[0046] Please see Figures 1-5 In this embodiment of the invention, a method for an intelligent wall painting system based on multimodal fusion positioning is provided. The method includes the following steps: Step S1, initialization stage: the wall surface is scanned by an intelligent base station array, a three-dimensional map is constructed with a point cloud density of 5cm, the coordinates of doors, windows, cracks, and obstacles are marked, and uploaded to the cloud platform; Step S2, path planning stage: the cloud platform analyzes the painting scheme data, generates a global spraying path based on the Dijkstra algorithm, and simulates the collision probability of the robotic arm movement using the Monte Carlo method; Step S3, real-time operation stage: the AR system calculates the pose in real time through the multimodal fusion positioning model. When a local depression depth on the wall surface is detected to be >5mm, the spraying pressure is automatically adjusted to the range of 0.3-0.5MPa; Step S4, quality control stage: after construction is completed, the wall surface image is acquired by the AR system and compared with the target scheme using SSIM structural similarity. When the difference area exceeds 2%, a repainting instruction is generated.

[0047] Step S3 includes the following steps: Step a, when the ambient light intensity change exceeds 2000 lux, activate the visual positioning enhancement mode and increase the sampling rate of the binocular camera to 30fps; Step b, for the gradient color area, generate the transition color spraying instruction through the HSV color space interpolation algorithm, and the single color gradient band resolution is up to 256 levels; Step c, when working in the corner area, switch to the high-precision LiDAR assisted positioning mode 20cm in advance to ensure that the corner connection error is less than 1cm.

[0048] Step S4 includes the following steps: Step a, when the overall wall coverage is detected to be less than 98%, the touch-up spraying mode is automatically triggered, and the touch-up spraying path is replanned based on the point cloud data of the uncovered area; Step b, the uniformity of the sprayed color is detected by the spectral analysis unit, and when the color difference ΔE>1, the spraying pressure and pigment ratio are automatically adjusted; Step c, after the construction is completed, the wall image is collected by the AR system, and a construction quality report is generated by combining the deep learning algorithm. The report includes key indicators such as coverage, color difference value and flatness.

[0049] The working principle of this invention is as follows: Based on multimodal fusion positioning, the intelligent wall painting method achieves fully automated construction and quality control through phased collaborative optimization. In the initialization phase (step S1), the intelligent base station array scans the wall surface with a point cloud density of 5cm, constructs a three-dimensional map, and marks the coordinates of obstacles such as doors, windows, and cracks, and uploads it to the cloud platform. In the path planning phase (step S2), the cloud platform analyzes the painting scheme data, generates a global spraying path based on the Dijkstra algorithm, and simulates the collision probability of the robotic arm movement using the Monte Carlo method to optimize path safety.

[0050] In the real-time operation phase (step S3), the AR system calculates the pose in real time using a multimodal fusion positioning model. When a local depression depth on the wall is detected to be greater than 5mm, the spraying pressure is automatically adjusted to the range of 0.3-0.5MPa. When the ambient light intensity changes by more than 2000 lux, the system activates the visual positioning enhancement mode, increasing the sampling rate of the binocular camera to 30fps to ensure positioning stability. For gradient color areas, transition color spraying instructions are generated using the HSV color space interpolation algorithm, with single-color gradients achieving a resolution of 256 color levels for fine color transitions. When operating in corner areas, the system switches to high-precision LiDAR-assisted positioning mode 20cm in advance to ensure that the corner connection error is less than 1cm.

[0051] In the quality control phase (step S4), after construction is completed, the AR system acquires wall images and compares them with the target design using SSIM structural similarity. If the difference exceeds 2%, a redraw instruction is generated. When the overall wall coverage is below 98%, a touch-up spraying mode is automatically triggered, and the touch-up spraying path is replanned based on the point cloud data of the uncovered areas. The spectral analysis unit detects the uniformity of the sprayed color; when the color difference ΔE > 1, the spraying pressure and pigment ratio are automatically adjusted to ensure the final spraying quality. After construction is completed, the system uses deep learning algorithms to generate a report containing key indicators such as coverage, color difference value, and flatness, providing data support for subsequent optimization.

[0052] A dynamic error model built using reinforcement learning algorithms analyzes operational posture, environmental disturbances, and equipment jitter data in real time, predicts path deviations, and automatically generates compensation commands, reducing the need for manual intervention. The system improves positioning stability by 80% in complex scenarios such as signal obstruction and electromagnetic interference, supports seamless mapping of curved surfaces and irregularly shaped walls, increases construction efficiency by 300% compared to traditional methods, reduces manual error correction operations by more than 90%, and is suitable for high-altitude operations (≥3 meters).

[0053] Example 3;

[0054] Please see Figures 1-5 In this embodiment of the invention, a specific implementation scheme for an intelligent wall painting system based on multimodal fusion positioning is provided, the technical details and parameter configuration of which are as follows:

[0055] The intelligent base station array deploys 4-8 base stations with dynamically adjustable spacing (10-30 meters). It is equipped with a U-blox ZED-F9P RTK module (positioning accuracy ±1cm), a 2.4GHz / 5GHz dual-band LoRa self-organizing network module (communication latency <50ms), and a 16-line LiDAR (angular resolution 0.5°, detection range 50m). The environmental perception module integrates an Intel RealSense D455 RGB-D camera (depth accuracy ±2mm@2m) to construct a 3D point cloud map of the wall (point cloud density 5cm) and mark areas with concave and convex defects (depth error <3mm).

[0056] The AR intelligent operation terminal uses HoloLens 2 AR glasses, integrating an MPU-9250IMU (100Hz sampling rate), a Leica BLK360 binocular vision module (4K resolution, 30fps), and an NVIDIA Jetson AGX Xavier edge computing unit (32TOPS computing power); the robotic arm uses the UR10e six-axis collaborative robotic arm (repeat positioning accuracy ±0.05mm), equipped with a PID controller to adjust the spraying pressure (0.3-0.5MPa) and the moving speed (0.1-1.5m / s).

[0057] In the initialization phase, wall scanning and modeling are performed. LiDAR scans the wall with a point cloud density of 5cm to generate a 3D map (including coordinates of doors, windows, and cracks). A greedy algorithm is used to optimize the base station location to ensure signal coverage ≥95%. For surface segmentation, the wall is divided into 0.5m×0.5m sub-patterns, which are classified and labeled based on the radius of curvature (>1m is a plane, <0.5m is a high curvature area) and uploaded to the cloud.

[0058] In the path planning stage, a global path is generated, and the Dijkstra algorithm is used in the cloud to plan the spraying path. Monte Carlo simulation of the robotic arm's movement trajectory (collision probability <0.1%) is used. Dynamic obstacle avoidance optimization is performed, with LiDAR detecting obstacles (such as temporary equipment) in real time and the path being dynamically adjusted through the A* algorithm, with an obstacle avoidance response time of <200ms.

[0059] During the real-time operation phase, multimodal positioning fusion is performed. RTK (10Hz), IMU (100Hz) and visual SLAM (30Hz) data are fused by extended Kalman filter (EKF) to output a six-degree-of-freedom pose (error <2mm). When operating at heights (≥3 meters), dual IMU cross-validation is activated, the positioning frequency is increased to 100Hz, and the visual weight is dynamically adjusted to 0.8.

[0060] Spraying control: The robotic arm sprays along the path. When the wall surface has a depression depth >5mm, the spraying pressure is automatically adjusted to 0.5MPa (flow control accuracy ±0.1mL / s). The gradient color area uses the HSV interpolation algorithm to generate 256 color levels. The printhead (Epson I3200-A1) outputs at a resolution of 2880dpi, and the smoothness of the transition color band is SSIM≥0.98.

[0061] During the quality control phase, color difference detection is performed using a built-in X-Rite eXact spectrophotometer to detect ΔE values ​​in real time. Areas exceeding the standard (ΔE>1) trigger closed-loop adjustment (pigment ratio is adjusted to meet the standard within 3 iterations). Coverage verification involves an AR system acquiring wall images and comparing them with the target scheme using the SSIM algorithm. When the difference area is >2%, a respraying path is generated (coverage is resprayed to ≥98%). Report generation uses a deep learning model (ResNet-50) to analyze construction data and output a quality report (including indicators such as flatness error <0.5mm and average color difference ΔE ≤1.2).

[0062] The working principle of this invention is as follows: the multimodal fusion positioning error is stabilized at ±2mm in complex scenes (electromagnetic interference, occlusion), which is 80% higher than that of traditional RTK (±10mm); in terms of construction efficiency, the robotic arm spraying speed reaches 15㎡ / hour, which is 300% higher than that of manual painting (2㎡ / day); the energy consumption optimization module reduces power consumption by 30%; the pigment dynamic ratio system achieves color uniformity of ΔE≤1.5 and supports 16-color mixing.

[0063] Working principle: The intelligent wall painting system and method based on multimodal fusion positioning achieves high-precision positioning and fully automated construction in complex scenarios through modular design and collaborative optimization. In the initialization stage, the system first scans the wall surface through an intelligent base station array to build a three-dimensional point cloud map and mark the coordinates of obstacles, providing an accurate environmental model for subsequent construction. The base station array dynamically adjusts its deployment position to optimize signal coverage and switches to a visual-inertial tightly coupled positioning mode when the signal is lost to ensure construction continuity.

[0064] During the real-time operation phase, the AR intelligent operation system combines RTK, IMU, and visual data to calculate pose in real time through a multimodal fusion positioning model, supporting high-precision positioning and spraying path planning in high-altitude operation mode. The system automatically generates repair plans based on wall defects and uses a spectral analysis unit to detect the color difference between the sprayed color and the target color card in real time, dynamically adjusting the pigment ratio to ensure color consistency. For gradient color areas, the system generates transition color spraying instructions through a color space interpolation algorithm to achieve fine color transition.

[0065] During the quality control phase, the system uses AR to acquire wall images and compares them with the target design to identify structural similarity, automatically identifying areas of difference and generating redrawing instructions. When insufficient wall coverage or color difference exceeding the threshold is detected, the system triggers a touch-up spraying mode and optimizes the touch-up spraying path. After construction is completed, the system uses deep learning algorithms to generate a construction quality report, providing data support for subsequent optimization.

[0066] Furthermore, the cloud-based management platform dynamically plans construction tasks and adjusts subsequent construction plans based on real-time progress to ensure timely project completion. The intelligent construction monitoring system monitors the construction environment and personnel behavior in real time, automatically triggering safety warnings and suspending equipment operation to ensure construction safety. Through dynamic base station tuning and algorithm optimization, the system's positioning stability in complex environments is significantly improved, supporting seamless mapping of curved surfaces and irregularly shaped walls, reducing the need for manual intervention, and significantly improving construction efficiency and quality.

[0067] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A smart wall painting system based on multimodal fusion positioning, characterized in that, Includes the following components: Module M1, intelligent base station array, consists of no less than 4 base stations, deployed around the perimeter of the wall with the spacing between base stations dynamically adjusted to within 10-30 meters according to the wall size. Each base station is equipped with a 2.4GHz / 5GHz dual-band LoRa protocol self-organizing network communication module to achieve distributed clock synchronization and dynamic power adjustment. Module M2, AR intelligent operation system, integrates RTK positioning unit, IMU inertial measurement unit and binocular vision positioning unit. It uses the lightweight convolutional neural network built into the edge computing unit to process wall defect identification and repair scheme generation in real time, and supports AR virtual image overlay onto real wall surface. Module M3, a cloud management platform, communicates with the intelligent base station array and AR system via wireless network, stores the painting scheme data, and dynamically issues control commands based on the construction progress; Module M4, a multimodal fusion localization model, is based on extended Kalman filtering and deep learning dynamic weighting algorithm to fuse RTK localization data, IMU motion data and visual feature point data, and outputs high-precision pose results with a localization error of less than 2 cm. The AR intelligent operation system specifically includes: In high-altitude operation mode, when the detected operation height exceeds 2 meters, the redundant data verification mechanism is automatically activated, the positioning frequency is increased to 100Hz and dual IMU cross-verification is enabled. The dynamic path planning module generates the spraying trajectory based on the wall point cloud map and the painting scheme data, and compensates for the joint angle error of the robotic arm in real time. The spectral analysis unit uses a built-in spectrophotometer to detect the LAB color difference between the sprayed color and the target color card in real time. When ΔE>3, it triggers closed-loop adjustment of the pigment mixing ratio.

2. The intelligent wall painting system based on multimodal fusion positioning according to claim 1, characterized in that, The intelligent base station array further includes: The environmental perception module integrates a 16-line LiDAR and an RGB-D camera to construct a 3D point cloud map of the wall with an angular resolution of 0.5° and mark the coordinates of the concave and convex defect areas. The base station self-test program optimizes the base station deployment location using a greedy algorithm upon startup, while simultaneously monitoring the communication latency between base stations in real time and dynamically adjusting the LoRa transmission power. The redundant positioning module switches to a SLAM-based visual-inertial tightly coupled positioning mode when a single base station signal loss is detected, maintaining positioning accuracy within 5 centimeters. The dynamic obstacle avoidance module combines LiDAR and visual data to detect obstacles in the construction area in real time, and dynamically adjusts the base station deployment position through algorithms to avoid signal blockage and interference.

3. The intelligent wall painting system based on multimodal fusion positioning according to claim 1, characterized in that, The AR intelligent operation system also includes: The robotic arm adaptive control module monitors the force state of the robotic arm in real time through a six-axis torque sensor, and dynamically adjusts the spraying pressure and speed in conjunction with a PID controller to ensure uniform coverage of complex walls. The user interface allows construction workers to view the construction progress, defect markings, and operation prompts in real time through AR glasses. They can also adjust the painting scheme or pause construction using voice commands.

4. The intelligent wall painting system based on multimodal fusion positioning according to claim 1, characterized in that, The implementation of the multimodal fusion localization model includes: In the data layer fusion stage, the timestamps of RTK satellite signals, IMU raw acceleration data and visual feature point coordinates are aligned, and the sliding window method is used to eliminate the timing deviation of heterogeneous data. In the feature fusion stage, the fusion weights of RTK, IMU and visual data are dynamically allocated through an attention mechanism, with the visual feature weights being increased to 0.6-0.8 in dynamic scenes; In the decision-making fusion stage, the localization drift trend is predicted based on the LSTM network, and the six-degree-of-freedom pose estimation results are output and fed back to the motion control module. The environment adaptive optimization module dynamically adjusts the data fusion strategy based on the light intensity, temperature, and humidity of the construction environment to ensure the stability of positioning accuracy under extreme conditions. The real-time error compensation module uses a Kalman filter to post-process the positioning results and combines historical data to predict and compensate for positioning drift.

5. The intelligent wall painting system based on multimodal fusion positioning according to claim 1, characterized in that, The system also includes: Module M5, the intelligent construction monitoring system, monitors the construction environment in real time through an embedded camera and an infrared thermal imaging module. It combines deep learning algorithms to identify the standardization of construction workers' actions and automatically triggers a safety warning and suspends equipment operation when abnormal behavior is detected. Module M6, the pigment management system, has a built-in high-precision weighing sensor and pigment mixing device. It dynamically adjusts the pigment ratio according to the painting scheme and supports the mixing of 16 basic pigments to ensure color consistency and construction efficiency.

6. The intelligent wall painting system based on multimodal fusion positioning according to claim 1, characterized in that, The intelligent construction monitoring system includes: The construction progress management module dynamically plans construction tasks based on Gantt charts and automatically adjusts subsequent construction plans in conjunction with real-time progress data to ensure that the project is completed on time. The resource allocation module dynamically allocates base station, robotic arm, and paint resources based on the area and complexity of the construction zone, supporting collaborative operation of multiple devices. The energy consumption optimization module uses deep reinforcement learning algorithms to adjust equipment power and operating modes in real time, thereby reducing construction energy consumption and extending equipment life.

7. A method for an intelligent wall painting system based on multimodal fusion positioning according to any one of claims 1-6, characterized in that, The method includes the following steps: Step S1, initialization stage: Scan the wall surface with a smart base station array, construct a 3D map with a point cloud density of 5cm, mark the coordinates of doors, windows, cracks and obstacles, and upload them to the cloud platform; Step S2, Path Planning Stage: The cloud platform analyzes the painting scheme data, generates a global spraying path based on the Dijkstra algorithm, and simulates the collision probability of the robotic arm movement using the Monte Carlo method. Step S3, real-time operation stage: The AR system calculates the pose in real time through the multimodal fusion positioning model. When the depth of a local depression on the wall is detected to be >5mm, the spraying pressure is automatically adjusted to the range of 0.3-0.5MPa. Step S4, quality control stage: After construction is completed, the wall image is collected through the AR system and compared with the target scheme using SSIM structural similarity. When the difference area exceeds 2%, a redraw instruction is generated.

8. The method of an intelligent wall painting system based on multimodal fusion positioning according to claim 7, characterized in that, Step S3 includes the following steps: Step a: When the ambient light intensity change exceeds 2000 lux, activate the visual positioning enhancement mode and increase the sampling rate of the binocular camera to 30fps. Step b: For the gradient color area, generate transition color spraying instructions using the HSV color space interpolation algorithm, with the single-color gradient band having a resolution of 256 color levels; Step c: When working in a corner area, switch to the high-precision lidar-assisted positioning mode 20cm in advance to ensure that the corner connection error is less than 1cm.

9. The method of an intelligent wall painting system based on multimodal fusion positioning according to claim 7, characterized in that, Step S4 includes the following steps: Step a: When the overall wall coverage is detected to be less than 98%, the respraying mode is automatically triggered, and the respraying path is replanned based on the point cloud data of the uncovered area. Step b: The uniformity of the sprayed color is detected by the spectral analysis unit. When the color difference ΔE>1, the spraying pressure and pigment ratio are automatically adjusted. Step c: After construction is completed, the wall image is collected through the AR system and a construction quality report is generated by combining it with a deep learning algorithm. The report includes key indicators such as coverage, color difference value and flatness.

Images