Intelligent wall surface colored drawing system and method based on multi-modal fusion positioning

Abstract

The invention relates to the technical field of colored drawing systems, and discloses an intelligent wall surface colored drawing system and method based on multi-modal fusion positioning, and the system comprises the following components: a module M1 and an intelligent base station array; the module M2 is an AR intelligent operation system; the module M3 is a cloud management platform; a module M4 which is a multi-modal fusion positioning model; the method comprises the following steps: S1, in an initialization stage, scanning a wall surface through an intelligent base station array; s2, in a path planning stage, the cloud platform analyzes the colored drawing scheme data; s3, in a real-time operation stage, the AR system calculates the pose in real time through a multi-mode fusion positioning model; and S4, a quality control stage. According to the invention, real-time high-precision pose calculation is realized, and a spraying track is generated based on a wall point cloud map through a dynamic path planning module; and the system can compensate the angle error of the mechanical arm joint in real time, so that the problem of deviation of a complex wall surface spraying track is effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of painting systems, and in particular to an intelligent wall painting system and method based on multimodal fusion positioning. Background Art

[0002] As an art form, wall painting is widely used in urban promotion and interior decoration. In urban promotion, wall painting can be used to promote civilized behavior, public welfare activities, and encourage healthy exercise, thereby shaping the image of a civilized city. In interior decoration, wall painting can be used to create creative scenes based on the homeowner's personal preferences and pursuits, achieving a personalized decorative effect.

[0003] A search revealed Chinese patent number CN110228298A, which discloses a wall painting device based on an automated control system. The device comprises universal wheels, a motion device, a feed device, a spray device, a drive device, and a frame. The universal wheels are mounted on the frame at the bottom of the painting machine, allowing for easy movement. The spray device acts as an actuator, atomizing the paint and spraying it onto the wall. The motion device drives the spray device horizontally and vertically, creating a line and a surface from point to point, allowing the spray device to paint the entire wall. The feed device continuously delivers paint from three paint bottles containing different colors (the three primary colors) to the spray device, maintaining the spraying process and ultimately completing the entire wall painting.

[0004] However, the existing system uses a feeding device for three primary colors (RGB) paint bottles to achieve color generation through physical mixing. However, due to the limitations of the uniformity of mechanical mixing and the accuracy of algorithm control, it is difficult to meet the high requirements of complex patterns for gradient and transition colors. For example, the superposition of dark and light colors is prone to color difference, resulting in a lack of pattern layering. In addition, the motion device of the existing equipment relies on a preset coordinate system to achieve horizontal and vertical movement, but lacks the dynamic perception capability of complex wall surfaces, such as concave and convex surfaces, corners, and obstacles. When the wall is uneven or there are obstacles such as doors and windows, the spraying trajectory may deviate from the expected position, resulting in pattern misalignment or interruption. In addition, there is no compensation mechanism for the positioning error of the equipment itself, such as universal wheel sliding and mechanical vibration, and long-term use is prone to accumulated positioning deviations. Based on this, the present invention designs an intelligent wall painting system and method based on multimodal fusion positioning to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent wall painting system and method based on multimodal fusion positioning, which solves the problem of lack of dynamic perception of complex wall surfaces and spraying deviation in the background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] An intelligent wall painting system based on multimodal fusion positioning includes the following components:

[0008] Module M1, the intelligent base station array, consists of no less than 4 base stations, which are deployed around the wall and the distance between the base stations is dynamically adjusted to a range of 10-30 meters according to the wall size. Each base station is equipped with a 2.4GHz / 5GHz dual-band LoRa protocol self-organizing network communication module to achieve distributed clock synchronization and dynamic power adjustment; module M2, the AR intelligent operation system, integrates the RTK positioning unit, IMU inertial measurement unit and binocular vision positioning unit, and uses the lightweight convolutional neural network built into the edge computing unit to process wall defect identification and repair plan generation in real time, and supports AR virtual images to be superimposed on the real wall; module M3, the cloud management platform, communicates with the intelligent base station array and AR system through a wireless network, stores painting plan data and dynamically issues control instructions based on the construction progress; module M4, the multimodal fusion positioning model, fuses RTK positioning data, IMU motion data and visual feature point data based on the extended Kalman filter and deep learning dynamic weighted algorithm, and outputs high-precision posture results with a positioning error of less than 2 cm.

[0009] Preferably, the intelligent base station array further includes: an environmental perception module, which integrates a 16-line laser radar and an RGB-D camera, constructs a three-dimensional point cloud map of the wall with an angular resolution of 0.5°, and marks the coordinates of the concave and convex defect areas; a base station self-test program, which optimizes the base station deployment position through a greedy algorithm when it is started, while monitoring the communication delay between base stations in real time and dynamically adjusting the LoRa transmission power; a redundant positioning module, which switches to the SLAM-based visual-inertial tightly coupled positioning mode when it detects that a single base station signal is lost, maintaining the positioning accuracy within 5 cm; a dynamic obstacle avoidance module, which combines laser radar and visual data to detect obstacles in the construction area in real time, and dynamically adjusts the base station deployment position through an algorithm to avoid signal blockage and interference.

[0010] Preferably, the AR intelligent operation system specifically includes: a high-altitude operation mode, when it is detected that the operation height exceeds 2 meters, the redundant data verification mechanism is automatically activated, the positioning frequency is increased to 100Hz and dual IMU cross-validation is enabled; a dynamic path planning module, based on the wall point cloud map and painting scheme data, generates a spraying trajectory through the A* algorithm, and compensates for the robot arm joint angle error in real time; a spectral analysis unit, which detects the LAB color difference value between the spray color and the target color card in real time through the built-in spectrophotometer, and triggers the closed-loop adjustment of the pigment mixing ratio when ΔE>3.

[0011] Preferably, the AR intelligent operation system also includes: a robotic arm adaptive control module, which monitors the stress state of the robotic arm in real time through a six-axis torque sensor, and dynamically adjusts the spraying pressure and speed in combination with a PID controller to ensure uniform coverage of complex wall surfaces; a user interaction interface, which supports construction personnel to view construction progress, defect marking and operation prompts in real time through AR glasses, and can adjust the painting plan or pause construction through voice commands.

[0012] Preferably, the implementation of the multimodal fusion positioning model includes: in the data layer fusion stage, the timestamps of the RTK satellite signal, IMU raw acceleration data and visual feature point coordinates are aligned, and the sliding window method is used to eliminate the timing deviation of heterogeneous data; in the feature layer fusion stage, the fusion weights of RTK, IMU and visual data are dynamically allocated through the attention mechanism, wherein the weight of visual features in dynamic scenes is increased to 0.6-0.8; in the decision layer fusion stage, the positioning drift trend is predicted based on the LSTM network, and the six-degree-of-freedom pose estimation result is output and fed back to the motion control module; the environmental adaptive optimization module dynamically adjusts the data fusion strategy according to the light intensity, temperature and humidity of the construction environment to ensure the stability of positioning accuracy under extreme conditions; the real-time error compensation module post-processes the positioning results through the Kalman filter, and predicts and compensates for positioning drift in combination with historical data.

[0013] Preferably, the system also includes: module M5, an intelligent construction monitoring system, which monitors the construction environment in real time through embedded cameras and infrared thermal imaging modules, combines deep learning algorithms to identify the standardization of construction workers' actions, and automatically triggers a safety warning and suspends equipment operation when abnormal behavior is detected; module M6, a pigment management system, with built-in high-precision weighing sensors and pigment mixing devices, dynamically adjusts the pigment ratio according to the painting scheme, supports the mixing of 16 basic pigments, and is used to ensure color consistency and construction efficiency.

[0014] Preferably, the intelligent construction monitoring system includes: a construction progress management module, which dynamically plans construction tasks based on a Gantt chart and automatically adjusts subsequent construction plans in combination with real-time progress data to ensure that the project is completed on time; a resource allocation module, which dynamically allocates base stations, robotic arms and pigment resources according to the area and complexity of the construction area, and supports collaborative operation of multiple devices; an energy consumption optimization module, which adjusts equipment power and working mode in real time through a deep reinforcement learning algorithm to reduce construction energy consumption and extend equipment life.

[0015] Preferably, a method of an intelligent wall painting system based on multimodal fusion positioning includes the following steps: step S1, an initialization stage, scanning the wall through an intelligent base station array, constructing a three-dimensional map with a point cloud density of 5cm, marking the coordinates of doors, windows, cracks and obstacles and uploading them to a cloud platform; step S2, a path planning stage, the cloud platform parses the painting scheme data, generates a global spraying path based on the Dijkstra algorithm, and simulates the collision probability of the robot arm movement through the Monte Carlo method; step S3, a real-time operation stage, the AR system solves the posture in real time through the multimodal fusion positioning model, and automatically triggers the spraying pressure to be adjusted to the range of 0.3-0.5MPa when it is detected that the depth of the local depression on the wall is greater than 5mm; step S4, a quality control stage, after the construction is completed, the wall image is collected through the AR system and compared with the target scheme for SSIM structural similarity, and a repainting instruction is generated when the difference area exceeds 2%.

[0016] Preferably, step S3 includes the following steps: step a, when it is recognized that the ambient light intensity changes by more than 2000 lux, starting the visual positioning enhancement mode and increasing the binocular camera sampling rate to 30fps; step b, for the gradient color area, generating transition color spraying instructions through the HSV color space interpolation algorithm, and the monochrome gradient band resolution is increased to 256 levels; step c, when working in the corner area, switching to the high-precision lidar assisted positioning mode 20 cm in advance to ensure that the corner connection error is less than 1 cm.

[0017] Preferably, step S4 includes the following steps: step a, when it is detected that the overall wall coverage is lower than 98%, the re-spraying mode is automatically triggered, and the re-spraying path is re-planned based on the point cloud data of the uncovered area; step b, the uniformity of the spraying color is detected by the spectral analysis unit, and when the color difference ΔE>1, the spraying pressure and pigment ratio are automatically adjusted; step c, after the construction is completed, the wall image is collected by the AR system, and a construction quality report is generated in combination with the deep learning algorithm, and the report content includes key indicators of coverage, color difference value and flatness.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] 1. To address the problem of spray trajectory deviation of existing equipment on complex wall surfaces, the present invention uses a multimodal fusion positioning model combined with RTK, IMU and visual data to achieve real-time high-precision pose solution, and generates a spray trajectory based on the wall point cloud map through a dynamic path planning module; the system can compensate for the robot arm joint angle error in real time and switch to a high-precision lidar-assisted positioning mode in the corner area, thereby effectively solving the problem of spray trajectory deviation on complex wall surfaces.

[0020] 2. To address the shortcomings of existing systems in generating gradient and transition colors, the present invention supports dynamic ratio adjustment of 16 basic pigments through a pigment management system, and combines a spectral analysis unit to detect the color difference between the spray color and the target color card in real time. When the color difference is detected to exceed the threshold, the system automatically adjusts the pigment mixing ratio and spraying parameters, significantly improving the uniformity and consistency of gradient and transition colors, solving the color difference problem caused by the superposition of dark and light colors, and enhancing the layering of the pattern.

[0021] 3. The present invention fuses multi-source data through an extended Kalman filter and a deep learning dynamic weighted algorithm to output high-precision posture results with a positioning error of less than 2 cm. In dynamic scenarios, the system dynamically allocates data fusion weights through the attention mechanism and combines it with the LSTM network to predict positioning drift trends, ensuring improved positioning stability in complex environments. In addition, the real-time error compensation module uses the Kalman filter combined with historical data to predict and compensate for positioning drift, significantly reducing the accumulated deviation in long-term use. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a system module structure diagram of the present invention;

[0023] Figure 2 This is a working diagram of the intelligent base station array of the present invention;

[0024] Figure 3 This is a workflow diagram of the AR intelligent operation system of the present invention;

[0025] Figure 4 This is a workflow diagram of the multimodal fusion positioning model of the present invention;

[0026] Figure 5 This is a workflow diagram of the intelligent construction monitoring system of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] Example 1;

[0029] See also Figure 1-Figure 5 In an embodiment of the present invention, an intelligent wall painting system based on multimodal fusion positioning includes the following components:

[0030] Module M1, an intelligent base station array, consists of no less than four base stations, deployed around the wall, with the base station spacing dynamically adjusted to a range of 10-30 meters based on the wall size. Each base station is equipped with a 2.4GHz / 5GHz dual-band LoRa protocol self-organizing network communication module to achieve distributed clock synchronization and dynamic power regulation;

[0031] Module M2, an AR intelligent operation system, integrates an RTK positioning unit, an IMU inertial measurement unit, and a binocular vision positioning unit. It uses a lightweight convolutional neural network built into the edge computing unit to process wall defect identification and repair solution generation in real time, and supports the overlay of AR virtual images onto real walls.

[0032] Module M3, a cloud-based management platform, communicates with the intelligent base station array and AR system via a wireless network, stores painting scheme data, and dynamically issues control instructions based on the construction progress;

[0033] Module M4, a multimodal fusion positioning model, fuses RTK positioning data, IMU motion data, and visual feature point data based on an extended Kalman filter and a deep learning dynamic weighted algorithm, outputting high-precision pose results with a positioning error of less than 2 cm.

[0034] The intelligent base station array further includes:

[0035] The environmental perception module integrates a 16-line lidar and an RGB-D camera to construct a three-dimensional point cloud map of the wall with an angular resolution of 0.5° and mark the coordinates of concave and convex defect areas; the base station self-test program optimizes the base station deployment position through a greedy algorithm at startup, while also monitoring the communication delay between base stations in real time and dynamically adjusting the LoRa transmission power; the redundant positioning module switches to the SLAM-based visual-inertial tightly coupled positioning mode when it detects the loss of a single base station signal, maintaining a positioning accuracy within 5 cm; the dynamic obstacle avoidance module combines lidar and visual data to detect obstacles in the construction area in real time, and dynamically adjusts the base station deployment position through an algorithm to avoid signal blockage and interference.

[0036] The AR intelligent operation system specifically includes: high-altitude operation mode, when it detects that the operation height exceeds 2 meters, it automatically activates the redundant data verification mechanism, increases the positioning frequency to 100Hz and enables dual IMU cross-validation; dynamic path planning module, based on the wall point cloud map and painting scheme data, generates the spraying trajectory through the A* algorithm, and compensates for the robot arm joint angle error in real time; spectral analysis unit, through the built-in spectrophotometer, real-time detection of the LAB color difference value between the spray color and the target color card, triggering the closed-loop adjustment of the pigment mixing ratio when ΔE>3.

[0037] The AR intelligent operation system also includes: a robotic arm adaptive control module, which monitors the stress status of the robotic arm in real time through a six-axis torque sensor, and dynamically adjusts the spraying pressure and speed in combination with a PID controller to ensure uniform coverage of complex wall surfaces; a user interaction interface that supports construction workers to view construction progress, defect annotations and operation prompts in real time through AR glasses, and can also adjust the painting plan or pause construction through voice commands.

[0038] The implementation of the multimodal fusion positioning model includes: in the data layer fusion stage, the timestamps of RTK satellite signals, IMU raw acceleration data and visual feature point coordinates are aligned, and the sliding window method is used to eliminate the timing deviation of heterogeneous data; in the feature layer fusion stage, the fusion weights of RTK, IMU and visual data are dynamically allocated through the attention mechanism, among which the weight of visual features in dynamic scenes is increased to 0.6-0.8; in the decision layer fusion stage, the positioning drift trend is predicted based on the LSTM network, and the six-degree-of-freedom pose estimation result is output and fed back to the motion control module; the environmental adaptive optimization module dynamically adjusts the data fusion strategy according to the light intensity, temperature and humidity of the construction environment to ensure the stability of positioning accuracy under extreme conditions; the real-time error compensation module post-processes the positioning results through the Kalman filter, combines historical data to predict and compensate for positioning drift.

[0039] The system also includes: Module M5, an intelligent construction monitoring system, which monitors the construction environment in real time through embedded cameras and infrared thermal imaging modules, combines deep learning algorithms to identify the standardization of construction workers' movements, and automatically triggers safety warnings and suspends equipment operation when abnormal behavior is detected; Module M6, a pigment management system, has built-in high-precision weighing sensors and pigment mixing devices, dynamically adjusts the pigment ratio according to the painting scheme, and supports the mixing of 16 basic pigments to ensure color consistency and construction efficiency.

[0040] The intelligent construction monitoring system includes: a construction progress management module, which dynamically plans construction tasks based on Gantt charts and automatically adjusts subsequent construction plans based on real-time progress data to ensure that projects are completed on time; a resource allocation module, which dynamically allocates base stations, robotic arms, and paint resources based on the area and complexity of the construction area, supporting multi-device collaborative operations; and an energy consumption optimization module, which uses deep reinforcement learning algorithms to adjust equipment power and operating modes in real time to reduce construction energy consumption and extend equipment life.

[0041] The working principle of the embodiment of the present invention is: an intelligent wall painting system based on multimodal fusion positioning realizes high-precision positioning and automated construction in complex scenes through the synergy between modules. The intelligent base station array (module M1) first deploys no less than 4 base stations around the wall, and dynamically adjusts the spacing to 10-30 meters according to the wall size. Each base station is equipped with a dual-band LoRa module to realize self-organizing network communication, and optimizes the deployment position through a greedy algorithm to ensure signal coverage ≥ 95%. The environmental perception module uses a 16-line laser radar and an RGB-D camera to construct a three-dimensional point cloud map of the wall with an angular resolution of 0.5°, marking the coordinates of the concave and convex defect areas, and providing an accurate environmental model for subsequent construction. When the base station signal is lost, the redundant positioning module switches to the SLAM-based visual-inertial tightly coupled positioning mode to maintain a positioning accuracy within 5 cm to ensure construction continuity.

[0042] The AR intelligent operation system (module M2) integrates RTK, IMU and binocular vision units, and uses the built-in lightweight CNN of the edge computing unit to process wall defect recognition in real time and generate repair plans. In high-altitude operation mode (height ≥ 3 meters), the system automatically activates the redundant data verification mechanism, increases the positioning frequency to 100Hz and enables dual IMU cross-validation to ensure that the positioning accuracy is controlled within ±2mm, which is better than traditional total stations (±5mm). The dynamic path planning module generates the spraying trajectory based on the A* algorithm, and compensates for the robot arm joint angle error in real time. It combines with the spectral analysis unit to detect the LAB color difference between the spray color and the target color card (triggers closed-loop adjustment of the pigment mixing ratio when ΔE>3) to achieve color consistency control.

[0043] The multimodal fusion positioning model (Module M4) fuses RTK, IMU, and visual data through an extended Kalman filter and a deep learning dynamic weighting algorithm, outputting high-precision position and pose results with a positioning error of less than 2 cm. In dynamic scenarios, the visual feature weight is dynamically increased to 0.6-0.8. Combined with an LSTM network to predict positioning drift trends, this ensures an 80% improvement in positioning robustness in complex environments. The intelligent construction monitoring system (Module M5) monitors the construction environment in real time using embedded cameras and infrared thermal imaging modules. It uses deep learning algorithms to identify the correctness of construction workers' movements. When abnormal behavior is detected, it automatically triggers safety warnings and suspends equipment operation to ensure construction safety.

[0044] The pigment management system (module M6) has built-in high-precision weighing sensors and pigment mixing devices, supports dynamic ratio adjustment of 16 basic pigments, and uses a spectral analysis unit to detect spray color uniformity in real time (automatically adjusts spray parameters when ΔE>1), ensuring color consistency and construction efficiency; the cloud management platform (module M3) communicates with each module via a wireless network, stores painting plan data, and dynamically issues control instructions based on the construction progress. It combines Gantt charts to plan construction tasks and automatically adjusts subsequent construction plans to ensure that the project is completed on time.

[0045] Example 2;

[0046] See also Figure 1-Figure 5 In an embodiment of the present invention, a method for an intelligent wall painting system based on multimodal fusion positioning is provided, the method comprising the following steps: Step S1, an initialization phase, scanning the wall through an intelligent base station array, constructing a three-dimensional map with a 5cm point cloud density, marking the coordinates of doors, windows, cracks and obstacles, and uploading it to a cloud platform; Step S2, a path planning phase, the cloud platform parses the painting scheme data, generates a global spraying path based on the Dijkstra algorithm, and simulates the collision probability of the robot arm motion through the Monte Carlo method; Step S3, a real-time operation phase, the AR system calculates the posture in real time through the multimodal fusion positioning model, and automatically triggers the spraying pressure to adjust to the range of 0.3-0.5MPa when it detects that the depth of a local depression on the wall is greater than 5mm; Step S4, a quality control phase, after construction is completed, the wall image is collected through the AR system and compared with the target scheme for SSIM structural similarity, and a repainting instruction is generated when the difference area exceeds 2%.

[0047] Step S3 includes the following steps: Step a, when it is recognized that the ambient light intensity changes by more than 2000 lux, the visual positioning enhancement mode is started, and the binocular camera sampling rate is increased to 30fps; Step b, for the gradient color area, the transition color spraying instruction is generated through the HSV color space interpolation algorithm, and the monochrome gradient band resolution is increased to 256 levels; Step c, when working in the corner area, switch to the high-precision lidar assisted positioning mode 20 cm in advance to ensure that the corner connection error is less than 1 cm.

[0048] Step S4 includes the following steps: step a, when it is detected that the overall wall coverage rate is lower than 98%, the re-spraying mode is automatically triggered, and the re-spraying path is re-planned based on the point cloud data of the uncovered area; step b, the uniformity of the spraying color is detected by the spectral analysis unit, and when the color difference ΔE>1, the spraying pressure and pigment ratio are automatically adjusted; step c, after the construction is completed, the wall image is collected through the AR system, and the construction quality report is generated in combination with the deep learning algorithm. The report content includes key indicators such as coverage rate, color difference value and flatness.

[0049] The working principle of the embodiment of the present invention is: an intelligent wall painting method based on multimodal fusion positioning realizes full-process automated construction and quality control through phased collaborative optimization. In the initialization stage (step S1), the intelligent base station array scans the wall with a point cloud density of 5cm, constructs a three-dimensional map and marks the coordinates of obstacles such as doors, windows, and cracks, and uploads it to the cloud platform; in the path planning stage (step S2), the cloud platform parses the painting plan data, generates a global spray path based on the Dijkstra algorithm, and simulates the collision probability of the robot arm movement through the Monte Carlo method to optimize the path safety.

[0050] In the real-time operation stage (step S3), the AR system uses a multimodal fusion positioning model to solve the posture in real time. When it detects that the depth of the local depression on the wall is >5mm, it automatically triggers the spray pressure to adjust to the range of 0.3-0.5MPa. When the ambient light intensity changes by more than 2000lux, the system starts the visual positioning enhancement mode and increases the binocular camera sampling rate to 30fps to ensure positioning stability. For gradient color areas, the HSV color space interpolation algorithm is used to generate transition color spray instructions, and the monochrome gradient band resolution is increased to 256 levels to achieve fine color transitions. When working in corner areas, switch to high-precision lidar assisted positioning mode 20cm in advance to ensure that the corner connection error is less than 1cm.

[0051] During the quality control phase (step S4), after construction is completed, the AR system collects images of the wall and compares them with the target solution using the SSIM structural similarity algorithm. A repainting instruction is generated when the difference area exceeds 2%. When the overall wall coverage is lower than 98%, the re-spraying mode is automatically triggered, and the re-spraying path is re-planned based on the point cloud data of the uncovered area. The spectral analysis unit detects the uniformity of the spraying color. When the color difference ΔE>1, the spraying pressure and pigment ratio are automatically adjusted to ensure the final spraying quality. After construction is completed, the system combines deep learning algorithms to generate a key indicator report including coverage, color difference value, and flatness, providing data support for subsequent optimization.

[0052] A dynamic error model built using a reinforcement learning algorithm analyzes operator posture, environmental disturbances, and device jitter data in real time, predicting plotting path deviations and automatically generating compensation instructions, reducing the need for manual intervention. The system improves positioning stability by 80% in complex scenarios such as signal obstruction and electromagnetic interference, supports seamless mapping of curved surfaces and irregularly shaped walls, and improves construction efficiency by 300% compared to traditional methods, reducing manual error correction by over 90%, making it suitable for high-altitude operations (≥3 meters).

[0053] Example 3;

[0054] See also Figure 1-Figure 5 In the embodiment of the present invention, a specific implementation scheme of an intelligent wall painting system based on multimodal fusion positioning is provided, and its technical details and parameter configuration are as follows:

[0055] The intelligent base station array deploys 4-8 base stations with dynamically adjustable spacing (10-30 meters). It is equipped with a U-blox ZED-F9P RTK module (positioning accuracy ±1cm), a 2.4GHz / 5GHz dual-band LoRa ad hoc network module (communication latency <50ms), and a 16-line LiDAR (angular resolution 0.5°, detection range 50m). The environmental perception module integrates an Intel RealSense D455 RGB-D camera (depth accuracy ±2mm@2m), constructing a 3D point cloud map of the wall (point cloud density 5cm) and annotating concave and convex defect areas (depth error <3mm).

[0056] The AR intelligent operation terminal uses HoloLens 2 AR glasses, integrating the MPU-9250IMU (sampling rate 100Hz), the Leica BLK360 binocular vision module (4K resolution, 30fps) and the NVIDIA Jetson AGX Xavier edge computing unit (computing power 32TOPS); the robotic arm uses the UR10e six-axis collaborative robotic arm (repeat positioning accuracy ±0.05mm), equipped with a PID controller to adjust the spraying pressure (0.3-0.5MPa) and moving speed (0.1-1.5m / s).

[0057] During the initialization phase, the wall is scanned and modeled. The lidar scans the wall at a 5cm point cloud density to generate a 3D map (including the coordinates of doors, windows, and cracks). A greedy algorithm is used to optimize the base station positions to ensure signal coverage of ≥95%. Surface segmentation is then performed to divide the wall into 0.5m×0.5m sub-surfaces, which are classified and labeled based on the curvature radius (>1m for flat surfaces, <0.5m for high curvature areas), and uploaded to the cloud.

[0058] During the path planning stage, a global path is generated. The Dijkstra algorithm is used in the cloud to plan the spraying path, and Monte Carlo simulation is used to simulate the motion trajectory of the robotic arm (collision probability <0.1%). Dynamic obstacle avoidance optimization is also used. LiDAR detects obstacles (such as temporary equipment) in real time, and the path is dynamically adjusted using the A* algorithm. The obstacle avoidance response time is <200ms.

[0059] During the real-time operation phase, multimodal positioning is integrated, and RTK (10Hz), IMU (100Hz) and visual SLAM (30Hz) data are fused through the extended Kalman filter (EKF) to output a six-degree-of-freedom pose (error <2mm). When working at high altitude (≥3 meters), dual IMU cross-validation is activated, the positioning frequency is increased to 100Hz, and the visual weight is dynamically adjusted to 0.8.

[0060] For spraying control, the robotic arm sprays along a path. When encountering a wall depression depth greater than 5mm, the spraying pressure is automatically adjusted to 0.5MPa (flow control accuracy ±0.1mL / s). The gradient color area uses the HSV interpolation algorithm to generate 256 levels of color. The printhead (Epson I3200-A1) outputs at a resolution of 2880dpi, and the transition color band smoothness SSIM ≥ 0.98.

[0061] During the quality control stage, color difference detection is performed using a built-in X-Rite eXact spectrophotometer to monitor ΔE values in real time. Exceeding standard areas (ΔE>1) trigger closed-loop adjustments (the pigment ratio must meet the standard within three iterations). For coverage verification, the AR system captures wall images and compares them with the target solution using the SSIM algorithm. When the difference area is >2%, a re-spraying path is generated (coverage is re-sprayed to ≥98%). For report generation, a deep learning model (ResNet-50) analyzes construction data and outputs a quality report (including indicators such as flatness error <0.5mm and color difference ΔE average ≤1.2).

[0062] The working principle of the embodiment of the present invention is as follows: the error of multimodal fusion positioning is stable at ±2mm in complex scenarios (electromagnetic interference, occlusion), which is 80% higher than traditional RTK (±10mm); in terms of construction efficiency, the robotic arm spraying speed reaches 15㎡ / hour, which is 300% higher than manual painting (2㎡ / day), and the energy consumption optimization module reduces power consumption by 30%; the dynamic pigment proportioning system achieves color uniformity of ΔE≤1.5 and supports 16-color mixing.

[0063] Working Principle: The intelligent wall painting system and method based on multimodal fusion positioning achieves high-precision positioning and full-process automated construction in complex scenarios through modular design and collaborative optimization. During the initialization phase, the system first scans the wall through an intelligent base station array, constructs a three-dimensional point cloud map, and marks the coordinates of obstacles, providing an accurate environmental model for subsequent construction. The base station array dynamically adjusts its deployment position to optimize signal coverage and switches to visual-inertial tightly coupled positioning mode when the signal is lost to ensure construction continuity.

[0064] During the real-time operation stage, the AR intelligent operation system combines RTK, IMU and visual data, and uses a multimodal fusion positioning model to solve the posture in real time, supporting high-precision positioning and spray path planning in high-altitude operation mode; the system automatically generates repair plans based on wall defects, and uses a spectral analysis unit to detect the color difference between the spray color and the target color card in real time, and dynamically adjusts the pigment ratio to ensure color consistency; for gradient color areas, the system generates transition color spray instructions through a color space interpolation algorithm to achieve fine color transitions.

[0065] During the quality control stage, the system uses AR to capture wall images and compare them with the target solution for structural similarity, automatically identifying areas of difference and generating redrawing instructions; when insufficient wall coverage or color difference exceeding the threshold is detected, the system triggers the re-spraying mode and optimizes the re-spraying path; after construction is completed, the system combines deep learning algorithms to generate a construction quality report to provide data support for subsequent optimization.

[0066] Furthermore, the cloud-based management platform dynamically plans construction tasks and adjusts subsequent construction plans based on real-time progress to ensure on-time project completion. The intelligent construction monitoring system monitors the construction environment and personnel compliance in real time, automatically triggering safety warnings and suspending equipment operations to ensure construction safety. Through dynamic base station tuning and algorithm optimization, the system's positioning stability in complex environments has been significantly improved, supporting seamless mapping of curved surfaces and irregularly shaped walls, reducing the need for manual intervention and significantly improving construction efficiency and quality.

[0067] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent wall painting system based on multimodal fusion positioning, characterized in that: Includes the following components: Module M1, an intelligent base station array, consists of no less than four base stations, deployed around the wall, with the base station spacing dynamically adjusted to a range of 10-30 meters based on the wall size. Each base station is equipped with a 2.4GHz / 5GHz dual-band LoRa protocol self-organizing network communication module to achieve distributed clock synchronization and dynamic power regulation; Module M2, an AR intelligent operation system, integrates an RTK positioning unit, an IMU inertial measurement unit, and a binocular vision positioning unit. It uses a lightweight convolutional neural network built into the edge computing unit to process wall defect identification and repair solution generation in real time, and supports the overlay of AR virtual images onto real walls. Module M3, a cloud-based management platform, communicates with the intelligent base station array and AR system via a wireless network, stores painting scheme data, and dynamically issues control instructions based on the construction progress; Module M4, a multimodal fusion positioning model, fuses RTK positioning data, IMU motion data, and visual feature point data based on an extended Kalman filter and a deep learning dynamic weighted algorithm, outputting high-precision pose results with a positioning error of less than 2 cm.

2. The intelligent wall painting system based on multimodal fusion positioning according to claim 1 is characterized in that: The intelligent base station array further comprises: The environmental perception module integrates a 16-line laser radar and an RGB-D camera to construct a 3D point cloud map of the wall with an angular resolution of 0.5° and mark the coordinates of concave and convex defect areas; The base station self-test program optimizes the base station deployment location through a greedy algorithm when it starts, while also monitoring the communication delay between base stations in real time and dynamically adjusting the LoRa transmission power; The redundant positioning module switches to the SLAM-based visual-inertial tightly coupled positioning mode when it detects the loss of a single base station signal, maintaining positioning accuracy within 5 cm. The dynamic obstacle avoidance module combines lidar and visual data to detect obstacles in the construction area in real time, and dynamically adjusts the base station deployment position through algorithms to avoid signal blockage and interference.

3. The intelligent wall painting system based on multimodal fusion positioning according to claim 1 is characterized in that: The AR intelligent operation system specifically includes: In high-altitude operation mode, when the operating height is detected to be over 2 meters, the redundant data verification mechanism is automatically activated, the positioning frequency is increased to 100Hz, and dual IMU cross-validation is enabled; The dynamic path planning module generates the spraying trajectory based on the wall point cloud map and painting plan data through the A* algorithm, and compensates for the robot arm joint angle error in real time; The spectral analysis unit uses a built-in spectrophotometer to detect the LAB color difference between the spray color and the target color card in real time. When ΔE>3, closed-loop adjustment of the pigment mixing ratio is triggered.

4. The intelligent wall painting system based on multimodal fusion positioning according to claim 1 is characterized in that: The AR intelligent operation system also includes: The robotic arm adaptive control module uses a six-axis torque sensor to monitor the force status of the robotic arm in real time, and combines it with a PID controller to dynamically adjust the spraying pressure and speed to ensure uniform coverage on complex wall surfaces; The user interaction interface supports construction workers to view construction progress, defect annotations and operation prompts in real time through AR glasses. At the same time, they can adjust the painting plan or pause construction through voice commands.

5. The intelligent wall painting system based on multimodal fusion positioning according to claim 1 is characterized in that: The implementation of the multimodal fusion positioning model includes: In the data layer fusion stage, the timestamps of RTK satellite signals, IMU raw acceleration data, and visual feature point coordinates are aligned, and the sliding window method is used to eliminate the timing deviation of heterogeneous data; During the feature layer fusion stage, the fusion weights of RTK, IMU, and visual data are dynamically allocated through the attention mechanism. The weight of visual features in dynamic scenes is increased to 0.6-0.

8. In the decision-making layer fusion stage, the positioning drift trend is predicted based on the LSTM network, and the six-degree-of-freedom pose estimation result is output and fed back to the motion control module; The environmental adaptive optimization module dynamically adjusts the data fusion strategy according to the light intensity, temperature and humidity of the construction environment to ensure the stability of positioning accuracy under extreme conditions; The real-time error compensation module post-processes the positioning results through the Kalman filter, combines historical data to predict and compensate for positioning drift.

6. The intelligent wall painting system based on multimodal fusion positioning according to claim 1 is characterized in that: The system further comprises: Module M5, an intelligent construction monitoring system, uses embedded cameras and infrared thermal imaging modules to monitor the construction environment in real time. It uses deep learning algorithms to identify the standardization of construction workers' movements. When abnormal behavior is detected, it automatically triggers a safety warning and suspends equipment operation. Module M6, a pigment management system, has a built-in high-precision weighing sensor and pigment mixing device. It dynamically adjusts the pigment ratio according to the painting plan and supports the mixing of 16 basic pigments to ensure color consistency and construction efficiency.

7. The intelligent wall painting system based on multimodal fusion positioning according to claim 1 is characterized in that: The intelligent construction monitoring system includes: The construction progress management module dynamically plans construction tasks based on the Gantt chart and automatically adjusts subsequent construction plans based on real-time progress data to ensure that the project is completed on time; The resource allocation module dynamically allocates base stations, robotic arms, and paint resources based on the size and complexity of the construction area, supporting multi-device collaborative operations; The energy consumption optimization module uses a deep reinforcement learning algorithm to adjust equipment power and operating mode in real time to reduce construction energy consumption and extend equipment life.

8. The method of an intelligent wall painting system based on multimodal fusion positioning according to any one of claims 1 to 7, characterized in that: The method comprises the following steps: Step S1, initialization phase, uses an intelligent base station array to scan the wall, constructs a 3D map with a 5cm point cloud density, marks the coordinates of doors, windows, cracks and obstacles, and uploads it to the cloud platform; Step S2, path planning phase, the cloud platform analyzes the painting scheme data, generates a global spraying path based on the Dijkstra algorithm, and simulates the collision probability of the robot arm movement through the Monte Carlo method; Step S3, real-time operation stage, the AR system uses the multimodal fusion positioning model to solve the posture in real time. When it detects that the depth of the local depression on the wall is greater than 5mm, it automatically triggers the spray pressure to be adjusted to the range of 0.3-0.5MPa; Step S4, the quality control stage, after the construction is completed, the wall image is collected through the AR system and compared with the target solution by SSIM structural similarity. If the difference area exceeds 2%, a redraw instruction is generated.

9. The method of the intelligent wall painting system based on multimodal fusion positioning according to claim 8, characterized in that: The step S3 comprises the following steps: Step a: When the ambient light intensity changes by more than 2000 lux, the visual positioning enhancement mode is activated and the binocular camera sampling rate is increased to 30 fps. Step b: for the gradient color area, generate transition color spraying instructions through the HSV color space interpolation algorithm, and increase the resolution of the monochrome gradient band to 256 levels; Step c: When working in a corner area, switch to high-precision lidar-assisted positioning mode 20 cm in advance to ensure that the corner connection error is less than 1 cm.

10. The method of the intelligent wall painting system based on multimodal fusion positioning according to claim 8, characterized in that: The step S4 comprises the following steps: Step a: When it is detected that the overall wall coverage is less than 98%, the re-spraying mode is automatically triggered, and the re-spraying path is re-planned based on the point cloud data of the uncovered area; Step b, detecting the uniformity of the spray color through a spectral analysis unit, and automatically adjusting the spray pressure and pigment ratio when the color difference ΔE>1; Step c: After construction is completed, the wall image is collected through the AR system and combined with the deep learning algorithm to generate a construction quality report. The report content includes key indicators such as coverage, color difference value and flatness.

Images