Vehicle spraying system and method based on 3D vision and automatic trajectory planning

Abstract

The present application discloses a vehicle spray painting system and method based on 3D vision and automatic trajectory planning, comprising: using 3D vision technology to collect vehicle surface data and establish a digital vehicle model; performing automatic trajectory planning based on the digital vehicle model to generate a spraying path; optimizing process parameters according to paint characteristics; positioning the vehicle, and having a robot execute the spraying process; and real-time monitoring of the spraying process and performing quality control.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to an intelligent vehicle spraying method and system based on 3D vision and automatic trajectory planning. Background Art

[0002] In the modern automobile manufacturing industry, vehicle spraying is a key process link, which directly affects the appearance quality and anti-corrosion performance of the product. The traditional vehicle spraying method mainly relies on manual operation or simple automated equipment, and there are several major problems:

[0003] Unstable spraying quality: Manual spraying is easily affected by the operator's experience and status, and it is difficult to ensure long-term stable spraying quality.

[0004] Low efficiency: The working efficiency of manual spraying or simple automated equipment is limited and it is difficult to meet the needs of large-scale production. Summary of the invention

[0005] In view of this, the present application provides an intelligent vehicle spraying method and system based on 3D vision and automatic trajectory planning, which solves the problems of unstable vehicle spraying quality and low efficiency in the prior art.

[0006] The embodiment of the present application provides an intelligent vehicle spraying method based on 3D vision and automatic trajectory planning, including:

[0007] Use 3D vision technology to collect vehicle surface data and build a digital vehicle model;

[0008] Performing automatic trajectory planning based on the digital vehicle model to generate a spraying path;

[0009] Optimize process parameters according to paint characteristics;

[0010] The vehicle is positioned and the robot performs the painting process;

[0011] Monitor the spraying process in real time and perform quality control.

[0012] Use 3D vision technology to collect vehicle surface data and build a digital vehicle model, including:

[0013] Deploy multiple 3D scanners to scan all around the vehicle;

[0014] Project structured light onto the vehicle surface and calculate the surface depth information by analyzing the deformation of the structured light;

[0015] Projecting phase-shifted patterns at different frequencies and improving the robustness of depth measurements by synthesizing multi-frequency phase maps;

[0016] Apply L1-Curl regularized divergence-free wavelet reconstruction technology to process the original scan data and obtain the reconstructed depth map;

[0017] Merge the depth maps corresponding to the reconstructed multi-view scanning data to generate a complete point cloud model;

[0018] The point cloud model is converted into a triangular mesh model and optimized through an edge-preserving smoothing algorithm and an adaptive mesh subdivision technique to reduce the data volume while retaining key geometric features.

[0019] Apply L1-Curl regularized divergence-free wavelet reconstruction technology to process raw scan data, including:

[0020] Preprocess the depth map obtained from the original scan, including denoising and outlier removal;

[0021] Converting the preprocessed depth map into a wavelet domain representation;

[0022] In the wavelet domain, an L1-Curl regularization constraint is applied to maintain edge sharpness and surface details in the depth map;

[0023] Construct an objective function based on L1-Curl regularization, including data fidelity terms and regularization terms;

[0024] The constructed optimization problem is solved using the alternating direction multiplier method ADMM, where the optimization goal is to minimize the objective function;

[0025] By means of an ADMM iteration process, wavelet coefficients, auxiliary variables and Lagrange multipliers of the depth map are alternately updated;

[0026] The optimized wavelet coefficients are inversely transformed back to the spatial domain to obtain the reconstructed depth map.

[0027] Automatic trajectory planning is performed based on the digital vehicle model, including:

[0028] Automatically identify vehicle parts and divide them into blocks based on vehicle structural features;

[0029] Apply edge-aware algorithms to identify key edges and features within each block;

[0030] Optimize spraying sequence and path connection based on inter-channel correlation analysis;

[0031] Generate initial path using improved fast marching method;

[0032] A local optimization algorithm is applied to adjust the initial path to meet the spraying uniformity and efficiency requirements.

[0033] Apply edge-aware algorithms to identify key edges and features within each tile, including:

[0034] Performing multi-scale Gaussian filtering on the digital vehicle model to generate a scale space representation;

[0035] In the scale-space representation, the curvature and normal vector change rate are calculated at each scale to identify potential edge points;

[0036] Use adaptive threshold technology to dynamically adjust edge detection parameters according to local area characteristics;

[0037] Apply edge tracking algorithm to connect discrete edge points to form a continuous edge curve;

[0038] Using principal curvature analysis, edges, corners and plane areas are distinguished to generate the final edge feature map;

[0039] The optimization of spraying sequence and path connection based on inter-channel correlation analysis includes:

[0040] Construct an adjacency graph of the spraying area, where each node represents a spraying block;

[0041] Calculate the feature similarity between adjacent blocks, including color, texture and geometric features;

[0042] Using a spectral clustering algorithm, grouping the blocks based on feature similarity;

[0043] In each group, an improved genetic algorithm was applied to optimize the spraying sequence and minimize the spray gun moving distance and posture changes;

[0044] The dynamic programming algorithm is used to optimize the transition paths between different groups to ensure optimal global spraying efficiency.

[0045] Optimize process parameters according to paint characteristics, including:

[0046] Establish a paint properties database containing data on the physical and chemical properties of different types of paints;

[0047] Record the performance data of paint under different environmental conditions;

[0048] Establish a correlation model between paint characteristics and spraying parameters;

[0049] Initialize spraying parameters based on vehicle model characteristics and paint type;

[0050] Applying anisotropic and isotropic total variation weighted negative binomial model to optimize the spraying parameters;

[0051] Solve optimization problems using the alternating direction multiplier method;

[0052] Simulate the spraying process in a virtual environment and evaluate the effect of optimized parameters;

[0053] The optimized parameters are associated with the vehicle model information and saved in the system database.

[0054] Apply the anisotropic and isotropic total variation weighted negative binomial model to optimize the spraying parameters, including:

[0055] Construct an objective function, including data fidelity term, anisotropic total variation regularization term and isotropic total variation regularization term;

[0056] The negative binomial distribution prior is introduced to model the sparsity and heavy-tail characteristics of spraying parameters;

[0057] Design a weighting strategy to adaptively adjust the weights of anisotropic and isotropic regularization terms according to the local image structure;

[0058] Use variational Bayesian inference to estimate the model's hyperparameters;

[0059] The spraying parameters are iteratively optimized by alternately minimizing the data fidelity term and the regularization term.

[0060] Use the alternating direction method of multipliers to solve optimization problems, including:

[0061] Decompose the original optimization problem into several sub-problems, each sub-problem corresponds to a spraying parameter or parameter group;

[0062] The augmented Lagrangian function is introduced to transform the constraints into penalty terms;

[0063] Alternately update the primal variables, dual variables, and Lagrange multipliers;

[0064] Design an adaptive step size strategy to dynamically adjust the update step size according to the convergence status during the optimization process;

[0065] Multiple stopping criteria are set, including the primal residual, dual residual, and the rate of change of the objective function value, to ensure that the algorithm converges to a high-quality solution.

[0066] The method further comprises:

[0067] Based on the results of 3D visual modeling, a virtual spraying environment is constructed, including a vehicle geometry model and a spraying equipment model;

[0068] Discretize the vehicle surface into grid points, each of which contains position, normal vector and current spraying status information;

[0069] Based on the fluid dynamics model, the spraying parameters of the spray gun are simulated, including spray cone angle, flow rate and distance attenuation parameters;

[0070] Design a reinforcement learning agent whose state space includes the current spray gun position, posture, distance from the vehicle surface, and the spray state of the surrounding area;

[0071] Define the action space, including the movement direction and speed of the spray gun, and the control of the spray switch;

[0072] Design a composite reward function based on spray uniformity, coverage, time efficiency, and paint usage;

[0073] The PPO algorithm is used to train the reinforcement learning agent to optimize the spraying strategy.

[0074] The embodiment of the present application also provides an intelligent vehicle spraying system based on 3D vision and automatic trajectory planning, which is used to implement the above method. The system includes:

[0075] 3D visual scanner for collecting vehicle models;

[0076] A main controller, used to generate a spraying path and control the robot to spray;

[0077] The robot body is used for vehicle spraying;

[0078] A paint spray gun, arranged on the robot body;

[0079] The external bracket is used to fix and support the 3D vision scanner, the main controller and the robot body.

[0080] The present application also provides a computer device, the computer device comprising:

[0081] at least one processor; and,

[0082] a memory communicatively connected to the at least one processor; wherein,

[0083] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned method of intelligent vehicle painting based on 3D vision and automatic trajectory planning.

[0084] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the above-mentioned method of intelligent vehicle painting based on 3D vision and automatic trajectory planning.

[0085] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above-mentioned method for intelligent vehicle painting based on 3D vision and automatic trajectory planning.

[0086] This application has the following technical effects:

[0087] Improve spraying quality: Accurately collect vehicle surface data through 3D vision technology, combined with automatic trajectory planning and process parameter optimization, the uniformity and coverage of spraying can be significantly improved, thereby improving the overall spraying quality.

[0088] Improve production efficiency: Automated spraying process and optimized trajectory planning can greatly improve spraying efficiency and reduce production time.

[0089] Reduce material waste: Through precise spray path planning and parameter optimization, excessive use of paint can be minimized and material costs can be reduced.

[0090] Enhanced adaptability: Methods based on 3D vision and automatic trajectory planning can quickly adapt to the spraying requirements of different models and special parts, and improve the flexibility of the production line.

[0091] Achieve process control: Through real-time monitoring and quality control, problems in the spraying process can be discovered and corrected in time to improve product consistency.

[0092] Support data traceability: The spraying parameters and quality data recorded by the system can be used for subsequent analysis and improvement to achieve continuous optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the disclosed embodiments in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0094] Figure 1 is a flow chart of an intelligent vehicle spraying method based on 3D vision and automatic trajectory planning provided in an embodiment of the present application;

[0095] Figure 2 This is a flow chart of collecting vehicle surface data with 3D vision and establishing a digital vehicle model provided by an embodiment of the present application;

[0096] Figure 3 This is a flow chart of automatic trajectory planning to generate a spraying path provided by an embodiment of the present application;

[0097] Figure 4 is a flow chart of paint characteristic process parameter optimization provided by an embodiment of the present application;

[0098] Figure 5 It is a block diagram of an intelligent vehicle spraying system based on 3D vision and automatic trajectory planning provided in an embodiment of the present application;

[0099] Figure 6 It is a structural schematic diagram of an intelligent vehicle spraying system based on 3D vision and automatic trajectory planning provided in an embodiment of the present application;

[0100] Figure 7 is a schematic diagram of the structure of an external bracket provided in an embodiment of the present application;

[0101] Figure 8 is a schematic diagram of the structure of a 3D visual scanner provided in an embodiment of the present application;

[0102] Fig. 9 is a schematic diagram of the structure of the main controller provided in an embodiment of the present application;

[0103] Fig.10 It is a schematic diagram of the structure of the paint spray gun provided in the embodiment of the present application;

[0104] Fig.11 is a schematic diagram of an actual painting operation provided by an embodiment of the present application;

[0105] Fig.12a is a schematic diagram of a vehicle model provided in an embodiment of the present application;

[0106] Figure 12b is another schematic diagram of a vehicle model provided in an embodiment of the present application;

[0107] Fig.12c It is a schematic diagram of a robot provided in an embodiment of the present application spraying a vehicle along an automatic trajectory. DETAILED DESCRIPTION

[0108] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0109] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0110] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0111] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0112] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0113] like Figure 1 As shown, the intelligent vehicle spraying method based on 3D vision and automatic trajectory planning provided in the embodiment of the present application includes:

[0114] S1: Use 3D vision technology to collect vehicle surface data and build a digital vehicle model;

[0115] The embodiments of the present application use advanced 3D vision technology to obtain precise geometric information of the vehicle surface and construct a high-quality digital vehicle model.

[0116] S1.1: Deploy multiple 3D scanners to scan the vehicle in all directions;

[0117] In order to obtain complete data on the vehicle surface, the embodiment of the present application needs to scan the vehicle from multiple angles. Generally, the embodiment of the present application evenly distributes multiple 3D scanners around the vehicle. These scanners can be devices based on structured light, time of flight (ToF) or triangulation principles.

[0118] For example, the embodiment of the present application can use 8 high-precision structured light scanners, which are placed at the front, rear, left, right and four 45-degree angle positions of the vehicle. The resolution of each scanner can reach 0.1mm, and the scanning range covers an area of ​​3m x 2m. Such a configuration can ensure that every part of the vehicle surface can be captured by at least one scanner.

[0119] S1.2: Project structured light onto the vehicle surface and calculate the surface depth information by analyzing the deformation of the structured light;

[0120] The principle of structured light 3D scanning is based on the projection and observation of known light patterns. In this process:

[0121] A projector projects a specific pattern of light (usually a stripe or grid pattern) onto the vehicle surface. A camera observes these projected patterns from slightly different angles. Due to the varying shape and depth of the vehicle surface, the projected light pattern will be deformed.

[0122] By analyzing these deformations, the system can calculate the 3D coordinates of the surface.

[0123] For example, the embodiment of the present application can use a blue LED light source to project 120 equally spaced light and dark stripes with a stripe width of 0.5 mm. When these stripes are projected onto the surface of the vehicle, they will be deformed due to the undulations of the surface. A high-resolution camera (such as 4K resolution) will capture these deformed stripe images. Through the principle of triangulation, the embodiment of the present application can calculate the depth of each point on the surface based on the offset of the stripes.

[0124] S1.3: Project phase-shift patterns at different frequencies and improve the robustness of depth measurements by synthesizing multi-frequency phase maps;

[0125] In order to further improve the accuracy and robustness of depth measurement, the embodiment of the present application adopts phase-shift structured light technology and combines it with a multi-frequency phase unwrapping method.

[0126] A1. Phase-shifted structured light: At each scanning position, the embodiment of the present application continuously projects multiple phase-shifted stripe patterns (usually 3-4). Each pattern has a certain phase offset (for example, 90 degrees) relative to the previous pattern. This allows the embodiment of the present application to sample each pixel multiple times, thereby improving the accuracy of the measurement.

[0127] A2. Multi-frequency phase unwrapping: The embodiment of the present application uses stripe patterns of different frequencies (i.e., different widths). For example, stripe patterns of 64, 16, and 4 stripes may be used in sequence. For each frequency, the embodiment of the present application performs phase shift scanning.

[0128] A3. Phase calculation: For each frequency, the embodiment of the present application calculates a phase map according to the phase shift pattern. The low-frequency phase map can provide unambiguous and rough depth information, while the high-frequency phase map provides accurate but possibly 2π ambiguous depth information.

[0129] A4. Phase unwrapping: By combining phase information of different frequencies, the present application embodiment can solve the ambiguity problem of high-frequency phase while maintaining its high accuracy. This process usually uses a hierarchical phase unwrapping algorithm.

[0130] Through this method, the embodiment of the present application can achieve sub-pixel level depth measurement accuracy, while effectively handling different situations such as surface discontinuities and high reflectivity areas.

[0131] S1.4: Apply L1-Curl regularized divergence-free wavelet reconstruction technology to process the original scan data to obtain a reconstructed depth map;

[0132] After obtaining the raw scan data, the embodiment of the present application needs to process it to obtain a high-quality depth map. Here, the embodiment of the present application adopts an advanced reconstruction technology - L1-Curl regularized divergence-free wavelet reconstruction. This method is particularly suitable for processing vehicle models with sharp edges and smooth surfaces. Specifically include:

[0133] S1.4.1: Preprocess the depth map obtained from the original scan, including denoising and outlier removal;

[0134] First, the embodiment of the present application needs to clean up the noise and outliers in the original data. This step includes:

[0135] B1. Median filtering: Use a 5x5 or 7x7 sliding window to perform median filtering on the depth map, which can effectively remove salt and pepper noise.

[0136] B2. Bilateral filtering: Apply a bilateral filter to smooth the depth map while keeping the edges sharp. You can set the spatial sigma to 2.0 and the depth sigma to 10.0.

[0137] B3. Outlier detection and removal: Use local statistical methods to detect outliers. For example, for each pixel, calculate the depth mean and standard deviation of its 8-neighborhood. If the depth value of the pixel deviates from the mean by more than 3 times the standard deviation, it is marked as an outlier and deleted.

[0138] B4. Small area filling: For missing areas of small area (such as less than 10 square pixels), the average depth value of the surrounding valid pixels is used to fill them.

[0139] S1.4.2: Convert the preprocessed depth map into wavelet domain representation;

[0140] The embodiment of the present application uses discrete wavelet transform (DWT) to convert the depth map into the wavelet domain. The advantage of this is that the image can be analyzed and processed at multiple scales.

[0141] C1. Selecting a suitable wavelet basis: For an object such as a vehicle surface that includes smooth areas and sharp edges, the embodiment of the present application may select a Daubechies wavelet (such as db4) or a biorthogonal wavelet (such as CDF9 / 7).

[0142] C2. Perform multi-layer wavelet decomposition: Usually 3-5 layers of decomposition are performed, which can capture details of different scales.

[0143] C3. Get wavelet coefficients: including approximate coefficients (LL) and detail coefficients (LH, HL, HH).

[0144] S1.4.3: In the wavelet domain, apply L1-Curl regularization constraints to preserve edge sharpness and surface details in the depth map.

[0145] L1-Curl regularization is an effective method to smooth the surface while keeping the edges sharp. Its principle is:

[0146] The L1 norm promotes sparse solutions and helps preserve edge sharpness.

[0147] The Curl operator suppresses the curl in the depth map, helping to maintain the smoothness of the surface.

[0148] Specifically, the following regularization terms are defined in this embodiment of the application:

[0149] R(x) = λ1‖∇x‖1 + λ2‖curl(x)‖1;

[0150] Among them, x is the depth map, ∇ represents the gradient operator, curl represents the curl operator, and λ1 and λ2 are weight parameters.

[0151] S1.4.4: Construct an objective function based on L1-Curl regularization, including data fidelity terms and regularization terms;

[0152] The complete objective function is as follows:

[0153] minimize F(x) = ‖Wx - y‖2 2 + λ1‖∇x‖1 + λ2‖curl(x)‖1;

[0154] Among them, W is the wavelet transform matrix, y is the observed depth data, the first term is the data fidelity term, and the last two terms are regularization terms.

[0155] S1.4.5: Use the alternating direction method of multipliers (ADMM) to solve the constructed optimization problem, where the optimization goal is to minimize the objective function;

[0156] ADMM is an effective optimization algorithm, especially suitable for solving problems with L1 regularization. The basic idea of ​​ADMM is to decompose the original problem into several sub-problems that are easier to solve, and then update the solutions of these sub-problems alternately.

[0157] The specific steps are as follows:

[0158] D1. Introduce auxiliary variables z1 = ∇x and z2 = curl(x) to transform the original problem into a constrained optimization problem.

[0159] D2. Construct the augmented Lagrangian function:

[0160] L(x, z1, z2, u1, u2) = ‖Wx - y‖2² + λ1‖z1‖1 + λ2‖z2‖1;

[0161] + (ρ / 2)(‖∇x - z1 + u1‖2² + ‖curl(x) - z2 + u2‖2²);

[0162] Among them, u1 and u2 are Lagrange multipliers, and ρ is the penalty parameter.

[0163] S1.4.6: Through the ADMM iterative process, the wavelet coefficients, auxiliary variables and Lagrange multipliers of the depth map are updated alternately.

[0164] The iterative process of ADMM is as follows:

[0165] E1. Update x:

[0166] xk+1 = argmin_x L(x, z1k, z2k, u1k, u2k);

[0167] This subproblem can be solved by solving a system of linear equations.

[0168] E2. Update z1 and z2:

[0169] z1k+1 = S_{λ1 / ρ}(∇xk+1 + u1k);

[0170] z2k+1 = S_{λ2 / ρ}(curl(xk+1) + u2k);

[0171] Among them, S_{} is the soft threshold operator.

[0172] E3. Update u1 and u2:

[0173] u1k+1= u1k + ∇xk+1 - z1k+1;

[0174] u2k+1= u2k + curl(xk+1) - z2k+1;

[0175] E4. Check the convergence conditions. If not, return to step E1.

[0176] S1.4.7: Inversely transform the optimized wavelet coefficients back to the spatial domain to obtain the reconstructed depth map;

[0177] Finally, the embodiment of the present application uses an inverse discrete wavelet transform (IDWT) to transform the optimized wavelet coefficients back to the spatial domain to obtain a reconstructed high-quality depth map.

[0178] The sample code for S1.4.1-S1.4.7 is as follows:

[0179] ```Python

[0180] import numpy as np

[0181] import cv2

[0182] from scipy.signal import convolve2d

[0183] import pywt

[0184] def preprocess_depth_map(depth_map):

[0185] """

[0186] Preprocess the depth map, including denoising and outlier removal

[0187] """

[0188] # Use median filtering to remove salt and pepper noise

[0189] depth_map = cv2.medianBlur(depth_map, 5)

[0190] # Use bilateral filtering to smooth the depth map while keeping the edges

[0191] depth_map = cv2.bilateralFilter(depth_map, 9, 75, 75)

[0192] # Outlier detection and removal

[0193] mean = np.mean(depth_map)

[0194] std = np.std(depth_map)

[0195] depth_map[abs(depth_map - mean) > 3 * std] = mean

[0196] return depth_map

[0197] def wavelet_transform(depth_map):

[0198] """

[0199] Perform wavelet transform on the depth map

[0200] """

[0201] coeffs = pywt.wavedec2(depth_map, 'db4', level=3)

[0202] return coeffs

[0203] def l1_curl_regularization(coeffs):

[0204] """

[0205] Apply L1-Curl regularization

[0206] """

[0207] # This is just a simplified example, the actual L1-Curl regularization is more complicated

[0208] for i in range(1, len(coeffs)):

[0209] coeffs[i] = [np.sign(c) * np.maximum(np.abs(c) - 0.1, 0) for c incoeffs[i]]

[0210] return coeffs

[0211] def optimize_coeffs(coeffs, observed_depth, lambda1, lambda2, rho):

[0212] """

[0213] Optimizing wavelet coefficients using ADMM algorithm

[0214] """

[0215] # Initialize auxiliary variables and Lagrange multipliers

[0216] z1 = np.zeros_like(coeffs[0])

[0217] z2 = np.zeros_like(coeffs[0])

[0218] u1 = np.zeros_like(coeffs[0])

[0219] u2 = np.zeros_like(coeffs[0])

[0220] for _ in range(100): # Iteration times

[0221] # Update x

[0222] x = pywt.waverec2(coeffs, 'db4')

[0223] x = x + rho * (np.gradient(z1 - u1)[0] + np.gradient(z2 - u2)[1])

[0224] x = (x + rho * observed_depth) / (1 + rho)

[0225] coeffs = pywt.wavedec2(x, 'db4', level=3)

[0226] # Update z1, z2

[0227] grad_x = np.gradient(x)

[0228] z1 = np.sign(grad_x[0] + u1) * np.maximum(np.abs(grad_x[0] + u1) -lambda1 / rho, 0)

[0229] z2 = np.sign(grad_x[1] + u2) * np.maximum(np.abs(grad_x[1] + u2) -lambda2 / rho, 0)

[0230] # Update u1, u2

[0231] u1 = u1 + grad_x[0] - z1

[0232] u2 = u2 + grad_x[1] - z2

[0233] return coeffs

[0234] def inverse_wavelet_transform(coeffs):

[0235] """

[0236] Perform inverse wavelet transform to reconstruct the depth map

[0237] """

[0238] reconstructed_depth = pywt.waverec2(coeffs, 'db4')

[0239] return reconstructed_depth

[0240] # Main processing flow

[0241] def process_depth_map(depth_map):

[0242] # S1.4.1: Preprocessing

[0243] preprocessed_depth = preprocess_depth_map(depth_map)

[0244] # S1.4.2: Wavelet transform

[0245] coeffs = wavelet_transform(preprocessed_depth)

[0246] # S1.4.3-S1.4.6: Apply L1-Curl regularization and ADMM optimization

[0247] regularized_coeffs = l1_curl_regularization(coeffs)

[0248] optimized_coeffs = optimize_coeffs(regularized_coeffs, preprocessed_depth, lambda1=0.1, lambda2=0.1, rho=1.0)

[0249] # S1.4.7: Inverse wavelet transform

[0250] reconstructed_depth = inverse_wavelet_transform(optimized_coeffs)

[0251] return reconstructed_depth

[0252] # Example

[0253] if __name__ == "__main__":

[0254] # Assume that there is a depth map in this embodiment of the application

[0255] depth_map = np.random.rand(256, 256) # Random data is used as an example here

[0256] # Processing depth map

[0257] result = process_depth_map(depth_map)

[0258] # Display results

[0259] cv2.imshow("Original Depth Map", depth_map)

[0260] cv2.imshow("Reconstructed Depth Map", result)

[0261] cv2.waitKey(0)

[0262] cv2.destroyAllWindows()

[0263] ```

[0264] This code implements the depth map reconstruction and optimization process, corresponding to steps S1.4.1 to S1.4.7. The following is a detailed explanation of the code:

[0265] S1.4.1: Preprocessing (preprocess_depth_map function);

[0266] Use median filtering to remove noise;

[0267] Apply bilateral filtering to smooth the depth map while keeping edges sharp;

[0268] Detect and remove outliers, and replace values ​​that are more than 3 standard deviations from the mean with the mean;

[0269] S1.4.2: wavelet transform (wavelet_transform function);

[0270] Use the pywt library for two-dimensional discrete wavelet transform;

[0271] Select 'db4' (Daubechies 4) as the wavelet basis and perform level 3 decomposition;

[0272] S1.4.3: L1-Curl regularization (l1_curl_regularization function);

[0273] A simplified version of L1 regularization is provided here; soft thresholding is performed on the wavelet coefficients to simulate the effect of L1 regularization;

[0274] S1.4.4-S1.4.6: ADMM optimization (optimize_coeffs function);

[0275] Implemented the main steps of the ADMM algorithm;

[0276] Alternately update the original variables (wavelet coefficients), auxiliary variables and Lagrange multipliers;

[0277] Use the gradient operator to approximate the Curl operation;

[0278] S1.4.7: Inverse wavelet transform (inverse_wavelet_transform function);

[0279] Use pywt.waverec2 to perform two-dimensional discrete wavelet inverse transform and reconstruct the depth map;

[0280] The main processing flow (process_depth_map function) calls these functions in sequence to complete the entire depth map reconstruction and optimization process.

[0281] S1.5: Merge the depth maps corresponding to the reconstructed multi-view scanning data to generate a complete point cloud model;

[0282] In this step, the embodiment of the present application needs to merge the depth maps from different perspectives into a unified point cloud model. This process is usually called point cloud registration. It includes:

[0283] F1. Initial registration: Use the known position and pose information of the scanning device to roughly align the depth maps.

[0284] F2. Fine registration: Use the iterative closest point (ICP) algorithm for precise alignment. The steps of the ICP algorithm are as follows:

[0285] a) Find the nearest neighbor for each point;

[0286] b) Calculate the transformation matrix (rotation and translation) to minimize the distance between corresponding points;

[0287] c) applying transformations;

[0288] d) Repeat the above steps until convergence;

[0289] F3. Global Optimization: Use graph optimization methods (such as g2o or GTSAM) to perform global consistency optimization to reduce cumulative errors.

[0290] F4. Point cloud fusion: Use voxel grid averaging or distance-based weighted averaging to fuse points in overlapping areas.

[0291] F5. Normal vector estimation: Estimating the normal vector of each point is important for subsequent mesh reconstruction. The principal component analysis (PCA) method can be used to estimate the normal vector in the local neighborhood of each point.

[0292] S1.6: Convert the point cloud model into a triangular mesh model and optimize it through edge-preserving smoothing algorithm and adaptive mesh subdivision technology to reduce the data volume while retaining key geometric features;

[0293] Converting point clouds into triangular mesh models can better represent the vehicle surface and provide a better basis for subsequent spraying path planning. Including:

[0294] G1. Poisson surface reconstruction: Use the Poisson surface reconstruction algorithm to convert the point cloud into an initial triangular mesh. This algorithm can effectively handle noisy and incomplete data.

[0295] G2. Edge-preserving smoothing: Apply bilateral filtering or Taubin λ|μ smoothing algorithm to smooth the mesh while maintaining sharp edges. This can remove high-frequency noise introduced during the scanning process.

[0296] G3. Adaptive mesh subdivision:

[0297] a) Curvature Adaptive Subdivision: Use denser triangles in areas of high curvature (such as corners and edges of vehicles).

[0298] b) Feature-preserving simplification: Use larger triangles in flat areas to reduce the amount of data. Simplification algorithms based on quadratic error metrics (such as the Garland-Heckbert algorithm) can be used.

[0299] G4. Topology Optimization: Repair possible topological errors in the mesh, such as non-manifold edges, isolated vertices, etc.

[0300] G5. UV Unwrapping: Generate UV coordinates for the mesh, which is helpful for subsequent texture mapping and spray path planning. You can use LSCM (Least Squares Conformal Mapping) or ABF++ (Angle Based Flattening) algorithms.

[0301] Through the above steps, the embodiment of the present application obtains a high-quality digital vehicle model, such as Fig.12a The vehicle model shown in a and b not only retains the key geometric features of the vehicle surface but also has a reasonable amount of data, laying the foundation for subsequent spraying path planning and parameter optimization.

[0302] S2: Automatic trajectory planning based on the digital vehicle model to generate the spraying path;

[0303] Automatic trajectory planning is a key step to achieve high-quality and efficient spraying. The goal of the embodiment of this application is to generate a comprehensive, uniform and efficient spraying path. Specifically including:

[0304] S2.1: Automatically identify vehicle parts and divide them into blocks based on vehicle structural features;

[0305] First, the embodiment of the present application needs to decompose the vehicle model into multiple logically independent components, so that the spraying plan can be better based on the characteristics of different components.

[0306] H1. Segmentation algorithm selection: The embodiments of the present application may use a graph-based segmentation algorithm, such as the Felzenszwalb algorithm, or a learning-based method such as PointNet++.

[0307] H2. Feature extraction: Extract geometric features (such as curvature, normal vector) and topological features (such as connectivity).

[0308] H3. Split execution:

[0309] a) Initial over-segmentation: Divide the model into many small blocks.

[0310] b) Region merging: Merge adjacent regions based on similarity criteria (e.g., planarity, curvature continuity).

[0311] c) Boundary optimization: Use graph cut algorithms (such as Graph Cuts) to optimize the segmentation boundaries.

[0312] H4. Semantic labeling: Use pre-trained deep learning models (such as PointNet++ or DGCNN) to semantically label the segmented parts, such as "door", "hood", "roof", etc.

[0313] H5. Result Verification: Use heuristic rules to check the rationality of the segmentation results, such as ensuring the consistency of left-right symmetrical parts.

[0314] S2.2: Apply edge-aware algorithms to identify key edges and features within each block;

[0315] Identifying critical edges and features is essential to generating high-quality spray paths, as these areas often require special treatment.

[0316] S2.2.1: Perform multi-scale Gaussian filtering on the digital vehicle model to generate a scale-space representation;

[0317] 1. Build a Gaussian pyramid: Starting from the original mesh model, apply Gaussian filtering and downsampling step by step to generate models of multiple scales. Usually 3-5 scale levels are used.

[0318] 2. For each scale level, calculate the mean curvature and Gaussian curvature of each vertex.

[0319] S2.2.2: In the scale-space representation, calculate the curvature and normal change rate at each scale to identify potential edge points.

[0320] 1. Normal vector change rate calculation: For each vertex, calculate the normal vector change rate within the 1-ring neighborhood.

[0321] 2. Multi-scale feature response: Combine the curvature and normal vector change rate information at different scales to construct a feature response map.

[0322] 3. Non-maximum suppression: Perform non-maximum suppression on the feature response graph to preliminarily identify potential edge points.

[0323] S2.2.3: Use adaptive thresholding technology to dynamically adjust edge detection parameters based on local area characteristics;

[0324] Calculate the mean and variance of the feature response in the neighborhood of each vertex. Dynamically adjust the threshold based on local statistics. For example, you can use T = mean + k * std, where k is an adjustable parameter. Mark points whose feature response exceeds the adaptive threshold as edge points.

[0325] S2.2.4: Apply edge tracking algorithm to connect discrete edge points to form a continuous edge curve;

[0326] 1. Seed point selection: Select the point with the strongest feature response from the unprocessed edge points as the seed point.

[0327] 2. Direction estimation: Initialize the tracking direction based on the principal curvature direction of the seed point.

[0328] 3. Iteration tracking:

[0329] a) Search for the next edge point in the current direction;

[0330] b) If found, update the tracking direction and continue; if not found, end the current edge.

[0331] 4. Edge optimization: Use the edge point sequence obtained by spline curve fitting to generate a smooth edge curve.

[0332] S2.2.5: Use principal curvature analysis to distinguish edges, corners, and plane areas, and generate the final edge feature map;

[0333] 1. Principal curvature calculation: Calculate the principal curvature and principal direction for each vertex.

[0334] 2. Feature classification:

[0335] If both principal curvatures are close to zero, the region is classified as planar.

[0336] If one principal curvature is close to zero and the other is significantly non-zero, it is classified as an edge.

[0337] If both principal curvatures are significantly non-zero and have the same sign, then it is classified as a corner.

[0338] 3. Feature map generation: Map the classification results onto a grid to generate a feature map containing edge, corner, and plane area information.

[0339] S2.3: Optimize spraying sequence and path connection based on inter-channel correlation analysis;

[0340] In order to improve the spraying efficiency, the embodiment of the present application needs to optimize the spraying sequence and the transition path between different areas.

[0341] S2.3.1: Construct an adjacency graph of the spraying area, where each node represents a spraying block;

[0342] 1. Node definition: Each segmented vehicle component is regarded as a node in the graph.

[0343] 2. Edge definition: If two parts are adjacent in 3D space, add an edge between the corresponding nodes.

[0344] 3. Edge weight calculation: The edge weight is calculated based on factors such as the distance and area similarity between two adjacent components.

[0345] S2.3.2: Calculate the feature similarity between adjacent blocks, including color, texture and geometric features;

[0346] 1. Color similarity: If there is color information, calculate the Euclidean distance between the average color values ​​of the two blocks.

[0347] 2. Texture similarity: Use local binary pattern (LBP) or Gabor filter to extract texture features, and then calculate the cosine similarity of the feature vectors.

[0348] 3. Geometric feature similarity:

[0349] a) Calculate the difference between the mean curvature and Gaussian curvature of the two blocks.

[0350] b) Compare the main direction distributions of the two blocks.

[0351] 4. Comprehensive similarity: The above similarities are weighted and combined to obtain a comprehensive similarity score.

[0352] S2.3.3: Use spectral clustering algorithm to group the blocks based on feature similarity;

[0353] 1. Similarity matrix construction: Use the calculated comprehensive similarity scores to construct a similarity matrix.

[0354] 2. Laplacian matrix calculation: Calculate the normalized Laplacian matrix based on the similarity matrix.

[0355] 3. Eigenvalue decomposition: Perform eigenvalue decomposition on the Laplacian matrix.

[0356] 4. Determine the number of clusters: Use methods such as eigenvalue gap method or silhouette coefficient to determine the optimal number of clusters.

[0357] 5. K-means clustering: Use K-means algorithm for final clustering in feature vector space.

[0358] S2.3.4: Within each group, apply the improved genetic algorithm to optimize the spraying sequence and minimize the spray gun movement distance and posture changes;

[0359] 1. Encoding: Encode the spraying order into chromosomes, and each gene represents the index of a spraying block.

[0360] 2. Initialization: Generate a certain number of random spraying orders as the initial population.

[0361] 3. Fitness evaluation:

[0362] a) Calculate the total spray path length.

[0363] b) Estimate the total amount of change in gun attitude.

[0364] c) Calculate the overall fitness score by combining the above factors.

[0365] 4. Selection: Use tournament selection to select outstanding individuals.

[0366] 5. Crossover: A new spraying order is generated using a Partial Mapping Crossover (PMX) operation.

[0367] 6. Mutation: Use swap mutation or insertion mutation operations to introduce randomness.

[0368] 7. Local search: Apply 2-opt or 3-opt local search to the optimal individuals to further optimize the spraying order.

[0369] 8. Iterative evolution: Repeat steps 3-7 until the termination condition is reached (such as maximum number of generations or fitness convergence).

[0370] S2.3.5: Use dynamic programming algorithm to optimize the transition paths between different groups to ensure optimal global spraying efficiency;

[0371] 1. State definition: Define the state as (i, j), which means that the current state is the jth block in the i-th group.

[0372] 2. Transfer equation: dp[i][j] = min(dp[i-1][k] + cost(k, j)), where cost(k, j) represents the cost of transferring from the kth block of the i-1th group to the jth block of the i-th group.

[0373] 3. Cost calculation: Consider factors such as the distance the gun moves, posture changes, and possible reloading time.

[0374] 4. Boundary conditions: the cost of setting the initial state.

[0375] 5. Solve from the bottom up: Start from the first group and work your way to the last group.

[0376] 6. Path backtracking: Starting from the last state, backtrack according to the recorded optimal decision to obtain the globally optimal transition path between groups.

[0377] Through the above steps, the embodiment of the present application can obtain an optimized spraying sequence and path connection scheme, which takes into account local efficiency and ensures global optimization.

[0378] S2.4: Generate initial path using improved fast marching method;

[0379] The Fast Marching Method (FMM) is an effective path planning algorithm, which can be improved in the embodiments of the present application to meet the special requirements of vehicle painting. It includes:

[0380] 1. Distance field calculation: Use FMM to calculate the distance field from the edge to each point on the vehicle surface.

[0381] 2. Gradient field construction: The gradient field is calculated based on the distance field, and the gradient direction points to the direction where the distance field increases fastest.

[0382] 3. Path generation:

[0383] a) Start from the edge point and move along the gradient direction.

[0384] b) As you advance, consider the coverage of the spray gun and ensure that the paths are spaced appropriately.

[0385] c) When encountering a painted area or the edge of a vehicle, end the current path and select a new starting point.

[0386] 4. Curvature Adaptation: In areas of high curvature, the path spacing is reduced to ensure uniform coverage.

[0387] 5. Obstacle avoidance: holes, logos and other areas on the vehicle that do not need to be painted are treated as obstacles and bypassed when generating paths.

[0388] S2.5: Apply local optimization algorithm to adjust the initial path to meet the spraying uniformity and efficiency requirements;

[0389] After obtaining the initial path, the embodiment of the present application needs to perform local optimization to improve the spraying quality and efficiency.

[0390] 1. Path smoothing: Use cubic spline interpolation to smooth the path, reduce sharp turns, and make the spray gun move more smoothly.

[0391] 2. Overlap optimization:

[0392] a) Simulate the spraying process and calculate the spraying amount at each surface point.

[0393] b) Identify areas of under or over spraying.

[0394] c) Balance the spray volume by adjusting the spacing between adjacent passes or adding supplemental passes.

[0395] 3. Speed ​​planning: Plan the appropriate moving speed for each section of the path based on the surface curvature and spraying requirements.

[0396] 4. Posture Optimization: Ensure the spray gun always maintains the optimal spraying angle perpendicular to the surface, adding transition points to smooth posture changes when necessary.

[0397] 5. Path merging: For adjacent short paths, try to merge them to reduce the number of starts and stops and improve efficiency.

[0398] 6. Iterative optimization: Repeat spray simulation and path adjustment until the preset uniformity and efficiency indicators are achieved.

[0399] Through this series of steps, the embodiment of the present application obtains a fully optimized spraying path, which takes into account both the integrity and uniformity of coverage and the spraying efficiency. This path will provide precise guidance for the subsequent actual spraying process.

[0400] Assume that there is a digital model of a medium-sized SUV in the embodiment of the present application. This model contains the precise geometric information of the vehicle, including all surfaces, edges and feature points. The automatic trajectory planning system of the embodiment of the present application will work according to the following steps:

[0401] 1. Vehicle segmentation: The system first divides the vehicle into multiple areas, such as the hood, roof, left and right doors, trunk lid, etc. For the SUV in the embodiment of the present application, it will be divided into about 15-20 main areas.

[0402] 2. Prioritization: Based on size, accessibility, and geometric complexity, the system prioritizes these blocks. For example, large flat areas such as the roof and hood may be prioritized.

[0403] 3. Path generation: For each block, the system will generate a detailed spray path. Take the hood as an example:

[0404] The system first determines a starting point, usually a corner of the hood.

[0405] It then creates a series of parallel spray lines covering the entire surface. These lines may be from front to back, spaced about 15-20 cm apart.

[0406] In the edge areas of the hood, the system generates additional contour lines to ensure that the edges are adequately covered.

[0407] For any bumps or dips on the hood, such as trim lines, the system adjusts the path to ensure even coverage.

[0408] 4. Path optimization: After generating the initial path, the system will optimize it:

[0409] Smoothing: Replace sharp turns with smooth curves to make robot movements smoother.

[0410] Minimize non-spraying movement: Optimize the transition between different blocks and reduce wasted movement time.

[0411] Pose planning: Ensure the spray gun always maintains the correct angle and distance, especially when dealing with complex surfaces.

[0412] 5. Collision check: The system simulates the entire spraying process to ensure that the robot does not collide with vehicles or other equipment when executing the path.

[0413] The resulting path might look something like this: starting on the hood, spraying along a predetermined path of parallel lines, then seamlessly transitioning to the front bumper, followed by the left front fender, then the left front door, and so on until the entire body is covered.

[0414] S3: Optimize process parameters according to paint characteristics;

[0415] Optimization of process parameters is crucial to achieving high-quality spraying effects. The embodiments of the present application need to adjust various parameters taking into account the characteristics of the paint, environmental factors, and the performance of the spraying equipment.

[0416] S3.1: Develop a paint properties database containing physical and chemical property data for different types of paints;

[0417] 1. Data Collection: Collect technical data sheets of various paints from paint suppliers.

[0418] 2. Characteristic parameters: Record the following key parameters:

[0419] Viscosity; solid content; volatile organic compound (VOC) content; drying time; recommended spraying temperature range; recommended spraying humidity range; hiding power; adhesion;

[0420] 3. Database structure design: Use a relational database (such as PostgreSQL) to design a data structure suitable for storage and fast retrieval.

[0421] 4. Data standardization: Unify data from different sources into a standard format to ensure data consistency and comparability.

[0422] S3.2: Record the performance data of the paint under different environmental conditions;

[0423] 1. Identification of environmental factors: Determine the key environmental factors that affect the spraying effect, such as temperature, humidity, air flow speed, etc.

[0424] 2. Experimental Design: Design orthogonal experiments to systematically test the performance of paint under different environmental conditions.

[0425] 3. Data collection:

[0426] Spraying uniformity; drying time; surface gloss; adhesion; durability (such as weather resistance and abrasion resistance);

[0427] 4. Data analysis: Use statistical methods (such as analysis of variance ANOVA) to analyze the impact of environmental factors on spraying results.

[0428] 5. Performance model establishment: Based on experimental data, a mathematical model between environmental conditions and spraying effects is established.

[0429] S3.3: Establish a correlation model between paint characteristics and spraying parameters;

[0430] 1. Parameter identification: Determine the key spraying parameters that need to be optimized, such as:

[0431] Spray gun pressure; nozzle size; spray distance; spray speed; spray angle; flow rate.

[0432] 2. Data preprocessing: normalize and standardize the collected data.

[0433] 3. Feature Engineering: Construct composite features that reflect paint properties and environmental conditions.

[0434] 4. Model selection: Consider using one of the following machine learning models:

[0435] Multiple linear regression; Support vector regression (SVR); Random forest regression; Gradient boosting tree (such as XGBoost);

[0436] 5. Model training and verification:

[0437] The model was trained using the cross-validation method, and the root mean square error (RMSE) and coefficient of determination (R²) were used to evaluate the model performance.

[0438] 6. Model interpretation: Use SHAP (SHapley Additive exPlanations) values ​​to analyze the contribution of each feature to the prediction results.

[0439] S3.4: Initialize spraying parameters based on vehicle model characteristics and paint type;

[0440] 1. Vehicle feature extraction: vehicle size; surface complexity (such as curvature distribution); material type (such as metal, plastic parts);

[0441] 2. Parameter initialization rule formulation: Based on expert experience and historical data, heuristic rules for parameter initialization are formulated; for example, for large flat surfaces, a larger nozzle size and a higher flow rate are initialized.

[0442] 3. Initial parameter generation: Input the current vehicle model characteristics and the selected paint type, and apply the initialization rules to generate the initial spraying parameter set.

[0443] S3.5: Apply the anisotropic and isotropic total variation weighted negative binomial model to optimize the spraying parameters;

[0444] This model combines the total variation (TV) regularization in image processing and the negative binomial distribution in statistics, and can effectively handle the noise and uncertainty in the spraying parameter optimization problem.

[0445] S3.5.1: Construct an objective function that includes a data fidelity term, an anisotropic total variation regularization term, and an isotropic total variation regularization term;

[0446] The objective function is as follows:

[0447] min_x {‖Ax - b‖2² + λ1TV_aniso(x) + λ2TV_iso(x)};

[0448] in:

[0449] x is the spraying parameter to be optimized; A is the system matrix, reflecting the relationship between the parameter and the spraying effect; b is the target spraying effect; TV_aniso and TV_iso are the anisotropic and isotropic total variations, respectively; λ1 and λ2 are weight coefficients.

[0450] S3.5.2: Introduce a negative binomial distribution prior to model the sparsity and heavy-tailed characteristics of spraying parameters.

[0451] The negative binomial distribution can better capture the long-tail characteristics of parameter distribution and is more suitable for modeling the uncertainty in the actual spraying process than the Gaussian distribution.

[0452] Modify the objective function to:

[0453] min_x {‖Ax - b‖2² + λ1TV_aniso(x) + λ2TV_iso(x) - log p(x)};

[0454] where p(x) is the prior probability based on the negative binomial distribution.

[0455] S3.5.3: Design a weighting strategy to adaptively adjust the weights of anisotropic and isotropic regularization terms according to the local image structure;

[0456] 1. Local structure analysis:

[0457] Compute local gradients of parameter fields; estimate the degree of local anisotropy;

[0458] 2. Weight adjustment:

[0459] Increase the weight of TV_aniso in highly anisotropic regions; increase the weight of TV_iso in nearly isotropic regions;

[0460] S3.5.4: Estimate model hyperparameters using variational Bayesian inference.

[0461] 1. Define hyperparameters: including λ1, λ2, and the parameters of the negative binomial distribution.

[0462] 2. Construct variational lower bound:

[0463] L(q) = E_q[log p(x, y)] E_q[log q(x)];

[0464] where q(x) is an approximation to the posterior distribution p(x|y).

[0465] 3. Iterative optimization:

[0466] Update the variational distribution q(x), update the hyperparameters, and repeat until convergence.

[0467] S3.5.5: Iteratively optimize the spraying parameters by alternately minimizing the data fidelity term and the regularization term;

[0468] 1. Initialize parameter x 0 and hyperparameters.

[0469] 2. At the kth iteration:

[0470] a) Fix the regularization term and optimize the data fidelity term:

[0471] x^(k+1 / 2) = argmin_x ‖Ax - b‖2²;

[0472] b) Fix the data fidelity term and optimize the regularization term:

[0473] x^(k+1) = argmin_x {λ1TV_aniso(x) + λ2TV_iso(x) - log p(x)};

[0474] c) Update hyperparameters;

[0475] 3. Check the convergence conditions. If not, return to step 2.

[0476] S3.6: Solve optimization problems using the alternating direction multiplier method;

[0477] The alternating direction method of multipliers (ADMM) is an efficient optimization algorithm, which is particularly suitable for solving large-scale optimization problems containing multiple independent subproblems.

[0478] S3.6.1: Decompose the original optimization problem into several sub-problems, each sub-problem corresponding to a spraying parameter or parameter group;

[0479] The original question can be restated as:

[0480] min_x,z {f(x) + g(z)};

[0481] st Ax + Bz = c;

[0482] where x and z contain different spraying parameters, respectively, and f(x) and g(z) are the corresponding objective functions.

[0483] S3.6.2: Introduce the augmented Lagrangian function to transform the constraints into penalty terms.

[0484] Augmented Lagrangian function:

[0485] L_ρ(x, z, y) = f(x) + g(z) + y^T(Ax + Bz - c) + (ρ / 2)‖Ax + Bz - c‖2²;

[0486] where y is the Lagrange multiplier and ρ is the penalty parameter.

[0487] S3.6.3: Alternately update the primal variables, dual variables, and Lagrange multipliers;

[0488] At the kth iteration:

[0489] 1. x-update: x^(k+1) = argmin_x L_ρ(x, z^k, y^k);

[0490] 2. z-update: z^(k+1) = argmin_z L_ρ(x^(k+1), z, y^k);

[0491] 3. y-update: y^(k+1) = y^k + ρ(Ax^(k+1) + Bz^(k+1) - c);

[0492] S3.6.4: Design an adaptive step size strategy to dynamically adjust the update step size according to the convergence status of the optimization process;

[0493] 1. Calculate the primal residual r^k = ‖Ax^k + Bz^k - c‖2 and the dual residual s^k = ‖ρA^TB(z^k- z^(k-1))‖2.

[0494] 2. Update step size:

[0495] If r^k > μs^k, increase ρ: ρ^(k+1) = τρ^k;

[0496] If s^k > μr^k, reduce ρ: ρ^(k+1) = ρ^k / τ;

[0497] where μ > 1 and τ > 1 are preset parameters.

[0498] S3.6.5: Set multiple stopping criteria, including the primal residual, the dual residual, and the rate of change of the objective function value, to ensure that the algorithm converges to a high-quality solution;

[0499] Stopping criteria:

[0500] 1. ‖r^k‖2 ≤ ε_pri and ‖s^k‖2 ≤ ε_dual;

[0501] 2. |(f(x^k) + g(z^k)) - (f(x^(k-1)) + g(z^(k-1)))| / |f(x^(k-1)) + g(z^(k-1))| ≤ ε_obj;

[0502] where ε_pri, ε_dual and ε_obj are preset tolerances.

[0503] S3.7: Simulate the spraying process in a virtual environment and evaluate the effect of optimized parameters:

[0504] 1. Build a virtual spray painting environment: import the optimized digital vehicle model, set up a virtual spray gun, and simulate the characteristics of an actual spray gun; realize the physical simulation of paint particles, taking into account factors such as gravity and air resistance.

[0505] 2. Perform virtual spraying:

[0506] Perform virtual spraying according to the optimized path and parameters, and record the spraying amount and coverage of each surface point.

[0507] 3. Result analysis:

[0508] Calculate spray uniformity metrics, evaluate coverage in difficult areas like edges and corners, and estimate overall spray time and material usage.

[0509] 4. Parameter fine-tuning:

[0510] Based on the virtual spraying results, the parameters are slightly adjusted and the virtual spraying process is repeated until the desired effect is achieved.

[0511] S3.8: Associate the optimized parameters with the vehicle model information and save them in the system database.

[0512] 1. Data structure design:

[0513] Vehicle model information table: contains basic information such as vehicle model ID, name, size, etc.

[0514] Spraying parameter table: store the optimized spraying parameters;

[0515] Association table: establish a many-to-many relationship between vehicle models and spraying parameters;

[0516] 2. Data storage: Use transaction mechanism to ensure data integrity; implement version control and record historical changes of parameters;

[0517] 3. Index optimization: Create appropriate indexes for commonly used query fields to improve retrieval efficiency;

[0518] 4. Data compression: compress and store large numerical data (such as detailed path information);

[0519] 5. Access control: Implement role-based access control to ensure data security.

[0520] Through the above steps, the embodiment of the present application completes the process of optimizing process parameters based on paint characteristics. This optimization process takes into account the physical and chemical properties of the paint, the influence of environmental factors, and uses advanced mathematical models and optimization algorithms to obtain the optimal spraying parameters. These parameters will be used to guide the actual spraying process to achieve high-quality and efficient vehicle spraying.

[0521] For example, the following code will show how to implement the anisotropic and isotropic total variation weighted negative binomial model optimization, and how to use the alternating direction method of multipliers (ADMM) to solve the optimization problem:

[0522] ```Python

[0523] import numpy as np

[0524] import cvxpy as cp

[0525] from scipy import sparse

[0526] class SprayParameterOptimizer:

[0527] def __init__(self, n_params, lambda1, lambda2, rho):

[0528] self.n_params = n_params

[0529] self.lambda1 = lambda1

[0530] self.lambda2 = lambda2

[0531] self.rho = rho

[0532] def anisotropic_tv(self, x):

[0533] """Calculate anisotropic total variation"""

[0534] diff_x = np.diff(x, axis=0)

[0535] diff_y = np.diff(x, axis=1)

[0536] return np.sum(np.abs(diff_x)) + np.sum(np.abs(diff_y))

[0537] def isotropic_tv(self, x):

[0538] """Calculate isotropic total variation"""

[0539] diff_x = np.diff(x, axis=0)

[0540] diff_y = np.diff(x, axis=1)

[0541] return np.sum(np.sqrt(diff_x**2 + diff_y**2))

[0542] def negative_binomial_prior(self, x, r, p):

[0543] """Compute the negative binomial distribution prior"""

[0544] return np.sum(np.log(1 + x / r)) - np.sum(x * np.log(p / (1-p)))

[0545] def optimize(self, A, b, r, p, max_iter=1000, tol=1e-4):

[0546] """Using ADMM to optimize spraying parameters"""

[0547] m, n = A.shape

[0548] x = np.zeros(n)

[0549] z = np.zeros(n)

[0550] u = np.zeros(n)

[0551] for i in range(max_iter):

[0552] # Update x

[0553] x_new = np.linalg.solve(AT @ A + self.rho * np.eye(n),

[0554] AT @ b + self.rho * (z - u))

[0555] # Update z

[0556] z_new = self._prox_operator(x_new + u, self.lambda1, self.lambda2, r,p)

[0557] # Update u

[0558] u = u + x_new - z_new

[0559] # Check convergence

[0560] primal_residual = np.linalg.norm(x_new - z_new)

[0561] dual_residual = self.rho * np.linalg.norm(z_new - z)

[0562] if primal_residual < tol and dual_residual < tol:

[0563] print(f"Converged after {i+1} iterations")

[0564] break

[0565] x, z = x_new, z_new

[0566] return x

[0567] def _prox_operator(self, v, lambda1, lambda2, r, p):

[0568] """Approximation operator, combined with TV regularization and negative binomial prior"""

[0569] def obj_func(x):

[0570] return (0.5 * cp.sum_squares(x - v) +

[0571] lambda1 * self.anisotropic_tv(x) +

[0572] lambda2 * self.isotropic_tv(x) +

[0573] self.negative_binomial_prior(x, r, p))

[0574] x = cp.Variable(v.shape)

[0575] objective = cp.Minimize(obj_func(x))

[0576] problem = cp.Problem(objective)

[0577] problem.solve()

[0578] return x.value

[0579] # Example

[0580] if __name__ == "__main__":

[0581] # Simulate data

[0582] np.random.seed(42)

[0583] n_params = 100

[0584] A = np.random.randn(200, n_params)

[0585] true_x = np.random.rand(n_params)

[0586] b = A @ true_x + np.random.normal(0, 0.1, 200)

[0587] # Set optimizer parameters

[0588] lambda1 = 0.1

[0589] lambda2 = 0.1

[0590] rho = 1.0

[0591] r = 2 # Negative binomial distribution parameter

[0592] p = 0.5 # Negative binomial distribution parameter

[0593] # Create an optimizer and run the optimization

[0594] optimizer = SprayParameterOptimizer(n_params, lambda1, lambda2, rho)

[0595] optimized_params = optimizer.optimize(A, b, r, p)

[0596] print("Optimized parameters:", optimized_params)

[0597] print("True parameters:", true_x)

[0598] print("Mean squared error:", np.mean((optimized_params - true_x)**2))

[0599] ```

[0600] This code implements the optimization of spraying parameters based on the weighted negative binomial model of anisotropic and isotropic total variation, and uses the ADMM algorithm to solve the optimization problem. The following is a detailed explanation of the code:

[0601] 1. Class initialization (__init__ method):

[0602] Initialize the optimizer, setting the number of parameters and various regularization parameters.

[0603] 2. Total variation calculation (anisotropic_tv and isotropic_tv methods):

[0604] anisotropic_tv: Computes the total anisotropic variation, considering the sum of the absolute values ​​of the differences in the x and y directions.

[0605] isotropic_tv: Computes the isotropic total variation, considering the Euclidean norm of the differences in the x- and y-directions.

[0606] 3. Negative binomial prior (negative_binomial_prior method):

[0607] Implements the log-likelihood of the negative binomial distribution, useful for modeling the sparsity and heavy-tailed nature of parameters.

[0608] 4. Main optimization method (optimize method):

[0609] Implement the main loop of the ADMM algorithm. Alternately update the primal variable x, the auxiliary variable z, and the dual variable u. Update x using the linear system solver. Update z using the approximation operator. Check the convergence conditions (primal and dual residuals).

[0610] 5. Proximity operator (_prox_operator method):

[0611] The CVXPY library is used to solve the subproblem, combining TV regularization and negative binomial prior. The objective function of this subproblem includes data fitting terms, TV regularization terms, and negative binomial prior terms.

[0612] 6. Main program:

[0613] Generate simulated data. Set optimizer parameters. Create optimizer instance and run optimization. Compare optimization results with real parameters.

[0614] The above implementation covers the main contents of S3.5 and S3.6:

[0615] S3.5: Apply the negative binomial model weighted by anisotropic and isotropic total variation:

[0616] Combined anisotropic and isotropic total variation in the objective function (anisotropic_tv and isotropic_tv methods). Introduced negative binomial prior (negative_binomial_prior method). Implemented weighting strategy in _prox_operator method.

[0617] S3.6: Solve the optimization problem using the alternating direction multiplier method:

[0618] The optimization method implements the main steps of ADMM, including variable update and convergence check. An adaptive step size strategy can be implemented by adjusting the rho parameter. The stopping criteria (primal residual and dual residual) are set.

[0619] Assume that a new type of water-based metallic paint is used in the embodiment of the present application, which has the following characteristics:

[0620] Solid content: 35%;

[0621] Recommended spraying viscosity: 20-22 seconds (4mm Ford cup);

[0622] Flash point: >80°C;

[0623] Recommended dry film thickness: 40-50 microns;

[0624] Based on these characteristics, the parameter optimization system of the embodiment of the present application works as follows:

[0625] 1. Initial parameter settings: spray gun pressure: 2.5 bar; nozzle diameter: 1.3 mm; spray distance: 15 cm; spray gun moving speed: 300 mm / s.

[0626] 2. Virtual spray test: The system uses these initial parameters to perform spray tests in a virtual environment.

[0627] 3. Result analysis: Assume that the initial test results show:

[0628] Average dry film thickness: 38 microns (slightly below target); Coverage uniformity: 85% (room for improvement); Paint mist: high;

[0629] 4. Parameter adjustment:

[0630] Increase the spray gun pressure to 2.7 bar to improve atomization and coverage. Reduce the spray gun movement speed to 280 mm / s to increase the coating thickness in a single pass. Adjust the spray distance to 14 cm to reduce paint mist and improve transfer efficiency.

[0631] 5. Iterative optimization: The system will repeatedly perform virtual testing and parameter adjustments, fine-tuning parameters each time and evaluating the results.

[0632] 6. Environmental considerations: The system will also consider environmental factors. If the ambient temperature is high (such as 30°C), the diluent ratio may be slightly increased to maintain the appropriate viscosity. If the humidity is high (such as 80%), the fan setting may be adjusted to speed up the drying process.

[0633] 7. Final parameters: After multiple iterations, the system obtains the following optimal parameters:

[0634] Gun pressure: 2.65 bar;

[0635] Nozzle diameter: 1.3 mm (remains unchanged);

[0636] Spraying distance: 14.5 cm;

[0637] Spray gun moving speed: 290 mm / s;

[0638] Diluent ratio: 3% (fine-tuned according to ambient temperature);

[0639] Overlap rate of each spraying: 60%;

[0640] These optimized parameters should enable an average dry film thickness of 45 microns and improved coverage uniformity to 95%, while reducing the amount of paint mist to acceptable levels.

[0641] In this way, the system is able to find the optimal spraying parameters for each specific paint and vehicle type combination, ensuring high-quality spraying results and production efficiency.

[0642] S4: Position the vehicle and the robot performs the spraying process;

[0643] Before starting the actual spraying, accurate vehicle positioning is crucial. The embodiment of the present application adopts a multi-sensor fusion method to achieve high-precision positioning. First, a laser scanner is used to preliminarily locate the vehicle in the spray room. During this process, the embodiment of the present application aligns the scanned point cloud with the standard vehicle model and uses the iterative closest point (ICP) algorithm to accurately determine the position and posture of the vehicle.

[0644] In order to further improve positioning accuracy, the embodiment of the present application sets visual markers at key locations (such as the center of the wheel, the edge of the door, etc.). High-resolution cameras capture these markers and accurately locate these key points through computer vision algorithms. Combining the results of laser scanning and visual positioning, the embodiment of the present application uses a Kalman filter for data fusion to obtain the final high-precision positioning result.

[0645] After positioning is completed, the robot starts to perform the spraying task, such as Fig.12c As shown. The embodiment of the present application uses a 6-axis or 7-axis industrial robot, which has a high degree of flexibility and precision and can well execute complex spray paths. The robot's motion control system converts the previously optimized spray path into a series of joint motion instructions. In this process, the embodiment of the present application uses an inverse kinematics algorithm to calculate the robot joint angle corresponding to each path point, while taking into account the robot's kinematic constraints and workspace limitations.

[0646] In order to ensure the smoothness and uniformity of spraying, the embodiment of the present application realizes smooth control of speed and acceleration. A smooth speed curve is generated using a quintic polynomial interpolation method to ensure that the robot can smoothly transition when turning and changing direction. At the same time, the embodiment of the present application monitors the posture of the robot in real time to ensure that the spray gun always maintains the optimal spraying angle and distance.

[0647] During the spraying process, the embodiments of the present application also require precise control of the parameters of the spray gun. A closed-loop control system is used to adjust the pressure, flow rate and atomization degree of the spray gun in real time. This system includes multiple sensors, such as pressure sensors, flow meters and temperature sensors, which continuously monitor the spraying conditions and feed back to the controller. The controller uses a PID (proportional-integral-differential) algorithm to quickly respond to parameter changes to ensure that the spraying parameters are always kept within the optimal range.

[0648] S5: Real-time monitoring of the spraying process and quality control;

[0649] High-quality spraying requires not only precise planning and execution, but also a powerful real-time monitoring and quality control system. The embodiment of the present application develops a multi-level monitoring system to ensure the quality of spraying.

[0650] First, a high-speed camera and a spectrometer are installed at the end of the robot. These devices can capture images and spectral data of the sprayed surface in real time. Image data is used to analyze the uniformity and coverage of the spraying. The embodiment of the present application uses computer vision algorithms, such as edge detection and texture analysis, to identify possible spraying defects, such as orange peel, sagging, or insufficient coverage. Spectral data is used to monitor paint film thickness and color consistency. By comparing the difference between real-time spectral data and standard samples, the embodiment of the present application can quickly discover and correct color difference problems.

[0651] Secondly, the embodiment of the present application sets up multiple environmental sensors in the spraying room to monitor parameters such as temperature, humidity, and air flow speed. These data are transmitted to the central control system in real time through the Internet of Things (IoT) technology. Using machine learning models such as random forests or gradient boosting trees, the embodiment of the present application can predict the potential impact of environmental changes on spraying quality and make corresponding parameter adjustments.

[0652] In order to process a large amount of real-time data, the embodiment of the present application adopts an edge computing architecture. Preliminary data processing and analysis are performed on the local edge device, and only key information and abnormal situations are transmitted to the cloud, which greatly reduces data transmission delays and improves the response speed of the system.

[0653] When a spraying anomaly is detected, the system responds immediately. For minor deviations, the control system automatically adjusts the spraying parameters, such as changing the spray gun speed or flow rate. For more serious problems, the system pauses the spraying process and alerts the operator. The embodiment of the present application also develops an expert system that can provide operators with advice on how to deal with abnormal situations based on historical data and predetermined rules.

[0654] Quality control is not limited to the spraying process itself, but also includes inspection and evaluation after spraying. The embodiment of the present application uses 3D scanning technology to perform a comprehensive scan of the vehicle after spraying to generate a high-precision 3D model. By comparing the difference between this model and the ideal model, the embodiment of the present application can accurately quantify the spraying quality, including indicators such as paint film thickness distribution and surface finish. These data are not only used for quality assessment of the current batch, but will also be stored in a database for continuous improvement of the spraying process.

[0655] The embodiment of the present application also includes:

[0656] S6: Based on the results of 3D visual modeling, a virtual spraying environment is constructed, including a vehicle geometry model and a spraying equipment model;

[0657] The construction of a virtual spraying environment is a key step in optimizing the spraying process. Based on the previously obtained high-precision 3D vehicle model, the embodiment of the present application uses computer graphics technology to construct a realistic virtual environment. This environment not only includes the vehicle geometry model, but also simulates the physical layout of the spraying room and the characteristics of the spraying equipment.

[0658] In a virtual environment, the embodiment of the present application uses a physics engine to simulate the movement of paint particles. Taking into account factors such as gravity, air resistance, and electrostatic effects, the embodiment of the present application can accurately simulate the entire process of paint from the spray gun to the surface of the vehicle body. This simulation allows the embodiment of the present application to evaluate the effects of different spraying strategies without consuming actual materials.

[0659] In order to improve the accuracy of the simulation, the embodiment of the present application collects a large amount of actual spraying data, including the flow characteristics and atomization patterns of different paints under various conditions. This data is used to train a machine learning model, such as a deep neural network, to predict the behavior of the paint under different conditions. This model is integrated into the virtual environment, greatly improving the authenticity of the simulation.

[0660] The virtual environment also includes a detailed spray gun model. The embodiment of the present application simulates various parameters of the spray gun, such as nozzle shape, pressure distribution, atomization pattern, etc. By adjusting these parameters, the operator can test different spray gun settings in the virtual environment and find the best configuration.

[0661] In addition, the embodiment of the present application implements real-time rendering technology in a virtual environment, allowing the operator to observe the spraying process from any angle. Using a real-time ray tracing algorithm, the embodiment of the present application can accurately simulate the optical properties of paint, including reflection, refraction, and scattering effects. This enables the operator to intuitively evaluate the spraying effect, which is particularly helpful in predicting the final surface gloss and color.

[0662] S6.1: Discretize the vehicle surface into grid points, each point contains position, normal vector and current spraying status information;

[0663] In order to accurately simulate the spraying process in a virtual environment, the embodiment of the present application needs to discretize the vehicle surface into high-density grid points. This process uses an adaptive meshing algorithm, using denser grids in areas with large curvature or more details, and using relatively sparse grids in flat areas. This method not only ensures the accuracy of the simulation, but also optimizes the use of computing resources.

[0664] Each grid point not only contains its 3D spatial position, but also records the surface normal vector information. The normal vector is crucial for simulating the spraying process because it determines the collision angle between the paint particles and the surface, which in turn affects the uniformity and adhesion of the spraying. The embodiment of the present application uses the barycentric coordinate method to calculate the normal vector of each grid point, ensuring the smoothness and accuracy of the normal vector field.

[0665] In addition, each grid point also maintains the current spraying status information. This includes the accumulated paint thickness, paint type, dryness, etc. During the simulation, this information is updated in real time. In order to process a large amount of grid point data, the embodiment of the present application adopts a space partitioning technology, such as an octree or a KD tree, to optimize the efficiency of spatial query.

[0666] This fine surface discretization and rich point attribute information provide a solid foundation for subsequent spraying simulation and optimization, which enables the embodiment of the present application to accurately track every detail of the spraying process, thereby achieving highly realistic virtual spraying simulation.

[0667] S6.2: Based on the fluid dynamics model, simulate the spray parameters of the spray gun, including spray cone angle, flow rate and distance attenuation parameters;

[0668] In a virtual environment, accurately simulating the behavior of a spray gun is key. This embodiment of the application develops a complex spray gun model based on the principles of computational fluid dynamics (CFD). This model takes into account factors such as the internal structure of the spray gun, nozzle shape, and pressure distribution, and can accurately predict the morphology and characteristics of the spray.

[0669] The spray cone angle is an important parameter that determines the range of spray coverage. The embodiment of the present application uses the Navier-Stokes equation to simulate the behavior of the fluid at the nozzle, combined with turbulence models (such as the k-ε model) to capture complex fluid dynamics effects. By adjusting the pressure of the spray gun, the nozzle shape and the atomizing airflow, the embodiment of the present application can accurately control the spray cone angle.

[0670] Flow control is the key to ensure uniformity of spraying. The embodiment of the present application establishes a dynamic flow model, which not only takes into account the spray gun settings, but also includes the influence of factors such as the viscosity and temperature of the paint. Using the Bernoulli equation and empirical formula, the embodiment of the present application can accurately predict the flow changes under different operating conditions.

[0671] The distance attenuation parameter simulates the characteristics of spray intensity changing with distance. The embodiment of the present application adopts a Gaussian attenuation model and is calibrated by a large amount of experimental data. This model takes into account air resistance, gravity and electrostatic effects, and can accurately predict the distribution and deposition pattern of paint particles in the air.

[0672] By integrating these parameters, the virtual spray gun model of the embodiment of the present application can simulate the actual spraying process with high fidelity, providing a reliable basis for subsequent optimization.

[0673] S6.3: Design a reinforcement learning agent whose state space includes the current spray gun position, posture, distance from the vehicle surface, and the spray state of the surrounding area;

[0674] In order to optimize the spraying process, the embodiment of the present application designs an intelligent agent based on deep reinforcement learning. The goal of this agent is to learn the optimal spraying strategy to achieve high-quality and efficient spraying.

[0675] The design of the state space is the key to the success of reinforcement learning. The state space of the embodiment of the present application contains the following key information:

[0676] 1. Gun position: Use three-dimensional coordinates (x, y, z) to represent the exact position of the gun in space.

[0677] 2. Spray gun posture: Use quaternion to represent the rotation of the spray gun, which can avoid the gimbal lock problem better than Euler angle.

[0678] 3. Distance from vehicle surface: Real-time distance data obtained using laser ranging or visual estimation.

[0679] 4. Spraying status of the surrounding area: In the embodiment of the present application, the area surrounding the spray gun is discretized into grids, and each grid stores information such as the current spraying thickness and uniformity.

[0680] In order to process this high-dimensional state space, the embodiment of the present application uses a convolutional neural network (CNN) to extract spatial features and a long short-term memory network (LSTM) to capture the temporal dependency of the spraying process. This structure enables the agent to understand complex spatial-temporal patterns and make smarter decisions.

[0681] S6.4: Define the action space, including the movement direction and speed of the spray gun, and the control of the spray switch;

[0682] The design of the action space needs to balance the precision of control and the difficulty of learning. The action space of the embodiment of the present application includes the following dimensions:

[0683] 1. Movement direction: The movement direction is expressed using the spherical coordinate system (r, θ, φ), which is more suitable for expressing the movement of the spray gun than the Cartesian coordinate system.

[0684] 2. Moving speed: Discretize the speed into multiple levels, such as slow, medium, and fast.

[0685] 3. Spray switch: binary action, turn spraying on or off.

[0686] 4. Adjustment of spray gun parameters: including fine-tuning of flow rate and atomization degree.

[0687] In order to make the learning process more efficient, the embodiment of the present application adopts a hierarchical reinforcement learning method. The top-level strategy is responsible for large-scale path planning, while the bottom-level strategy is responsible for local fine control. This hierarchical structure greatly reduces the complexity of learning, allowing the agent to master complex spraying techniques more quickly.

[0688] S6.5: Design a composite reward function based on spray uniformity, coverage, time efficiency, and paint usage;

[0689] The design of the reward function directly affects the results of reinforcement learning. The reward function of the embodiment of the present application takes into account multiple key factors:

[0690] R = w1 * Uniform + w2 * Coverage + w3 * Efficiency - w4 * PaintUsage+ w5 * QualityBonus;

[0691] in:

[0692] Uniform: Spray uniformity, measured using the coefficient of variation (standard deviation divided by the mean). Coverage: Coverage, calculated as the percentage of the surface that reaches the target thickness. Efficiency: Time efficiency, inversely proportional to the time required to complete the spraying. PaintUsage: The amount of paint used, to encourage material conservation. QualityBonus: Quality bonus, given when a specific quality standard is achieved. Weights w1 to w5 are used to balance the importance of different objectives, and these weights can be adjusted according to specific needs.

[0693] In order to deal with the problem of sparse rewards, the embodiment of the present application also introduces some intermediate rewards. For example, when the agent successfully avoids obstacles or maintains an ideal spraying distance, it will receive a small positive reward. This helps to speed up the learning process, especially in the early stages of training.

[0694] S6.6: Using the PPO algorithm to train the reinforcement learning agent to optimize the spraying strategy;

[0695] The present embodiment selects the Proximal Policy Optimization (PPO) algorithm to train the reinforcement learning agent. The PPO algorithm performs well in terms of stability and sample efficiency, and is particularly suitable for continuous control tasks such as spray optimization.

[0696] The training process is divided into the following steps:

[0697] 1. Data collection: The agent performs the painting task in the virtual environment and collects data on state, action, reward, and next state.

[0698] 2. Odds Estimation: The odds value at each time step is calculated using Generalized Odds Estimation (GAE), which helps reduce the variance of the value estimate.

[0699] 3. Policy update: Use the collected data to update the policy network. The core idea of ​​PPO is to ensure stable updates by limiting the difference between the old and new policies. This embodiment of the application uses a clipping objective function:

[0700] L = min(r_t(θ) * A_t, clip(r_t(θ), 1-ε, 1+ε) * A_t);

[0701] where r_t(θ) is the probability ratio of the new and old policies, A_t is the estimated advantage, and ε is the clipping parameter.

[0702] 4. Value function update: The value function is also updated to provide a more accurate baseline estimate.

[0703] 5. Environment reset: Reset the virtual environment regularly to ensure that the agent can adapt to different initial conditions.

[0704] In order to speed up the training process, the embodiment of the present application adopts a distributed training framework. Multiple agents explore in different virtual environment instances at the same time, and the collected experience is aggregated to the central learner for updating. This greatly improves the efficiency of data collection and the speed of learning.

[0705] In addition, the embodiment of the present application also implements a progressive learning strategy. In the early stages of training, the embodiment of the present application starts with simple spraying tasks, such as flat spraying. As the agent's ability improves, the complexity of the task is gradually increased, and finally transitions to complete car body spraying. This progressive learning method helps the agent master complex spraying techniques more effectively.

[0706] Through repeated training and optimization, the reinforcement learning agent can eventually master efficient and high-quality spraying strategies and provide valuable guidance for actual production. This method based on virtual environment and reinforcement learning not only greatly reduces the need for actual experiments, but also continuously optimizes the spraying process to adapt to different vehicle models and spraying requirements.

[0707] In the virtual spraying environment of the embodiment of the present application, the embodiment of the present application accurately replicates the 3D model of a mid-size sedan. This model includes all details, from the main curved surfaces of the car body to small parts such as door handles and rearview mirrors. The embodiment of the present application also simulates the environment of the spraying room, including changes in lighting conditions, temperature and humidity. The spray gun model is based on the actual HVLP (high volume low pressure) spray gun, and accurately simulates its spray pattern, atomization effect and pressure changes.

[0708] The state space of the reinforcement learning agent includes the six-dimensional position and posture information of the spray gun, and a 32x32x32 three-dimensional grid, each grid point stores the current spray thickness. The action space includes the movement of the spray gun in the three directions of x, y, and z (continuous values ​​from -1 to 1 in each direction), and the rotation of two angles (pitch and yaw, ranging from -30 degrees to 30 degrees). In addition, a binary action is included to control the switch of the spray gun.

[0709] The design of the reward function takes into account multiple factors. For spray uniformity, the embodiment of the present application calculates the coefficient of variation of the spray thickness of the entire vehicle body surface. The smaller the coefficient of variation, the higher the reward. Coverage is measured by calculating the proportion of the surface that reaches the target thickness (such as 50 microns). Efficiency is inversely proportional to the total time required to complete the spraying. In order to control material usage, the embodiment of the present application imposes a penalty on areas that exceed the target thickness by 20%. In addition, the embodiment of the present application also adds a small reward item to encourage the spray gun to maintain an optimal angle perpendicular to the vehicle body surface.

[0710] During the training process of the PPO algorithm, the embodiment of the present application uses a dual network structure. The policy network uses a neural network containing three convolutional layers and two fully connected layers to output actions directly from the state space. The value network has a similar structure, but finally outputs a scalar value representing the estimated value of the current state. The embodiment of the present application uses an experience replay buffer of size 10,000 to update the network every 4 time steps. The learning rate starts at 1e-4 and is gradually reduced using a cosine annealing strategy.

[0711] During the training process, the embodiment of the present application observed that the agent gradually mastered some interesting strategies. For example, it learned to slow down when dealing with complex areas such as vehicle corners, and speed up on the flat sides of the car body. It also developed a zigzag spraying pattern that improved uniformity while ensuring coverage. Interestingly, the agent autonomously discovered a "double-layer spraying" technique similar to that used by professional human spray painters - first quickly spraying a thin film, and then spraying another layer on it to achieve the ideal thickness.

[0712] To increase the robustness of the training, the embodiment of the present application introduces random perturbations, such as sudden airflow changes or gun pressure fluctuations. This helps the agent learn to quickly adapt to changing conditions. The embodiment of the present application also implements curriculum learning, starting with simple planes and gradually increasing to complete car bodies. This greatly speeds up learning and improves the final performance.

[0713] After training for about 1 million simulated sprayings, the reinforcement learning agent of the embodiment of the present application achieved performance close to that of human experts. On the test set, it achieved 98% uniform coverage while reducing material usage by 15% and shortening spraying time by 20%. These results show that methods based on virtual environments and reinforcement learning have great potential in optimizing complex industrial processes.

[0714] The present application also provides an intelligent vehicle painting system based on 3D vision and automatic trajectory planning, such as Figure 5 and Figure 6 As shown, for executing the above method, the system includes:

[0715] 3D visual scanner for capturing vehicle models;

[0716] A main controller, used to generate a spraying path and control the robot to spray;

[0717] The robot body is used for vehicle spraying;

[0718] A paint spray gun, arranged on the robot body;

[0719] The external bracket is used to fix and support the 3D vision scanner, the main controller and the robot body.

[0720] in,

[0721] 1. 3D vision scanner, such as Figure 8 As shown: As the "eyes" of the system, it is used to accurately capture the three-dimensional structure of the vehicle. The present application embodiment may use multiple high-precision structured light 3D scanners, such as the Keyence LJ-X8000 series equipped with blue light LEDs. These scanners are usually placed in different locations in the spray booth to ensure that all surface details of the vehicle can be captured.

[0722] Specific features include:

[0723] Scanning accuracy: up to ±0.5μm; scanning speed: 64,000 3D points can be collected per second; field of view: about 150mm x100mm; working distance: about 200mm. These scanners will work simultaneously to quickly generate complete 3D point cloud data of the vehicle.

[0724] 2. Main controller, such as Fig. 9 Shown: This is the "brain" of the system, responsible for data processing, path planning, and overall control. It may use a high-performance industrial computer such as a Dell Precision 7920 workstation equipped with an Intel Xeon processor and an NVIDIA Quadro RTX GPU.

[0725] The main functions of the main controller include:

[0726] Process raw data from 3D scanners to build accurate digital vehicle models; run complex path planning algorithms to generate optimal spray paths; execute parameter optimization algorithms to adjust spray parameters based on paint characteristics and environmental conditions; control the robot's movement and spraying process in real time; process feedback from various sensors for real-time adjustments and quality control.

[0727] In terms of software, it will run customized industrial control software that integrates modules such as computer vision, machine learning, and robotic control.

[0728] 3. Robot Body: This is the "arm" of the system and performs the actual spraying action. It may use a 6-axis or 7-axis industrial robot designed specifically for spraying, such as the Han's or ABBIRB 5500 series.

[0729] Specific features may include: load capacity: about 13kg (enough to support the spray gun and paint supply system); working range: up to about 2.7m; repeatability: about ±0.15mm; explosion-proof level: meet the requirements of the spraying environment. The robot should be equipped with high-precision servo motors and encoders to accurately execute complex spraying paths.

[0730] 4. Paint spray gun, such as Fig.10 Shown: This is the "tip" of the system, which is directly responsible for spraying. It may be used with a digital spray gun such as the SATAjet X 5500 HVLP.

[0731] Specific features include:

[0732] Atomization technology: HVLP (High Volume Low Pressure); Nozzle size: Replaceable, ranging from 1.1mm to 1.5mm; Working pressure: Adjustable, up to 2.0 bar; Transfer efficiency: up to more than 80%.

[0733] Spray guns may come with automatic cleaning systems and quick-change mechanisms for easy color switching and maintenance.

[0734] 5. External bracket, e.g. Figure 7 Shown: This is the "skeleton" of the entire system, providing stable support and precise positioning. It may be a custom steel structural frame with a high-rigidity, low-vibration design.

[0735] Specific features include:

[0736] Material: high-strength carbon steel with anti-corrosion surface treatment; Structure: modular design for easy installation and adjustment; Vibration reduction system: equipped with active vibration reduction device to minimize the impact of environmental vibration on accuracy; Adjustability: the installation position of each component can be precisely adjusted to suit different models.

[0737] The bracket also integrates a cable management system, pneumatic and paint supply lines, and safety guards.

[0738] The whole system can be installed in different paint shops, such as Fig.11 As shown, the workshop is a confined space, and the overall workflow of the system is as follows:

[0739] 1. The vehicle enters the spray booth and is quickly scanned by a 3D scanner.

[0740] 2. The main controller processes the scan data, generates a digital model, and plans the spray path.

[0741] 3. The robot starts spraying according to the planned path, and the main controller monitors and adjusts parameters in real time.

[0742] 4. During the whole process, external brackets ensure that all equipment remains stable and precisely positioned.

[0743] 5. After spraying is completed, the system performs self-cleaning and prepares for the next vehicle to be sprayed.

[0744] This integrated system enables highly automated, precise and consistent spraying results, greatly improving production efficiency and quality.

[0745] The system of the embodiments of the present disclosure can execute the method provided by the embodiments of the present disclosure, and the implementation principles are similar. The actions performed by each module in the system of each embodiment of the present disclosure correspond to the steps in the method of each embodiment of the present disclosure. For the detailed functional description of each module of the system, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0746] The above is only an optional implementation method for some implementation scenarios of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the scheme of the present disclosure, other similar implementation methods based on the technical ideas of the present disclosure are also within the protection scope of the embodiments of the present disclosure.

Claims

1. An intelligent vehicle spraying method based on 3D vision and automatic trajectory planning, characterized in that: The following steps are involved: Use 3D vision technology to collect vehicle surface data and build a digital vehicle model; Performing automatic trajectory planning based on the digital vehicle model to generate a spraying path; Optimize process parameters according to paint characteristics; The vehicle is positioned and the robot performs the painting process; Monitor the spraying process in real time and perform quality control; Among them, 3D vision technology is used to collect vehicle surface data and establish a digital vehicle model, including: Deploy multiple 3D scanners to scan all around the vehicle; Project structured light onto the vehicle surface and calculate the surface depth information by analyzing the deformation of the structured light; Projecting phase-shifted patterns at different frequencies and improving the robustness of depth measurements by synthesizing multi-frequency phase maps; Apply L1-Curl regularized divergence-free wavelet reconstruction technology to process the original scan data and obtain the reconstructed depth map; Merge the depth maps corresponding to the reconstructed multi-view scanning data to generate a complete point cloud model; The point cloud model is converted into a triangular mesh model and optimized through an edge-preserving smoothing algorithm and an adaptive mesh subdivision technique to reduce the data volume while retaining key geometric features.

2. The method according to claim 1, characterized in that The L1-Curl regularized divergence-free wavelet reconstruction technology is applied to process the original scan data, including: Preprocess the depth map obtained from the original scan, including denoising and outlier removal; Converting the preprocessed depth map into a wavelet domain representation; In the wavelet domain, an L1-Curl regularization constraint is applied to maintain edge sharpness and surface details in the depth map; Construct an objective function based on L1-Curl regularization, including data fidelity terms and regularization terms; The constructed optimization problem is solved using the alternating direction multiplier method ADMM, where the optimization goal is to minimize the objective function; By means of an ADMM iteration process, wavelet coefficients, auxiliary variables and Lagrange multipliers of the depth map are alternately updated; The optimized wavelet coefficients are inversely transformed back to the spatial domain to obtain the reconstructed depth map.

3. The method according to claim 1, characterized in that The process parameter optimization according to the paint characteristics includes: Establish a paint properties database containing data on the physical and chemical properties of different types of paints; Record the performance data of paint under different environmental conditions; Establish a correlation model between paint characteristics and spraying parameters; Initialize spraying parameters based on vehicle model characteristics and paint type; Applying anisotropic and isotropic total variation weighted negative binomial model to optimize the spraying parameters; Solve optimization problems using the alternating direction multiplier method; Simulate the spraying process in a virtual environment and evaluate the effect of optimized parameters; The optimized parameters are associated with the vehicle model information and saved in the system database.

4. The method according to claim 3, characterized in that The application of the anisotropic and isotropic total variation weighted negative binomial model to optimize the spraying parameters includes: Construct an objective function, including data fidelity term, anisotropic total variation regularization term and isotropic total variation regularization term; The negative binomial distribution prior is introduced to model the sparsity and heavy-tail characteristics of spraying parameters; Design a weighting strategy to adaptively adjust the weights of anisotropic and isotropic regularization terms according to the local image structure; Use variational Bayesian inference to estimate the model's hyperparameters; The spraying parameters are iteratively optimized by alternately minimizing the data fidelity term and the regularization term.

5. The method according to claim 3, characterized in that: The method of using the alternating direction multiplier method to solve the optimization problem includes: Decompose the original optimization problem into several sub-problems, each sub-problem corresponds to a spraying parameter or parameter group; Introduce the augmented Lagrangian function to transform the constraints into penalty terms; Alternately update the primal variables, dual variables, and Lagrange multipliers; Design an adaptive step size strategy to dynamically adjust the update step size according to the convergence status during the optimization process; Multiple stopping criteria are set, including the primal residual, dual residual, and the rate of change of the objective function value, to ensure that the algorithm converges to a high-quality solution.

6. An intelligent vehicle spraying system based on 3D vision and automatic trajectory planning, characterized in that: For implementing the method described in any one of claims 1 to 5, the system comprises: 3D visual scanner for capturing vehicle models; A main controller, used to generate a spraying path and control the robot to spray; The robot body is used for spraying vehicles; A paint spray gun, arranged on the robot body; The external bracket is used to fix and support the 3D vision scanner, the main controller and the robot body.

Images