Intelligent Painting Method and System Based on Optical Processing

Abstract

The present invention relates to an intelligent painting method and system based on optical processing. The intelligent painting method described in the present invention includes: real-time collecting holographic light field data, user interaction signals, and environmental parameters through a multi-dimensional sensor; performing time-frequency alignment and noise filtering on multi-source heterogeneous data to generate holographic image data representing multi-view projection deformation gradients and user intentions; adaptively compensating for projection geometric errors based on a neural radiance field model and a dynamic deformation field function, generating phase correction parameters and optimizing the energy distribution of holographic layers to suppress inter-layer crosstalk interference; combining a photon-electron clock synchronization mechanism and a predictive rendering algorithm to achieve ultra-low latency interactive projection in virtual and real spaces. This method deeply integrates optical processing and intelligent algorithms, solves the problems of multi-view deformation accumulation, holographic layer crosstalk, and excessive interaction latency, and significantly improves the positioning accuracy, projection clarity, and real-time response performance of 3D painting in complex scenarios.

Description

Technical Field

[0001] The present invention relates to the field of optical processing and 3D interaction technology, and in particular to an intelligent painting method and system based on optical processing. Background Art

[0002] With the growing demand for high-precision 3D interaction in fields such as virtual reality, digital art creation, and industrial design, intelligent painting technologies based on optical processing have gradually become a core means of achieving the fusion of virtual and real creation. This type of technology integrates optical sensing, holographic projection, and real-time computing models to transform the user's physical painting movements into dynamic virtual brushstrokes in three-dimensional space, and uses holographic imaging technology to present a stereoscopic visual effect. In practical applications, such systems must address core challenges such as the geometric consistency of multi-view projection, the optical adaptability of complex media, and the real-time nature of human-computer interaction. The technical difficulty lies in how to achieve a high-precision, low-latency immersive creative experience through the coordinated optimization of optical processing and intelligent algorithms.

[0003] However, existing intelligent painting systems based on optical processing still have significant defects in complex scenes. First, due to the insufficient deformation compensation capability of multi-view projection, when the user switches the viewing angle or the curvature of the medium surface changes, millimeter-level offsets are likely to occur between the virtual brushstrokes and the real space, resulting in a significant decrease in painting positioning accuracy. Secondly, in holographic layered rendering scenes, the projection light beams of different depths of field layers produce cross-interference due to the diffraction effect, resulting in ghosting of auxiliary lines or model contours, seriously reducing the clarity of complex structure drawing. In addition, the existing system has a high response delay to user interaction signals, which makes it difficult to match the real-time requirements of human natural painting, especially when quickly outlining or correcting details. It is easy to produce a sense of operational discontinuity. These problems seriously limit the application value of such technologies in high-precision scenarios such as medical visualization and industrial simulation. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide an intelligent painting method and system based on optical processing that can dynamically compensate for multi-view deformation, effectively suppress inter-layer crosstalk and achieve ultra-low latency rendering.

[0005] The purpose of the present invention is achieved by the following scheme:

[0006] In a first aspect, the present invention provides an intelligent painting method based on optical processing, comprising the following steps:

[0007] S1: Acquire holographic light field data, user interaction signals and environmental parameters based on multi-dimensional sensors;

[0008] S2: Perform time-frequency alignment and noise filtering on the holographic light field data, user interaction signals, and environmental parameters to generate holographic image data, which is used to indicate the geometric deformation gradient of multi-view projection and the user interaction intention;

[0009] S3: Calculate the multi-view projection deformation gradient of the holographic image data based on the neural radiance field model and the deformation field function to generate phase compensation parameters;

[0010] S4: Perform holographic layer energy optimization on the phase compensation parameters based on the orthogonal polarization encoding algorithm and the dynamic coherence control algorithm to generate an optimized holographic layer distribution;

[0011] S5: Align the photon-electron clock for the optimized holographic layer distribution based on the predictive rendering algorithm, and combine to generate a low-latency holographic projection output, which is used for the interactive display of real-time 3D painting.

[0012] In one embodiment, S2 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0013] S21: Perform spectral filtering on the ambient light of the environmental parameters based on an optical filter to generate an effective projection wavelength range;

[0014] S22: Based on the stylus pose signal of the user interaction signal and the holographic light field data, perform time synchronization alignment processing through the time reference signal generated by an optical crystal oscillator to generate synchronized data;

[0015] S23: Process the synchronized data and the effective projection wavelength range based on a data dimensionality reduction algorithm, compress the four-dimensional holographic light field data to a 256-dimensional low-dimensional feature vector, and generate holographic image data.

[0016] In one embodiment, S3 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0017] S31: Based on the multi-view images of the holographic image data, construct a three-dimensional geometric deformation field through the neural radiance field model to generate initial deformation gradient data;

[0018] S32: Perform non-linear correction on the initial deformation gradient data based on the deformation field function to generate corrected gradient data;

[0019] S33: Calculate the phase modulation parameters for the corrected gradient data to generate phase compensation parameters.

[0020] In one embodiment, S31 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0021] S311: Extract convolutional neural network features from multi-view images in the holographic image data to generate multi-scale feature maps;

[0022] S312: Based on the neural radiance field model, perform light field integration calculations for three-dimensional spatial coordinates and viewing directions on the multi-scale feature maps to generate initial light field distribution data;

[0023] S313: Calculate the partial derivatives of the initial light field distribution data based on the deformation field function to generate initial deformation gradient data covering three-dimensional geometric deformation gradients.

[0024] In one embodiment, step S4 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0025] S41: Based on the orthogonal polarization algorithm, perform polarization assignment processing by allocating non-overlapping polarization direction references to each holographic layer of the phase compensation parameters to generate orthogonal polarization references for each holographic layer;

[0026] S42: Based on the dynamic coherence control algorithm, dynamically adjust the spectral width of the laser light source of the orthogonal polarization reference to generate optimized laser parameters;

[0027] S43: Based on Jones vector operations, perform optimization processing on the energy matching of the optimized laser parameters to generate an optimized holographic layer distribution that suppresses crosstalk in non-holographic layers.

[0028] In one embodiment, step S43 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0029] S431: Based on Jones matrix operations, perform polarization state matching processing on the optimized laser parameters to generate polarization matching parameters;

[0030] S432: Based on the energy distribution optimization algorithm, perform energy weight assignment processing on the polarization matching parameters to generate an optimized energy distribution;

[0031] S433: Reconstruct the phase and amplitude distributions of the holographic layers according to the optimized energy distribution to generate an optimized holographic layer distribution that suppresses crosstalk in non-holographic layers.

[0032] In one embodiment, step S5 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0033] S51: Based on the optimized holographic layer distribution, establish a unified clock for the optical and electronic devices and perform clock alignment processing on the unified clock to generate a unified time reference;

[0034] S52: Perform trajectory prediction processing on the stylus acceleration and speed data of the user interaction signal based on a unified time reference to generate future phase distribution parameters;

[0035] S53: Perform 4K resolution rendering processing on the user focus area of the optimized holographic layer based on the future phase distribution parameters to generate a low-latency holographic projection output.

[0036] In a second aspect, the present invention provides an optical processing-based intelligent painting system, including

[0037] A data acquisition module, configured to acquire holographic light field data, user interaction signals, and environmental parameters based on a multi-dimensional sensor;

[0038] A data preprocessing module, configured to perform time-frequency alignment and noise filtering processing on the holographic light field data, user interaction signals, and environmental parameters to generate holographic image data, and the holographic image data is used to indicate the geometric deformation gradient of multi-view projection and the user interaction intention;

[0039] A holographic gradient calculation module, configured to perform multi-view projection deformation gradient calculation on the holographic image data based on a neural radiance field model and a deformation field function to generate phase compensation parameters;

[0040] A holographic distribution optimization module, configured to perform holographic layer energy optimization processing on the phase compensation parameters based on an orthogonal polarization encoding algorithm and a dynamic coherence control algorithm to generate an optimized holographic layer distribution;

[0041] A holographic rendering output module, configured to perform photon-electron clock alignment on the optimized holographic layer distribution based on a predictive rendering algorithm, and combine to generate a low-latency holographic projection output, and the holographic projection output is used for the interactive display of real-time 3D painting.

[0042] In a third aspect, the present application provides a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements any one of the above optical processing-based intelligent painting methods.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements any one of the above optical processing-based intelligent painting methods.

[0044] In summary, the intelligent painting method based on optical processing provided by the present invention obtains comprehensive data information through multi-dimensional sensors. After time-frequency alignment and noise filtering processing, the accuracy and reliability of the data are ensured. The neural radiance field model and deformation field function are used to deeply process the holographic image data to generate phase compensation parameters, further optimizing the energy distribution of the holographic layer. Finally, through the predictive rendering algorithm and photon-electron clock alignment technology, low-latency holographic projection output can be achieved, providing users with a high-quality real-time 3D painting interactive display experience. This process has significant beneficial effects in improving holographic projection accuracy, enhancing user interactivity, reducing system latency, etc., promoting the application and development of holographic technology in the field of real-time interactive display.

[0045] For better understanding and implementation, the present invention will be described in detail below with reference to the accompanying drawings. Brief Description of the Drawings

[0046] Figure 1 It is a schematic flowchart of the intelligent painting method based on optical processing provided by an embodiment of the present application;

[0047] Figure 2 It is a schematic flowchart of generating phase compensation parameters provided by an embodiment of the present application;

[0048] Figure 3 It is a schematic flowchart of generating an optimized holographic layer distribution provided by an embodiment of the present application;

[0049] Figure 4 It is a schematic structural diagram of an intelligent painting system based on optical processing provided by another embodiment of the present application. Detailed Embodiments

[0050] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Preferred embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the invention more thorough and comprehensive.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0052] In one embodiment, as Figure 1As shown, an intelligent painting method based on optical processing is provided. In this embodiment, this method is exemplified by being applied to a terminal. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0053] S1: Obtain holographic light field data, user interaction signals, and environmental parameters based on a multi-dimensional sensor.

[0054] Specifically, for the holographic light field data, the system uses a high-precision CMOS image sensor, which has the characteristics of high resolution, high sensitivity, and low noise, and can accurately capture light field information. The sensor array is installed at the data acquisition end of the painting system in a specific geometric arrangement, capturing the light distribution in the scene from multiple perspectives simultaneously, ensuring that the obtained holographic light field data has sufficient spatial and angular resolution to meet the accuracy requirements of subsequent three-dimensional reconstruction and deformation compensation.

[0055] When the user performs a physical painting action in the painting space, light propagates from different directions and is captured by the sensor array. Each sensor records information such as the intensity, phase, and direction of the light, forming the original light field data. These data contain the three-dimensional structure of the painting object and the optical characteristics information of the surrounding environment, providing basic data support for subsequent holographic image generation and deformation gradient calculation.

[0056] For the user interaction signals, the system uses a professional hand-drawn digital tablet or a pressure-sensitive stylus. These devices can real-time sense parameters such as the pressure, speed, and angle of the user's painting action and convert them into electrical signals. At the same time, the system can also integrate gesture recognition sensors, further enriching the dimension of the interaction signals by capturing the user's hand movements and posture changes, and realizing more natural and flexible painting operation control.

[0057] The interaction signals generated by the user during the painting process, such as the pressure change of the brushstroke, the contact position and movement trajectory of the pen tip with the painting surface, etc., are collected in real-time by the interaction device. The collected analog signals are precisely digitized by an analog-to-digital converter (ADC) and converted into digital signals for subsequent computer processing and analysis. These digital signals contain the direct expression of the user's painting intention and are the key data source for the system to understand the user's creative intention.

[0058] To address environmental parameters, a variety of environmental monitoring sensors are deployed around the painting system, including light sensors, temperature sensors, and humidity sensors. Light sensors measure the intensity and spectral distribution of ambient light, enabling the system to appropriately compensate for brightness and color in the holographic projection based on actual ambient light conditions. Temperature and humidity sensors monitor changes in the painting space's temperature and humidity in real time. These parameters influence the performance of optical components and the quality of holographic imaging. The system can fine-tune the optical system based on these environmental parameters to ensure stable imaging.

[0059] Environmental monitoring sensors collect environmental parameter data at set intervals and transmit it to the data processing module. The data processing module performs preliminary filtering and calibration on these parameters to remove potential noise and errors, ensuring their accuracy and reliability. This processed environmental parameter data serves as an important reference for subsequent data alignment and holographic image generation, compensating for the impact of environmental factors on the painting system's performance.

[0060] S2: Perform time-frequency alignment and noise filtering on the holographic light field data, user interaction signals, and environmental parameters to generate holographic image data. The holographic image data is used to indicate the geometric deformation gradient of the multi-view projection and the user interaction intention.

[0061] Specifically, after acquiring holographic light field data, user interaction signals, and environmental parameters, the system first performs time-frequency alignment on these data. Because different sensors may have different sampling frequencies and timestamps, the goal of time-frequency alignment is to unify all data into the same time-frequency coordinate system to ensure data synchronization and consistency. This step is achieved through time series analysis and frequency domain transformation algorithms, interpolating and resampling the data to eliminate time deviations and frequency mismatches.

[0062] After completing time-frequency alignment, the system performs noise filtering on the aligned data. Noise can originate from electronic noise within the sensor itself, environmental interference, and errors during data transmission. Noise filtering can utilize advanced filtering algorithms, such as wavelet transform filtering and adaptive filtering, to specifically remove or suppress noise components based on the statistical characteristics of the data and the spectral distribution of the noise, while preserving as much useful information as possible. The data after time-frequency alignment and noise filtering is further processed to generate holographic image data. The holographic image data not only contains the light field information of the user's drawing action but also incorporates the user's interaction intent and the influence of environmental parameters. This data is used to indicate the geometric deformation gradients for subsequent multi-view projections and more accurately interpret the user's interaction intent. This process involves complex image reconstruction algorithms, which convert the processed light field data into a holographic image format that can be recognized and presented by the display device.

[0063] S3: Calculate the multi-view projection deformation gradient of the holographic image data based on the neural radiance field model and the deformation field function to generate phase compensation parameters.

[0064] Specifically, the system uses the neural radiance field (NeRF) model and the deformation field function to calculate the multi-view projection deformation gradient of the holographic image data. The neural radiance field model is a deep learning-based three-dimensional scene representation method that can reconstruct the geometric and appearance information of a three-dimensional scene from two-dimensional images. The deformation field function is used to describe the geometric deformation caused by the change of the viewing angle or the curvature of the medium surface under different viewing angles. Specifically, the system inputs the generated holographic image data into the trained neural radiance field model. The model calculates the geometric deformation gradient that may occur in the multi-view projection case based on the three-dimensional information contained in the image data and in combination with the deformation field function. This gradient reflects the position offset trend of the virtual stroke relative to the real space under different viewing angles.

[0065] Based on the calculated deformation gradient, the system further generates phase compensation parameters. The phase compensation parameters are used to adjust the phase of the light wave in the subsequent holographic projection process to compensate for the change in the optical path difference caused by the deformation, so as to ensure the precise alignment and positioning of the virtual stroke under multiple viewing angles.

[0066] S4: Perform holographic layer energy optimization processing on the phase compensation parameters based on the orthogonal polarization encoding algorithm and the dynamic coherence control algorithm to generate an optimized holographic layer distribution.

[0067] After obtaining the phase compensation parameters, the system uses the orthogonal polarization encoding algorithm and the dynamic coherence control algorithm to perform holographic layer energy optimization processing on them. The orthogonal polarization encoding algorithm divides the light beam into orthogonal components with different polarization directions, and encodes and modulates these components separately to improve the utilization efficiency of light energy and reduce energy loss. The dynamic coherence control algorithm adjusts the coherence characteristics of light in real time according to the changes of environmental parameters and user interaction signals, and optimizes the distribution of light energy and the interference effect in the holographic layer.

[0068] Specifically, the system combines the phase compensation parameters with the orthogonal polarization encoding algorithm to control and encode the polarization state of the light beam in the holographic projection, so that the light energy can be transmitted and distributed more effectively under different viewing angles and medium conditions. At the same time, the dynamic coherence control algorithm adjusts parameters such as the coherence length and coherence bandwidth of light in real time according to the current environmental light intensity, color temperature, and user interaction signals such as the painting speed and pressure, to adapt to the complex optical environment and user operation requirements, and ensure that the light energy distribution of each pixel point in the holographic layer reaches the best state, thereby generating an optimized holographic layer distribution.

[0069] S5: Based on the predictive rendering algorithm, photon - electron clock alignment is performed on the optimized holographic layer distribution, combined with generating a low - latency holographic projection output for real - time 3D painting interactive display.

[0070] Specifically, the predictive rendering algorithm predicts the user's next painting action and interaction intention in advance based on the historical data and current trends of the user interaction signal, thereby reducing the waiting time during the rendering process and improving the response speed. The photon - electron clock alignment is to ensure the high - degree synchronization of the photon emission moment and the electron signal processing rhythm by precisely controlling them, avoiding problems such as display latency or screen tearing caused by clock asynchronization.

[0071] The optimized holographic layer distribution after photon - electron clock alignment is further processed to generate a low - latency holographic projection output. This holographic projection output is displayed in real - time through a high - performance holographic display device, providing an immersive 3D painting interactive experience for users. During the display process, the system continuously monitors the changes in user interaction signals and environmental parameters, and adjusts the parameters of the holographic projection in real - time to ensure the smoothness and accuracy of the painting process, meeting the high - precision painting requirements of users in complex scenarios.

[0072] In summary, an intelligent painting method based on optical processing provided by the present invention obtains comprehensive data information through multi - dimensional sensors. After time - frequency alignment and noise filtering processing, the accuracy and reliability of the data are ensured. The neural radiance field model and deformation field function are used to deeply process the holographic image data to generate phase compensation parameters, further optimizing the energy distribution of the holographic layer. Finally, through the predictive rendering algorithm and photon - electron clock alignment technology, a low - latency holographic projection output can be achieved, providing users with a high - quality real - time 3D painting interactive display experience. This process has significant beneficial effects in improving holographic projection accuracy, enhancing user interactivity, reducing system latency, etc., promoting the application and development of holographic technology in the field of real - time interactive display.

[0073] In one of the embodiments, step S2 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0074] S21: Perform spectral filtering on the ambient light of environmental parameters based on an optical filter to generate an effective projection wavelength range.

[0075] Specifically, the optical filter adopts the multi-layer dielectric film interference filtering technology, which consists of multi-layers of dielectric thin films with different refractive indices. It realizes the passage of light within a specific wavelength range through the principle of optical interference, while the light of the remaining wavelengths is reflected or absorbed. According to the requirements of the painting scene and the characteristics of the holographic projection device, the effective projection wavelength range is preset in advance, usually covering the red, green, and blue (RGB) primary color bands in the visible light, to ensure the color richness and accuracy of the holographic image. During the actual filtering process, the optical filter is installed at the optical path entrance of the holographic projection system to filter the ambient light in real time, remove the light of interfering wavelengths, and ensure that the optical signal entering the system meets the requirements of holographic projection, laying a foundation for the subsequent generation of high-quality holographic images.

[0076] Apply the determined effective projection wavelength range to the light source control and holographic image generation process of the holographic projection system. At the light source end, adjust the emission spectrum of the light source according to the effective projection wavelength range, so that its main energy is concentrated within this wavelength range; at the holographic image generation end, perform corresponding color correction and modulation on the holographic image data according to this wavelength range to ensure that the generated holographic image can obtain the best display effect within the selected wavelength range. At the same time, the system continuously monitors the change of the ambient light, and fine-tunes and updates the effective projection wavelength range according to the dynamic change of the ambient light spectrum to adapt to different painting environment conditions and ensure the stability and high quality of the holographic projection.

[0077] S22: Based on the stylus pose signal and the holographic light field data of the user interaction signal, perform time synchronization and alignment processing through the time reference signal generated by the optical crystal oscillator to generate synchronized data.

[0078] Specifically, a high-precision six-axis inertial measurement unit (IMU) is integrated inside the stylus, including a three-axis accelerometer and a three-axis gyroscope, which are used to measure the linear acceleration and angular velocity of the stylus in three-dimensional space in real time. At the same time, a pressure sensor is equipped to obtain the pressure information when the pen tip touches the painting surface, and an electromagnetic induction coil is used to determine the position and direction of the stylus in the painting space. These sensors together constitute the acquisition system of the stylus pose signal, which can comprehensively capture the dynamic characteristics of the user's painting actions.

[0079] The stylus's pose sensor collects pose data in real time at a set sampling frequency (e.g., 1000Hz) and transmits this data to the drawing system's main control computer via a wireless communication module (e.g., Bluetooth or Wi-Fi). The collected pose signals undergo preliminary preprocessing in the main control computer, including data filtering, denoising, and compensation. A Kalman filter algorithm is used to filter the inertial measurement unit data, removing sensor noise and interference and improving pose data accuracy. Simultaneously, the pen tip position is compensated based on pressure sensor data, taking into account the impact of the elastic deformation of the pen tip under pressure on the drawing position, ensuring that the pose signal accurately reflects the actual drawing movement of the stylus.

[0080] Specifically, the system utilizes an optical crystal oscillator to generate a highly stable time reference signal, which serves as the time reference for the entire painting system. Optical crystal oscillators offer high precision, low drift, and excellent temperature stability, providing accurate clock pulses. During system initialization, the optical crystal oscillator is calibrated. By comparing the oscillator's frequency and phase with an external high-precision time source (such as a GPS clock or atomic clock), the oscillator's frequency and phase are adjusted to ensure that its output time reference signal remains synchronized with the standard time.

[0081] After receiving the stylus pose signal and holographic light field data, the main control computer performs time synchronization and alignment based on their timestamp information and a time reference signal. First, the stylus pose signal and holographic light field data are arranged and interpolated in timestamp order, ensuring that they have the same sampling frequency and time interval in the temporal dimension. Then, the time starting points of the two are aligned based on the time reference signal. By adjusting the time offset of the data, the stylus pose signal and holographic light field data are strictly synchronized in time. This ensures that subsequent processing accurately matches the user's drawing movements with the corresponding light field change information, generating synchronized data with the correct time relationship.

[0082] S23: Process the synchronization data and the effective projection wavelength range based on the data dimensionality reduction algorithm, compress the four-dimensional holographic light field data into a 256-dimensional low-dimensional feature vector, and generate holographic image data.

[0083] Specifically, given the high-dimensional nature of four-dimensional holographic light field data, the principal component analysis (PCA) algorithm can be used as the primary method for data dimensionality reduction. The PCA algorithm is an unsupervised dimensionality reduction technique based on linear transformations. It projects high-dimensional data into a low-dimensional space by identifying the directions of the principal components in the data, while preserving the data's variance and key characteristic structures as much as possible. Its advantages include high computational efficiency, simple implementation, and significant effectiveness when processing high-dimensional data with linear correlations. It is suitable for situations where strong correlations exist between different dimensions in holographic light field data.

[0084] For a given four - dimensional holographic light field data set, the PCA algorithm first calculates its covariance matrix, which describes the correlation between data in different dimensions; then, it solves the eigenvalues and eigenvectors of the covariance matrix. The magnitude of the eigenvalues represents the data variance in the corresponding principal component directions, and the eigenvectors indicate the directions of the principal components; finally, it selects the first 256 eigenvectors in descending order of eigenvalues to form a dimensionality reduction transformation matrix, and projects the original four - dimensional data into the low - dimensional subspace spanned by these 256 eigenvectors to achieve dimensionality reduction of the data.

[0085] Based on the 256 - dimensional low - dimensional feature vectors after dimensionality reduction, combined with the user interaction information and the effective projection wavelength range in the synchronous data, the system reconstructs and generates holographic image data. Using the holographic imaging principle and algorithms, the information in the low - dimensional feature vectors is converted into the light intensity distribution, phase information, color information, etc. of the holographic image. During the reconstruction process, fully consider the influence of the user's painting actions on the positioning, shape, and color of the virtual brushstrokes in three - dimensional space, as well as the limitation of the effective projection wavelength range on the display effect of the holographic image. By adjusting the parameters of the holographic image data, it can accurately reflect the user's painting intention and the physical characteristics of the painting scene.

[0086] Preferably, the generated holographic image data can be further optimized, including operations such as image contrast enhancement, edge sharpening, noise suppression, and color correction. The multi - scale analysis method based on wavelet transform can be used to enhance the contrast and sharpen the edges of the holographic image, adjusting the detailed information of the image at different scales to improve the clarity and visual effect of the image; at the same time, an adaptive filtering algorithm is applied to suppress the noise in the holographic image, removing possible artifacts and interference; finally, according to the effective projection wavelength range and the environmental light spectral characteristics, the color of the holographic image is corrected to ensure that the color of the holographic projection output image is accurate, natural, highly consistent with the user's painting color intention, and provides a high - quality holographic painting display experience for the user.

[0087] In one of the embodiments, as Figure 2 shown, step S3 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0088] S31: Based on the multi - perspective images of the holographic image data, construct a three - dimensional geometric deformation field through a neural radiance field model to generate initial deformation gradient data.

[0089] Specifically, the system extracts a sequence of two-dimensional images from the holographic image data at different perspectives. These images cover the observation results of the user's painting scene from multiple angles. Each perspective image corresponds to a specific observation direction and position, containing rich spatial information and painting details. The system performs preliminary preprocessing operations on the extracted multi-perspective images, including grayscale conversion, normalization, and noise filtering of the images. Grayscale conversion converts a color image into a grayscale image, reducing the amount of data and computational complexity; normalization maps pixel values to the range of [0, 1] or [-1, 1], which is beneficial for the subsequent training and convergence of the neural network; noise filtering uses methods such as median filtering or Gaussian filtering to remove salt-and-pepper noise and Gaussian noise in the image, improving the image quality.

[0090] Preferably, the initial deformation gradient data is generated through the following steps:

[0091] S311: Perform convolutional neural network feature extraction on the multi-perspective images in the holographic image data to generate multi-scale feature maps.

[0092] Specifically, the system constructs a deep convolutional neural network (CNN) for feature extraction of multi-perspective images. The network architecture includes multiple convolutional layers, pooling layers, and activation function layers. The convolutional layers use convolutional kernels of different sizes (such as 3×3, 5×5) to extract local features of the image, such as edges, textures, and shapes, etc.; the pooling layers are used to reduce the resolution of the feature maps, reducing the computational amount and improving the robustness of the network; the activation function (such as ReLU) introduces non-linearity to enhance the expression ability of the network.

[0093] After completing the construction of the neural network, the preprocessed multi-perspective images are input into the CNN. Through successive convolutional, pooling, and activation operations, feature maps of different scales are generated. The shallow convolutional layers mainly extract low-level features of the image, such as edges and textures; the deep convolutional layers can capture more high-level semantic features, such as the shape and structure of objects. Through feature maps at different levels, the feature information of the image at multiple scales can be comprehensively described, providing a rich feature basis for the subsequent construction of the three-dimensional geometric deformation field.

[0094] S312: Perform light field integration calculation of the three-dimensional spatial coordinates and viewing directions on the multi-scale feature maps based on the neural radiance field model to generate initial light field distribution data.

[0095] Specifically, a deep learning-based Neural Radiance Field (NeRF) model consists of a multi-layer perceptron (MLP). The input is the point coordinates in three-dimensional space and the viewing angle information, and the output is the volume density and radiance color of the point. The hidden layer of the model uses neurons with periodic activation functions (such as the Sine activation function) to better capture the high-frequency details and complex geometric structures in the scene. At the same time, skip connections are introduced during the model training process to directly transfer the input information to subsequent layers, enhancing the model's learning ability for detailed features.

[0096] Specifically, the system combines the multi-scale feature maps extracted by the CNN with the neural radiance field model to perform the light field integration calculation of three-dimensional space coordinates and viewing directions. Specifically, for each point in space, according to the feature map information at different viewing angles, the NeRF model is used to calculate the volume density and radiance color of the point, and integration is performed along the light ray path to obtain the final initial light field distribution data. This data describes the propagation, scattering, and absorption of light in the three-dimensional space in the painting scene, as well as the color information observed from different viewing angles, providing a comprehensive light field basis for the subsequent deformation gradient calculation.

[0097] S313: Calculate the partial derivatives of the initial light field distribution data based on the deformation field function to generate initial deformation gradient data covering three-dimensional geometric deformation gradients.

[0098] Specifically, the system defines a deformation field function that describes the geometric deformation of virtual strokes in three-dimensional space due to factors such as changes in the user's viewing angle and changes in the curvature of the medium surface. The deformation field function can usually be represented as a vector field, whose input is the point coordinates in the original three-dimensional space and relevant deformation parameters (such as viewing angle, medium curvature, etc.), and the output is the new coordinate position of the point after deformation. The specific form of the function can be derived based on the principles of geometric optics and the theory of elasticity, considering the bending of the light ray propagation path, the refractive index distribution of the medium, and the stretching, compression, and rotation of virtual strokes caused by user interaction actions. Among them, assume the deformation field function is , where p = (x, y, z) represents the point coordinates in three-dimensional space, represents the deformation parameters, which can include rotation angle, scaling factor, translation vector, etc. The deformation field function can be defined as:

[0099] ;

[0100] Among them, is the deformation vector, indicating the displacement of point p under the deformation parameter . For example, consider a simple deformation field function, where the deformation consists of rotation and translation:

[0101] ;

[0102] Here, is the rotation matrix, t is the translation vector. For rotation in three-dimensional space, the rotation matrix can be expressed as:

[0103] ;

[0104] This deformation field function describes the point p at the rotation angle θ and the translation vector t under the action of the new position.

[0105] In practical applications, the deformation field function may be more complex and needs to consider the combination of various deformation factors, such as non-linear deformation, local deformation, etc. For example, in holographic projection, the deformation field function may need to consider the influence of factors such as the change of the user's viewing angle and the curvature of the medium surface on the position of the virtual stroke.

[0106] Based on the partial derivative of the deformation field function with respect to the initial light field distribution data, the initial deformation gradient data covering the three-dimensional geometric deformation gradient is calculated. Specifically, by calculating the partial derivative of the deformation field function with respect to the coordinate position at each point in three-dimensional space, the deformation rate and direction information in different directions at that point are obtained, that is, the deformation gradient. These deformation gradient data reflect the geometric deformation situation of the virtual stroke in three-dimensional space caused by various factors, providing the basic data support for subsequent non-linear correction and phase compensation parameter calculation. Among them, the deformation gradient is obtained through the following formula:

[0107] ;

[0108] Among them, represents the deformation gradient, represents the gradient operator, which is used to calculate the partial derivative, is the deformation field function, which is used to describe the geometric deformation, is the initial light field distribution data. This formula calculates the gradient of the initial light field distribution and the deformation field function to obtain the deformation gradient at each point in three-dimensional space, which is used for subsequent non-linear correction and phase compensation parameter calculation, so as to achieve a high-precision immersive 3D painting experience.

[0109] S32: Based on the deformation field function, non-linear correction is performed on the initial deformation gradient data to generate corrected gradient data.

[0110] Specifically, after generating the initial deformation gradient data, the system further performs non-linear correction on it to generate more accurate corrected gradient data. This process is based on the deformation field function and takes into account non-linear factors in the actual optical system, such as the inhomogeneity of the medium, the distortion of the projection device, etc.

[0111] Specifically, the system inputs the initial deformation gradient data into a non-linear correction model. According to the definition of the deformation field function, this model combines the actual environmental parameters and user interaction signals to adjust the gradient data. For example, when there is a curvature change on the surface of the medium, the correction model will compensate the gradient data according to the magnitude and direction of the curvature to eliminate the error caused by the deformation of the medium. After non-linear correction, the corrected gradient data better conforms to the actual optical projection conditions and can more accurately reflect the real displacement of the virtual stroke under multiple perspectives, providing more reliable data support for subsequent phase compensation.

[0112] S33: Calculate the phase modulation parameters for the corrected gradient data to generate phase compensation parameters.

[0113] Specifically, the system calculates the phase modulation parameters based on the corrected gradient data to generate phase compensation parameters. The purpose of this process is to compensate for the change in optical path difference caused by deformation by adjusting the phase of the light wave, thereby ensuring the precise alignment of the virtual stroke under multi-perspective projection.

[0114] In specific implementation, the system converts the corrected gradient data into phase modulation parameters. This step is achieved through a physical optics model, considering the propagation characteristics of light waves in the medium, interference effects, etc. For example, the system calculates the required phase delay for each point based on the gradient data to offset the optical path difference caused by deformation. These phase modulation parameters are further encoded as the phase information of the hologram to guide the holographic projection device to perform precise phase control on the light wave. The generated phase compensation parameters will be used for subsequent holographic layer energy optimization processing to ensure that the virtual stroke can be accurately presented at the expected position under different perspectives and medium conditions, providing a high-precision immersive painting experience.

[0115] The above intelligent painting method based on optical processing deeply processes the holographic image data through technologies such as neural radiance field models and convolutional neural networks, generates initial deformation gradient data, and performs non-linear correction and phase compensation parameter calculation on it. This series of steps can effectively improve the accuracy and quality of holographic projection, providing high-quality data support for subsequent holographic projection output. At the same time, this process has significant advantages in dealing with complex scenes and multi-perspective projections and can meet the requirements of high-demand application scenarios such as real-time 3D painting interaction display.

[0116] In one embodiment, such as Figure 3As shown in the figure, step S4 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0117] S41: Based on the orthogonal polarization algorithm, non-overlapping polarization direction references are assigned to each holographic layer of the phase compensation parameters for polarization assignment processing, generating orthogonal polarization references for each holographic layer.

[0118] Specifically, the orthogonal polarization algorithm makes each layer independent during projection by assigning non-overlapping polarization directions to different holographic layers, avoiding optical interference. Based on the polarization characteristics of light, the oscillation direction of polarized light is consistent with the transmission axis of the polarizer. This characteristic can be used to achieve independent control of different depth holographic layers.

[0119] The system divides the holographic space into multiple holographic layers according to the holographic image data and the depth information of the painting scene. Each holographic layer corresponds to a specific depth range and painting details. Determine the polarization direction assignment scheme for each holographic layer to ensure that the polarization directions of adjacent layers are orthogonal. Preferably, optical elements such as polarizers or liquid crystal polarization modulators can be used to modulate the polarization state of each holographic layer according to the predetermined polarization direction. By precisely controlling the angles and characteristics of these elements, ensure that the polarization directions of each holographic layer are accurately assigned according to the orthogonal principle.

[0120] By performing polarization assignment processing on each holographic layer of the phase compensation parameters, orthogonal polarization references for each holographic layer are generated. These orthogonal polarization references will be used as the reference standards for polarization state control in the subsequent holographic projection process, ensuring that optical interference between different holographic layers is effectively suppressed under multi-view projection, and improving the clarity and contrast of the holographic image.

[0121] S42: Based on the dynamic coherence control algorithm, the spectral width of the laser light source of the orthogonal polarization reference is dynamically adjusted to generate optimized laser parameters.

[0122] Specifically, the dynamic coherence control algorithm optimizes the coherent superposition effect between holographic layers by adjusting the coherence length and coherence phase of the holographic projection light source in real time. The coherence of the light source determines the interference degree between different depth holographic layers. Dynamic adjustment can reduce unnecessary interference and improve the image quality.

[0123] During the holographic projection process, the system uses a coherence detection device to monitor parameters such as the coherence length and coherence phase of the laser light source in real time, and feeds the monitoring results back to the control system. At the same time, according to the changes in the holographic layer distribution and the painting scene, the required coherence adjustment trend is predicted to provide a basis for dynamic adjustment. Based on the feedback information and prediction results, by controlling parameters such as the pump current, cavity length or temperature of the laser, the spectral width of the laser light source is dynamically adjusted, narrowing the spectral width to increase the coherence length, or appropriately broadening the spectral width to reduce coherence, so as to optimize the coherent superposition effect between holographic layers, improve the stability and image quality of holographic projection, and generate optimized laser parameters through the dynamic adjustment of the spectral width of the laser light source. These parameters include the adjusted laser wavelength, spectral width, coherence length and coherence phase, etc., which will be applied to the subsequent holographic projection process to ensure that the holographic image can maintain high definition, high contrast and stable display effects under different painting scenes and viewing angles.

[0124] S43: Optimize the energy matching based on the Jones vector operation for the optimized laser parameters to generate an optimized holographic layer distribution that suppresses the crosstalk of non-holographic layers.

[0125] Preferably, S43 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0126] S431: Perform polarization state matching processing on the optimized laser parameters based on the Jones matrix operation to generate polarization matching parameters.

[0127] Specifically, the system uses the Jones matrix operation to describe the propagation and change of light in different polarization elements and holographic layers. According to the optimized laser parameters and the orthogonal polarization reference of the holographic layer, a corresponding Jones matrix model is established to simulate the polarization state change of light in the holographic system.

[0128] By solving the Jones matrix equation, a parameter combination that makes the polarization state of the holographic layer best match the polarization state of the laser light source is found, that is, the polarization matching parameters. These parameters include the angle of the polarizer, the phase delay amount, and the polarization direction adjustment amount of the holographic layer, etc., which are used to guide the subsequent energy weight distribution processing to ensure the efficient utilization and precise distribution of light energy in the holographic layer.

[0129] S432: Perform energy weight distribution processing on the polarization matching parameters based on the energy distribution optimization algorithm to generate an optimized energy distribution.

[0130] Specifically, based on the polarization matching parameters and the geometric and optical properties of the holographic layers, the system uses an energy distribution optimization algorithm to calculate the energy weights that should be allocated to each holographic layer at different depths and positions. The algorithm comprehensively considers factors such as the thickness, refractive index, absorption coefficient of the holographic layer, and the lighting requirements of the painting scene, etc., to ensure that the energy distribution not only meets the brightness and contrast requirements of image display but also can effectively suppress the crosstalk of non-holographic layers. The system generates optimized energy distribution data according to the energy weight allocation results. These data detail the energy intensity distribution of each holographic layer in space, providing an energy-based guiding basis for the subsequent reconstruction of the phase and amplitude distributions of the holographic layers, enabling the holographic image to present the best visual effects and optical properties during projection.

[0131] S433: Reconstruct the phase and amplitude distributions of the holographic layers according to the optimized energy distribution to generate an optimized holographic layer distribution that suppresses the crosstalk of non-holographic layers.

[0132] Specifically, the system combines the optimized energy distribution data and the holographic image information of the painting scene, and uses a holographic reconstruction algorithm to reconstruct the phase and amplitude distributions of each holographic layer. According to the guidance of the energy distribution, adjust the phase values and amplitude values of each pixel point in the holographic layer, so that while meeting the energy allocation requirements, it can accurately reproduce the three-dimensional structure and optical properties of the painting scene. And integrate the reconstructed phase and amplitude distribution data of the holographic layers to generate the final optimized holographic layer distribution. This optimized distribution not only achieves reasonable energy allocation but also effectively suppresses the optical crosstalk between non-holographic layers through precise control of phase and amplitude, improving the clarity, contrast, and color accuracy of the holographic image, providing users with a more realistic and immersive painting experience.

[0133] The above intelligent painting method based on optical processing finely regulates the optical parameters in holographic projection through technical means such as the orthogonal polarization algorithm, dynamic coherence control algorithm, and Jones vector operation. First, the orthogonal polarization algorithm is used to assign non-overlapping polarization direction references to each holographic layer to avoid energy crosstalk. Then, the dynamic coherence control algorithm is used to optimize the spectral width of the laser light source to improve the contrast and resolution of the holographic projection. Finally, the Jones vector operation is used to perform energy matching optimization processing on the laser parameters to generate an optimized holographic layer distribution that can suppress the crosstalk of non-holographic layers. This series of steps effectively improves the quality and efficiency of the holographic projection, providing users with a clearer and more accurate 3D painting interactive display experience.

[0134] In one embodiment, step S5 of the intelligent painting method based on optical processing provided by the present invention specifically includes the following steps:

[0135] S51: Establish a unified clock for optical and electronic devices based on the optimized holographic layer distribution, and perform clock alignment processing on the unified clock to generate a unified time reference.

[0136] Specifically, in the intelligent painting system, optical devices (such as holographic projectors, optical sensors) and electronic devices (such as styluses, computers) have their own clock systems, and there are deviations and asynchronisms between these clocks. To ensure the coordinated operation of each device in the system and achieve low-latency holographic projection output, a unified clock needs to be established. In the specific implementation, a high-performance FPGA chip is used as the clock control core, combined with high-speed clock signal transmission lines and precise clock synchronization algorithms, to integrate and align the clock signals of optical and electronic devices. Through clock alignment processing, the clock deviations between different devices are compensated, generating a unified time reference, enabling all devices in the system to work under a unified time framework, and providing an accurate time reference for subsequent trajectory prediction processing and rendering processing.

[0137] S52: Perform trajectory prediction processing on the stylus acceleration and speed data of the user interaction signal based on the unified time reference to generate future phase distribution parameters.

[0138] Specifically, the stylus acceleration and speed data are collected in real time by the inertial measurement unit (IMU) in the stylus, and these data reflect the dynamic characteristics of the user's painting actions. The system uses prediction algorithms such as the Kalman filter algorithm and long short-term memory network (LSTM), combined with the unified time reference, to predict the movement trajectory of the stylus in the short term in the future. During the prediction process, factors such as the user's painting habits, current painting speed, and acceleration change trend are fully considered to generate future phase distribution parameters. These parameters predict the user's demand for the phase distribution of the holographic layer at future moments, providing forward-looking data support for subsequent rendering processing, enabling the system to prepare the rendering content in advance and reduce latency.

[0139] S53: Based on the future phase distribution parameters, perform 4K resolution rendering processing on the user focus area of the optimized holographic layer to generate a low-latency holographic projection output.

[0140] First, based on the future phase distribution parameters, determine the position and range of the user focus area in the optimized holographic layer. The user focus area refers to the part of the holographic layer that the user is currently focusing on and performing painting operations, usually corresponding to the spatial area near the stylus. Then, use a high-performance graphics rendering engine to perform high-precision rendering of the user focus area at 4K resolution. During the rendering process, utilize parallel computing technology and multi-threaded optimization algorithms to improve the rendering speed and ensure the generation of high-quality images in a short time. At the same time, combine the optimized holographic layer distribution and phase compensation parameters generated in the previous steps to perform final optical processing on the rendered image, ensuring that the holographic projection output image has high clarity, high contrast, and accurate color reproduction. The finally generated low-latency holographic projection output can respond to the user's painting operations in real time, providing a smooth and natural immersive creation experience, meeting the strict requirements for real-time 3D painting in high-precision scenarios such as medical visualization and industrial simulation.

[0141] In summary, an intelligent painting method based on optical processing provided by the present invention ensures the temporal synchronization between an optical device and an electronic device by establishing a unified clock for them and performing clock alignment processing. Based on the unified time reference, trajectory prediction processing is performed on the user interaction signal, which can generate future phase distribution parameters in advance to support the real-time update of the holographic projection. Finally, 4K resolution rendering processing is performed on the user focus area of the optimized holographic layer to generate a low-latency holographic projection output. This not only improves the quality and detail performance of the projected image but also greatly reduces the system latency, enabling the user to perform 3D painting interaction operations more naturally and smoothly, enhancing the user experience and the overall performance of the system.

[0142] Preferably, as Figure 4 shown, the present invention provides an intelligent painting system 600 based on optical processing, and this intelligent painting system is configured with the following modules:

[0143] A data acquisition module 610, which is used to acquire holographic light field data, user interaction signals, and environmental parameters based on a multi-dimensional sensor;

[0144] A data preprocessing module 620, which is used to perform time-frequency alignment and noise filtering processing on the holographic light field data, user interaction signals, and environmental parameters to generate holographic image data, and the holographic image data is used to indicate the geometric deformation gradient of multi-view projection and the user interaction intention;

[0145] A holographic gradient calculation module 630, which is used to perform multi-view projection deformation gradient calculation on the holographic image data based on a neural radiance field model and a deformation field function to generate phase compensation parameters;

[0146] The holographic distribution optimization module 640 is used to perform holographic layer energy optimization processing on the phase compensation parameters based on the orthogonal polarization coding algorithm and the dynamic coherence control algorithm, and generate an optimized holographic layer distribution;

[0147] The holographic rendering output module 650 is used to perform photon-electron clock alignment on the optimized holographic layer distribution based on the predictive rendering algorithm, and combine to generate a low-latency holographic projection output, which is used for the interactive display of real-time 3D painting.

[0148] In summary, an intelligent painting system based on optical processing provided by the present invention obtains comprehensive data information through a multi-dimensional sensor, and through time-frequency alignment and noise filtering processing, ensures the accuracy and reliability of the data. The neural radiance field model and the deformation field function are used to deeply process the holographic image data, generate phase compensation parameters, and further optimize the energy distribution of the holographic layer. Finally, through the predictive rendering algorithm and the photon-electron clock alignment technology, a low-latency holographic projection output can be realized, providing users with a high-quality real-time 3D painting interactive display experience. This process has significant beneficial effects in improving the accuracy of holographic projection, enhancing user interactivity, reducing system latency, etc., and promotes the application and development of holographic technology in the field of real-time interactive display.

[0149] Preferably, the data preprocessing module 620 is configured with the following units:

[0150] The spectral filtering unit 621 is used to perform spectral filtering on the ambient light of the environmental parameters based on an optical filter, and generate an effective projection wavelength range;

[0151] The synchronization processing unit 622 is used to perform time synchronization alignment processing on the touch pen pose signal of the user interaction signal and the holographic light field data through the time reference signal generated by an optical crystal oscillator, and generate synchronized data;

[0152] The holographic image generation unit 623 is used to process the synchronized data and the effective projection wavelength range based on the data dimensionality reduction algorithm, compress the four-dimensional holographic light field data into a 256-dimensional low-dimensional feature vector, and generate holographic image data.

[0153] Preferably, the holographic gradient calculation module 630 is configured with the following units:

[0154] The gradient generation unit 631 is used to construct a three-dimensional geometric deformation field through a neural radiance field model based on the multi-view images of the holographic image data, and generate initial deformation gradient data;

[0155] Preferably, the gradient generation unit 631 is configured with the following sub-units:

[0156] The feature map generation subunit 6311 is configured to perform convolutional neural network feature extraction on the multi-view images in the holographic image data to generate multi-scale feature maps;

[0157] The light field data generation subunit 6312 is configured to perform light field integration calculation of three-dimensional spatial coordinates and viewing directions on the multi-scale feature maps based on the neural radiance field model to generate initial light field distribution data;

[0158] The deformation gradient calculation subunit 6313 is configured to calculate the partial derivatives of the initial light field distribution data based on the deformation field function to generate initial deformation gradient data covering three-dimensional geometric deformation gradients.

[0159] The gradient correction unit 632 is configured to perform non-linear correction on the initial deformation gradient data based on the deformation field function to generate corrected gradient data;

[0160] The phase parameter calculation unit 633 is configured to calculate phase modulation parameters for the corrected gradient data to generate phase compensation parameters.

[0161] Preferably, the holographic distribution optimization module 640 is configured with the following units:

[0162] The polarization allocation unit 641 is configured to perform polarization allocation processing on each holographic layer of the phase compensation parameters based on the orthogonal polarization algorithm to allocate non-overlapping polarization direction references, and generate orthogonal polarization references for each holographic layer;

[0163] The parameter optimization unit 642 is configured to dynamically adjust the spectral width of the laser light source of the orthogonal polarization reference based on the dynamic coherence control algorithm to generate optimized laser parameters;

[0164] The distribution optimization unit 643 is configured to perform energy matching optimization processing on the optimized laser parameters based on Jones vector operations to generate an optimized holographic layer distribution that suppresses crosstalk in non-holographic layers.

[0165] Preferably, the distribution optimization unit 643 is configured with the following subunits:

[0166] The polarization matching subunit 6431 is configured to perform polarization state matching processing on the optimized laser parameters based on Jones matrix operations to generate polarization matching parameters;

[0167] The energy allocation subunit 6432 is configured to perform energy weight allocation processing on the polarization matching parameters based on the energy distribution optimization algorithm to generate an optimized energy distribution;

[0168] The holographic layer reconstruction subunit 6433 is configured to reconstruct the phase and amplitude distributions of the holographic layer according to the optimized energy distribution to generate an optimized holographic layer distribution that suppresses crosstalk in non-holographic layers.

[0169] Preferably, the holographic rendering output module 650 is configured with the following units:

[0170] A time reference establishment unit 651, configured to establish a unified clock for the optical and electronic devices based on the optimized holographic layer distribution, perform clock alignment processing on the unified clock, and generate a unified time reference;

[0171] A trajectory prediction unit 652, configured to perform trajectory prediction processing on the stylus acceleration and speed data of the user interaction signal based on the unified time reference, and generate future phase distribution parameters;

[0172] A holographic projection output unit 653, configured to perform 4K resolution rendering processing on the user focus area of the optimized holographic layer based on the future phase distribution parameters, and generate a low-latency holographic projection output.

[0173] In one embodiment, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned intelligent painting method based on optical processing.

[0174] In one embodiment, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the above-mentioned intelligent painting method based on optical processing.

[0175] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0176] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial descriptions of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0177] As described above, this is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various changes or substitutions thereof, and these should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An intelligent painting method based on optical processing, characterized in that: The following steps are involved: S1: Acquire holographic light field data, user interaction signals and environmental parameters based on multi-dimensional sensors; S2: performing time-frequency alignment and noise filtering on the holographic light field data, the user interaction signal, and the environmental parameters to generate holographic image data, where the holographic image data is used to indicate the geometric deformation gradient of the multi-view projection and the user interaction intention; S3: performing multi-view projection deformation gradient calculation on the holographic image data based on the neural radiation field model and the deformation field function to generate phase compensation parameters; S4: performing holographic layer energy optimization processing on the phase compensation parameters based on an orthogonal polarization encoding algorithm and a dynamic coherence control algorithm to generate an optimized holographic layer distribution; S5: Performing photon-electron clock alignment on the optimized holographic layer distribution based on a predictive rendering algorithm, and combining this to generate a low-latency holographic projection output, wherein the holographic projection output is used for interactive display of real-time 3D painting.

2. The intelligent painting method according to claim 1, characterized in that: The S2 includes: S21: spectrally filtering the ambient light of the environmental parameters based on the optical filter to generate an effective projection wavelength range; S22: Based on the stylus pen posture signal of the user interaction signal and the holographic light field data, perform time synchronization alignment processing using a time reference signal generated by an optical crystal oscillator to generate synchronization data; S23: Processing the synchronization data and the effective projection wavelength range based on a data dimensionality reduction algorithm, compressing the four-dimensional holographic light field data into a 256-dimensional low-dimensional feature vector, and generating holographic image data.

3. The intelligent painting method according to claim 1, characterized in that: The S3 includes: S31: Based on the multi-view images of the holographic image data, construct a three-dimensional geometric deformation field through a neural radiation field model to generate initial deformation gradient data; S32: performing nonlinear correction on the initial deformation gradient data based on the deformation field function to generate correction gradient data; S33: Calculate phase modulation parameters on the correction gradient data to generate phase compensation parameters.

4. The intelligent painting method according to claim 3, characterized in that: The S31 includes: S311: performing convolutional neural network feature extraction on the multi-view images in the holographic image data to generate a multi-scale feature map; S312: Performing light field integral calculation of the three-dimensional spatial coordinates and the viewing direction on the multi-scale feature map based on the neural radiation field model to generate initial light field distribution data; S313: Calculating partial derivatives of the initial light field distribution data based on the deformation field function to generate initial deformation gradient data covering three-dimensional geometric deformation gradients.

5. The intelligent painting method according to claim 1, characterized in that: The S4 includes: S41: performing polarization allocation processing on each holographic layer of the phase compensation parameter to allocate non-overlapping polarization direction references based on an orthogonal polarization algorithm to generate orthogonal polarization references for each holographic layer; S42: Dynamically adjust the spectrum width of the laser light source of the orthogonal polarization reference based on a dynamic coherence control algorithm to generate optimized laser parameters; S43: Optimizing the energy matching of the optimized laser parameters based on Jones vector calculation to generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

6. The intelligent painting method according to claim 5, characterized in that: The S43 includes: S431: performing polarization state matching processing on the optimized laser parameters based on Jones matrix operation to generate polarization matching parameters; S432: Performing energy weight distribution processing on the polarization matching parameters based on an energy distribution optimization algorithm to generate an optimized energy distribution; S433: Reconstructing the phase and amplitude distribution of the holographic layer according to the optimized energy distribution to generate an optimized holographic layer distribution that suppresses crosstalk from non-holographic layers.

7. The intelligent painting method according to any one of claims 1 to 6, characterized in that: The S5 includes: S51: establishing a unified clock for optical and electronic devices based on the optimized holographic layer distribution, and performing clock alignment processing on the unified clock to generate a unified time reference; S52: performing trajectory prediction processing on the stylus acceleration and velocity data of the user interaction signal based on the unified time reference to generate future phase distribution parameters; S53: Based on the future phase distribution parameters, perform 4K resolution rendering processing on the user focus area of the optimized holographic layer to generate the low-latency holographic projection output.

8. An intelligent painting system based on optical processing, characterized in that: The system comprises: A data acquisition module is used to acquire holographic light field data, user interaction signals and environmental parameters based on a multi-dimensional sensor; a data preprocessing module, configured to perform time-frequency alignment and noise filtering on the holographic light field data, the user interaction signal, and the environmental parameters to generate holographic image data, wherein the holographic image data is used to indicate the geometric deformation gradient of the multi-view projection and the user interaction intention; A holographic gradient calculation module, configured to perform multi-view projection deformation gradient calculation on the holographic image data based on a neural radiation field model and a deformation field function, and generate phase compensation parameters; A holographic distribution optimization module is used to perform holographic layer energy optimization processing on the phase compensation parameters based on an orthogonal polarization encoding algorithm and a dynamic coherence control algorithm to generate an optimized holographic layer distribution; A holographic rendering output module is used to perform photon-electron clock alignment on the optimized holographic layer distribution based on a predictive rendering algorithm, and to generate a low-latency holographic projection output, wherein the holographic projection output is used for interactive display of real-time 3D paintings.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images