Intelligent drawing method and system based on optical processing

Abstract

The invention relates to an intelligent drawing method and system based on optical processing. The intelligent drawing method comprises the steps that holographic light field data, user interaction signals and environment parameters are collected in real time through a multi-dimensional sensor; performing time-frequency alignment and noise filtering on the multi-source heterogeneous data to generate holographic image data representing a multi-view projection deformation gradient and a user intention; based on the neural radiation field model and the dynamic deformation field function, adaptive compensation is carried out on projection geometric errors, phase correction parameters are generated, holographic layer energy distribution is optimized, and inter-layer crosstalk interference is inhibited; and a photon-electronic clock synchronization mechanism and a prediction rendering algorithm are combined to realize virtual-real space ultra-low delay interactive projection. According to the method, through deep fusion of optical processing and an intelligent algorithm, the problems of multi-view deformation accumulation, holographic layering crosstalk and too high interaction delay are solved, and the positioning precision, the projection definition and the real-time response performance of 3D painting in a complex scene are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of optical processing and 3D interaction technology, and in particular to an intelligent painting method and system based on optical processing. Background Art

[0002] With the growing demand for high-precision 3D interaction in fields such as virtual reality, digital art creation, and industrial design, intelligent painting technology based on optical processing has gradually become the core means to achieve the fusion of virtual and real creation. This type of technology transforms the user's physical painting movements into dynamic virtual brushstrokes in three-dimensional space by integrating optical sensing, holographic projection, and real-time computing models, and uses holographic imaging technology to present stereoscopic visual effects. In practical applications, such systems need to solve core challenges such as the geometric consistency of multi-view projection, the optical adaptability of complex media, and the real-time nature of human-computer interaction. The technical difficulty lies in how to achieve a high-precision, low-latency immersive creation experience through the coordinated optimization of optical processing and intelligent algorithms.

[0003] However, the existing intelligent painting systems based on optical processing still have significant defects in complex scenes. First, due to the insufficient deformation compensation capability of multi-view projection, when the user switches the viewing angle or the curvature of the medium surface changes, millimeter-level offsets are likely to occur between the virtual brushstrokes and the real space, resulting in a significant decrease in the positioning accuracy of the painting. Secondly, in the holographic layered rendering scene, the projection light beams of different depth of field layers produce cross-interference due to the diffraction effect, resulting in ghosting of auxiliary lines or model contours, which seriously reduces the clarity of the drawing of complex structures. In addition, the existing system has a high response delay to user interaction signals, which makes it difficult to match the real-time requirements of human natural painting, especially when quickly outlining or correcting details. It is easy to produce a sense of operational discontinuity. These problems seriously limit the application value of such technologies in high-precision scenarios such as medical visualization and industrial simulation. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide an intelligent painting method and system based on optical processing that can dynamically compensate for multi-viewing deformation, effectively suppress inter-layer crosstalk and achieve ultra-low latency rendering.

[0005] The purpose of the present invention is achieved by the following scheme: In a first aspect, the present invention provides an intelligent painting method based on optical processing, comprising the following steps: S1: Acquire holographic light field data, user interaction signals and environmental parameters based on multi-dimensional sensors; S2: Perform time-frequency alignment and noise filtering on the holographic light field data, user interaction signals and environmental parameters to generate holographic image data. The holographic image data is used to indicate the geometric deformation gradient of multi-view projection and the user interaction intention; S3: Based on the neural radiation field model and deformation field function, multi-view projection deformation gradient calculation is performed on the holographic image data to generate phase compensation parameters; S4: Based on the orthogonal polarization coding algorithm and the dynamic coherence control algorithm, the phase compensation parameters are optimized for the holographic layer energy to generate the optimized holographic layer distribution; S5: Photon-electron clock alignment of optimized holographic layer distribution based on predictive rendering algorithm, combined with generation of low-latency holographic projection output, which is used for interactive display of real-time 3D paintings.

[0006] In one embodiment, S2 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S21: spectrally filter the ambient light of the environmental parameters based on the optical filter to generate an effective projection wavelength range; S22: Based on the stylus pen posture signal of the user interaction signal and the holographic light field data, a time reference signal generated by an optical crystal oscillator is used to perform time synchronization alignment processing to generate synchronization data; S23: Based on the data dimension reduction algorithm, the synchronization data and the effective projection wavelength range are processed, and the four-dimensional holographic light field data is compressed into a 256-dimensional low-dimensional feature vector to generate holographic image data.

[0007] In one embodiment, S3 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S31: Based on the multi-view images of the holographic image data, a three-dimensional geometric deformation field is constructed through a neural radiation field model to generate initial deformation gradient data; S32: performing nonlinear correction on the initial deformation gradient data based on the deformation field function to generate correction gradient data; S33: Calculate the phase modulation parameters of the correction gradient data to generate phase compensation parameters.

[0008] In one embodiment, S31 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S311: performing convolutional neural network feature extraction on multi-view images in the holographic image data to generate a multi-scale feature map; S312: performing light field integral calculation of the three-dimensional spatial coordinates and the viewing direction of the multi-scale feature map based on the neural radiation field model to generate initial light field distribution data; S313: Calculate the partial derivative of the initial light field distribution data based on the deformation field function to generate initial deformation gradient data covering the three-dimensional geometric deformation gradient.

[0009] In one embodiment, S4 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S41: performing polarization allocation processing on non-overlapping polarization direction references of each holographic layer of phase compensation parameters based on an orthogonal polarization algorithm to generate orthogonal polarization references of each holographic layer; S42: Dynamically adjust the spectrum width of the orthogonal polarization-based laser source based on the dynamic coherence control algorithm to generate optimized laser parameters; S43: Optimizing the energy matching of the optimized laser parameters based on Jones vector calculation to generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

[0010] In one embodiment, S43 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S431: performing polarization state matching processing on the optimized laser parameters based on Jones matrix operation to generate polarization matching parameters; S432: performing energy weight distribution processing on polarization matching parameters based on an energy distribution optimization algorithm to generate an optimized energy distribution; S433: Reconstructing the phase and amplitude distribution of the holographic layer according to the optimized energy distribution to generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

[0011] In one embodiment, S5 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S51: establishing a unified clock for optical and electronic devices based on the optimized holographic layer distribution, and performing clock alignment processing on the unified clock to generate a unified time reference; S52: performing trajectory prediction processing on the acceleration and velocity data of the stylus of the user interaction signal based on a unified time reference to generate future phase distribution parameters; S53: Based on the future phase distribution parameters, 4K resolution rendering processing is performed on the user focus area of ​​the optimized holographic layer to generate a low-latency holographic projection output.

[0012] In a second aspect, the present invention provides an intelligent painting system based on optical processing, comprising: A data acquisition module, used to acquire holographic light field data, user interaction signals and environmental parameters based on a multi-dimensional sensor; A data preprocessing module is used to perform time-frequency alignment and noise filtering on the holographic light field data, user interaction signals and environmental parameters to generate holographic image data. The holographic image data is used to indicate the geometric deformation gradient of multi-view projection and the user interaction intention; A holographic gradient calculation module is used to perform multi-view projection deformation gradient calculation on holographic image data based on a neural radiation field model and a deformation field function to generate phase compensation parameters; Holographic distribution optimization module, used to perform holographic layer energy optimization processing on phase compensation parameters based on orthogonal polarization encoding algorithm and dynamic coherence control algorithm to generate optimized holographic layer distribution; The holographic rendering output module is used to perform photon-electron clock alignment on the optimized holographic layer distribution based on the predictive rendering algorithm, combined with the generation of low-latency holographic projection output, which is used for the interactive display of real-time 3D paintings.

[0013] In a third aspect, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, any one of the above-mentioned intelligent painting methods based on optical processing is implemented.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements any of the above-mentioned intelligent painting methods based on optical processing when the computer program is executed by a processor.

[0015] In summary, the intelligent painting method based on optical processing provided by the present invention obtains comprehensive data information through multi-dimensional sensors, and ensures the accuracy and reliability of the data through time-frequency alignment and noise filtering. The holographic image data is deeply processed using the neural radiation field model and the deformation field function to generate phase compensation parameters, which further optimizes the energy distribution of the holographic layer. Finally, through the predictive rendering algorithm and photon-electron clock alignment technology, low-latency holographic projection output can be achieved, providing users with a high-quality real-time 3D painting interactive display experience. This process has significant beneficial effects in improving the accuracy of holographic projection, enhancing user interactivity, and reducing system latency, and has promoted the application and development of holographic technology in the field of real-time interactive display.

[0016] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a process of an intelligent painting method based on optical processing provided in an embodiment of the present application; Figure 2 A schematic diagram of a process for generating phase compensation parameters provided in an embodiment of the present application; Figure 3 A schematic diagram of a process for generating an optimized holographic layer distribution provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an intelligent painting system based on optical processing provided in another embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the invention more thorough and comprehensive.

[0019] Unless otherwise defined, all technical terms and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0020] In one embodiment, Figure 1 As shown, a smart painting method based on optical processing is provided. This embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps: S1: Acquire holographic light field data, user interaction signals and environmental parameters based on multi-dimensional sensors.

[0021] Specifically, for holographic light field data, the system uses high-precision CMOS image sensors, which have the characteristics of high resolution, high sensitivity and low noise, and can accurately capture light field information. The sensor array is installed at the data acquisition end of the painting system in a specific geometric arrangement, capturing the light distribution in the scene from multiple perspectives at the same time, ensuring that the acquired holographic light field data has sufficient spatial and angular resolution to meet the accuracy requirements of subsequent 3D reconstruction and deformation compensation.

[0022] When the user performs physical painting actions in the painting space, light propagates from different directions and is captured by the sensor array. Each sensor records information such as the intensity, phase, and direction of the light to form raw light field data. This data contains the three-dimensional structure of the painting object and the optical properties of the surrounding environment, providing basic data support for subsequent holographic image generation and deformation gradient calculation.

[0023] For user interaction signals, the system uses a professional hand-drawn digital tablet or a pressure-sensitive stylus that can sense the pressure, speed, angle and other parameters of the user's painting movements in real time and convert them into electrical signals. At the same time, the system can also integrate gesture recognition sensors to further enrich the dimensions of interactive signals by capturing the user's hand movements and posture changes, and achieve more natural and flexible painting operation control.

[0024] The interactive signals generated by the user during the painting process, such as the pressure change of the brush stroke, the contact position and movement trajectory of the brush tip and the painting surface, are collected in real time by the interactive device. The collected analog signals are accurately digitized by the analog-to-digital converter (ADC) and converted into digital signals for subsequent computer processing and analysis. These digital signals contain the direct expression of the user's painting intention and are the key data source for the system to understand the user's creative intention.

[0025] For environmental parameters, a variety of environmental monitoring sensors are arranged around the painting system, including light sensors, temperature sensors, and humidity sensors. Light sensors are used to measure the intensity and spectral distribution of ambient light so that the system can perform appropriate brightness and color compensation for holographic projection according to actual ambient light conditions; temperature and humidity sensors monitor the temperature and humidity changes in the painting space in real time. These parameters have a certain impact on the performance of optical components and the quality of holographic imaging. The system can fine-tune the optical system according to environmental parameters to ensure stable imaging effects.

[0026] The environmental monitoring sensor collects environmental parameter data at set time intervals and transmits it to the data processing module. The data processing module performs preliminary filtering and calibration on these parameters to remove possible noise and errors and ensure the accuracy and reliability of environmental parameters. The processed environmental parameter data will serve as an important reference for subsequent data alignment and holographic image generation, and will be used to compensate for the impact of environmental factors on the performance of the painting system.

[0027] S2: Perform time-frequency alignment and noise filtering on the holographic light field data, user interaction signals and environmental parameters to generate holographic image data. The holographic image data is used to indicate the geometric deformation gradient of the multi-view projection and the user interaction intention.

[0028] Specifically, after acquiring the holographic light field data, user interaction signals and environmental parameters, the system first performs time-frequency alignment on these data. Since the sampling frequencies and timestamps of different sensors may differ, the purpose of time-frequency alignment is to unify all data into the same time-frequency coordinate system to ensure the synchronization and consistency of the data. This step is achieved through time series analysis and frequency domain transformation algorithms, which interpolate and resample the data to eliminate time deviation and frequency mismatch problems.

[0029] After completing the time-frequency alignment, the system performs noise filtering on the aligned data. Noise may come from the electronic noise of the sensor itself, environmental interference, and errors in the data transmission process. Noise filtering can use advanced filtering algorithms, such as wavelet transform filtering, adaptive filtering, etc., to remove or suppress noise components in a targeted manner according to the statistical characteristics of the data and the spectral distribution of the noise, while retaining the useful information in the data as much as possible. The data after time-frequency alignment and noise filtering are further processed to generate holographic image data. The holographic image data not only contains the light field information of the user's painting action, but also integrates the influence of the user's interaction intention and environmental parameters, which is used to indicate the geometric deformation gradient of subsequent multi-view projection and more accurately analyze the user's interaction intention. This process involves complex image reconstruction algorithms to convert the processed light field data into a holographic image format that can be recognized and presented by the display device.

[0030] S3: Based on the neural radiation field model and deformation field function, multi-view projection deformation gradient calculation is performed on the holographic image data to generate phase compensation parameters.

[0031] Specifically, the system uses the neural radiance field (NeRF) model and deformation field function to calculate the multi-view projection deformation gradient of the holographic image data. The neural radiance field model is a three-dimensional scene representation method based on deep learning, which can reconstruct the geometry and appearance information of the three-dimensional scene from the two-dimensional image. The deformation field function is used to describe the geometric deformation caused by changes in the observation angle or the curvature of the medium surface at different viewing angles. Specifically, the system inputs the generated holographic image data into the trained neural radiance field model. The model calculates the geometric deformation gradient that may appear in the case of multi-view projection based on the three-dimensional information contained in the image data and the deformation field function. This gradient reflects the positional offset trend of the virtual brush stroke relative to the real space at different viewing angles.

[0032] Based on the calculated deformation gradient, the system further generates phase compensation parameters, which are used to adjust the phase of the light wave in the subsequent holographic projection process to compensate for the change in optical path difference caused by deformation, thereby ensuring the precise alignment and positioning of the virtual pen strokes under multiple viewing angles.

[0033] S4: Based on the orthogonal polarization coding algorithm and the dynamic coherence control algorithm, the phase compensation parameters are optimized for the holographic layer energy to generate the optimized holographic layer distribution.

[0034] After obtaining the phase compensation parameters, the system uses the orthogonal polarization encoding algorithm and the dynamic coherence control algorithm to optimize the energy of the holographic layer. The orthogonal polarization encoding algorithm divides the light beam into orthogonal components with different polarization directions, and encodes and modulates these components separately to improve the efficiency of light energy utilization and reduce energy loss. The dynamic coherence control algorithm adjusts the coherence characteristics of light in real time according to changes in environmental parameters and user interaction signals, and optimizes the distribution and interference effect of light energy in the holographic layer.

[0035] Specifically, the system combines phase compensation parameters with orthogonal polarization encoding algorithms to regulate and encode the polarization state of the light beam in the holographic projection, so that light energy can be transmitted and distributed more effectively under different viewing angles and medium conditions. At the same time, the dynamic coherence control algorithm adjusts the coherence length, coherence bandwidth and other parameters of the light in real time according to the current ambient light intensity, color temperature, and interactive signals such as the user's painting speed and pressure to adapt to complex optical environments and user operation requirements, ensuring that the light energy distribution of each pixel in the holographic layer reaches the optimal state, thereby generating an optimized holographic layer distribution.

[0036] S5: Photon-electron clock alignment of optimized holographic layer distribution based on predictive rendering algorithm, combined with generation of low-latency holographic projection output, which is used for interactive display of real-time 3D paintings.

[0037] Specifically, the predictive rendering algorithm predicts the user's next drawing action and interaction intention in advance based on the historical data and current trends of the user's interaction signals, thereby reducing waiting time and improving response speed during the rendering process. Photon-electron clock alignment ensures that the two are highly synchronized in time by precisely controlling the emission time of photons and the processing rhythm of electronic signals, avoiding problems such as display delays or screen tearing caused by clock asynchrony.

[0038] The optimized holographic layer distribution after photon-electron clock alignment is further processed to generate a low-latency holographic projection output. The holographic projection output is displayed in real time through a high-performance holographic display device, providing users with an immersive 3D painting interactive experience. During the display process, the system continuously monitors changes in user interaction signals and environmental parameters, and adjusts the parameters of the holographic projection in real time to ensure the smoothness and accuracy of the painting process, meeting the user's high-precision painting needs in complex scenes.

[0039] In summary, the intelligent painting method based on optical processing provided by the present invention obtains comprehensive data information through multi-dimensional sensors, and ensures the accuracy and reliability of the data through time-frequency alignment and noise filtering. The holographic image data is deeply processed using the neural radiation field model and the deformation field function to generate phase compensation parameters, which further optimizes the energy distribution of the holographic layer. Finally, through the predictive rendering algorithm and photon-electron clock alignment technology, low-latency holographic projection output can be achieved, providing users with a high-quality real-time 3D painting interactive display experience. This process has significant beneficial effects in improving the accuracy of holographic projection, enhancing user interactivity, and reducing system latency, and has promoted the application and development of holographic technology in the field of real-time interactive display.

[0040] In one embodiment, S2 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S21: spectrally filter the ambient light of the environmental parameters based on the optical filter to generate an effective projection wavelength range.

[0041] Specifically, the optical filter uses multi-layer dielectric film interference filtering technology, which is composed of multiple layers of dielectric films with different refractive indices. It uses the principle of optical interference to allow light in a specific wavelength range to pass through, while light of other wavelengths is reflected or absorbed. According to the requirements of the painting scene and the characteristics of the holographic projection equipment, the effective projection wavelength range is pre-set, usually covering the three primary color bands of red, green, and blue (RGB) in visible light to ensure the color richness and accuracy of the holographic image. In the actual filtering process, the optical filter is installed at the entrance of the optical path of the holographic projection system to filter the ambient light in real time, remove the light of interfering wavelengths, and ensure that the light signal entering the system meets the requirements of holographic projection, laying the foundation for the subsequent generation of high-quality holographic images.

[0042] The determined effective projection wavelength range is applied to the light source control and holographic image generation process of the holographic projection system. At the light source end, the emission spectrum of the light source is adjusted according to the effective projection wavelength range so that its main energy is concentrated within this wavelength range; at the holographic image generation end, the holographic image data is color corrected and modulated accordingly according to the wavelength range to ensure that the generated holographic image can obtain the best display effect within the selected wavelength range. At the same time, the system continuously monitors the changes in ambient light, and fine-tunes and updates the effective projection wavelength range according to the dynamic changes in the ambient light spectrum to adapt to different painting environment conditions and ensure the stability and high quality of holographic projection.

[0043] S22: Based on the stylus pen posture signal of the user interaction signal and the holographic light field data, a time reference signal generated by an optical crystal oscillator is used to perform time synchronization alignment processing to generate synchronization data.

[0044] Specifically, a high-precision six-axis inertial measurement unit (IMU) is integrated inside the stylus, including a three-axis accelerometer and a three-axis gyroscope, which are used to measure the linear acceleration and angular velocity of the stylus in three-dimensional space in real time. At the same time, a pressure sensor is equipped to obtain pressure information when the pen tip contacts the drawing surface, and an electromagnetic induction coil is used to determine the position and direction of the stylus in the drawing space. These sensors together constitute a stylus posture signal acquisition system that can fully capture the dynamic characteristics of the user's drawing movements.

[0045] The stylus posture sensor collects posture data in real time according to the set sampling frequency (such as 1000Hz), and transmits the data to the main control computer of the painting system through a wireless communication module (such as Bluetooth or Wi-Fi). In the main control computer, the collected posture signals are preliminarily preprocessed, including data filtering, denoising and compensation. The Kalman filter algorithm is used to filter the inertial measurement unit data to remove the noise and interference of the sensor itself and improve the accuracy of the posture data; at the same time, the pen tip position is compensated and corrected according to the pressure sensor data, considering the influence of the elastic deformation of the pen tip under pressure on the painting position, to ensure that the posture signal can accurately reflect the actual painting action of the stylus.

[0046] Specifically, the system uses an optical crystal oscillator to generate a high-stability time reference signal, which serves as the time reference standard for the entire painting system. Optical crystal oscillators have the characteristics of high precision, low drift, and good temperature stability, and can provide accurate clock pulses. During the system initialization phase, the optical crystal oscillator is calibrated, and the frequency and phase of the oscillator are adjusted by comparing it with an external high-precision time source (such as a GPS clock or an atomic clock) to ensure that the time reference signal it outputs is synchronized with the standard time.

[0047] After the main control computer receives the stylus posture signal and holographic light field data, it performs time synchronization alignment processing according to the timestamp information and time reference signal of the two. First, the stylus posture signal and the holographic light field data are arranged and interpolated in the order of timestamps so that the two have the same sampling frequency and time interval in the time dimension; then, the time starting point of the two is aligned according to the time reference signal, and the time offset of the data is adjusted to make the stylus posture signal and the holographic light field data strictly synchronized in time, ensuring that in the subsequent processing process, the user's painting action can be accurately matched with the corresponding light field change information, and synchronized data with the correct time relationship is generated.

[0048] S23: Based on the data dimension reduction algorithm, the synchronization data and the effective projection wavelength range are processed, and the four-dimensional holographic light field data is compressed into a 256-dimensional low-dimensional feature vector to generate holographic image data.

[0049] Specifically, in view of the high-dimensional characteristics of four-dimensional holographic light field data, the principal component analysis (PCA) algorithm can be selected as the main method for data dimensionality reduction. The PCA algorithm is an unsupervised dimensionality reduction technology based on linear transformation. It can project high-dimensional data into low-dimensional space by finding the direction of the principal components in the data, while retaining the variance information and main characteristic structure of the data as much as possible. Its advantages are high computational efficiency, simple implementation, and significant effect when processing high-dimensional data with linear correlation. It is suitable for situations where there is a strong correlation between different dimensions in holographic light field data.

[0050] For a given four-dimensional holographic light field data set, the PCA algorithm first calculates its covariance matrix, which describes the correlation between the data in different dimensions; then, it solves the eigenvalues ​​and eigenvectors of the covariance matrix. The size of the eigenvalue represents the size of the data variance in the direction of the corresponding principal component, and the eigenvector indicates the direction of the principal component; finally, the first 256 eigenvectors are selected in descending order of the eigenvalue to form a dimensionality reduction transformation matrix, and the original four-dimensional data is projected into a low-dimensional subspace spanned by these 256 eigenvectors to achieve data dimensionality reduction processing.

[0051] The system reconstructs and generates holographic image data based on the 256-dimensional low-dimensional feature vector after dimensionality reduction, combined with the user interaction information and effective projection wavelength range in the synchronization data. Using the principles and algorithms of holographic imaging, the information in the low-dimensional feature vector is converted into the light intensity distribution, phase information, and color information of the holographic image. In the reconstruction process, the influence of the user's painting action on the positioning, shape, and color of the virtual brushstrokes in the three-dimensional space, as well as the limitation of the effective projection wavelength range on the display effect of the holographic image, are fully considered. By adjusting the parameters of the holographic image data, it can accurately reflect the user's painting intention and the physical characteristics of the painting scene.

[0052] Preferably, the generated holographic image data can be further optimized, including image contrast enhancement, edge sharpening, noise suppression, and color correction. A multi-scale analysis method based on wavelet transform can be used to perform contrast enhancement and edge sharpening on holographic images, adjust image detail information at different scales, and improve image clarity and visual effects; at the same time, an adaptive filtering algorithm is applied to suppress noise in holographic images and remove possible artifacts and interference; finally, the color of the holographic image is corrected according to the effective projection wavelength range and the spectral characteristics of ambient light to ensure that the image color output by the holographic projection is accurate and natural, highly consistent with the user's painting color intention, and provide users with a high-quality holographic painting display experience.

[0053] In one embodiment, if Figure 2As shown, S3 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S31: Based on the multi-view images of the holographic image data, a three-dimensional geometric deformation field is constructed through the neural radiation field model to generate initial deformation gradient data.

[0054] Specifically, the system extracts two-dimensional image sequences at different perspectives from the holographic image data. These images cover the observation results of the user's painting scene at multiple angles. Each perspective image corresponds to a specific observation direction and position, and contains rich spatial information and painting details. The system performs preliminary preprocessing operations on the extracted multi-perspective images, including image grayscale, normalization, and noise filtering. Grayscale processing converts color images into grayscale images, reducing the amount of data and computational complexity; normalization operations map pixel values ​​to the range of [0, 1] or [-1, 1], which is conducive to the subsequent training and convergence of neural networks; noise filtering uses methods such as median filtering or Gaussian filtering to remove salt and pepper noise and Gaussian noise in the image and improve image quality.

[0055] Preferably, the initial deformation gradient data is generated by the following steps: S311: Perform convolutional neural network feature extraction on the multi-view images in the holographic image data to generate a multi-scale feature map.

[0056] Specifically, the system constructs a deep convolutional neural network (CNN) for feature extraction of multi-view images. The network architecture includes multiple convolutional layers, pooling layers, and activation function layers. The convolutional layer uses convolution kernels of different sizes (such as 3×3, 5×5) to extract local features of the image, such as edges, textures, and shapes; the pooling layer is used to reduce the resolution of the feature map, reduce the amount of calculation, and improve the robustness of the network; the activation function (such as ReLU) introduces nonlinear factors to enhance the network's expressive power.

[0057] After the neural network is built, the preprocessed multi-view images are input into the CNN. After layer-by-layer convolution, pooling, and activation operations, feature maps of different scales are generated. The shallow convolution layer mainly extracts low-level features of the image, such as edges and textures; the deep convolution layer can capture higher-level semantic features, such as the shape and structure of objects. Through feature maps of different levels, the feature information of the image at multiple scales can be fully described, providing a rich feature foundation for the subsequent construction of the three-dimensional geometric deformation field.

[0058] S312: Based on the neural radiation field model, the light field integral calculation of the three-dimensional spatial coordinates and the viewing direction of the multi-scale feature map is performed to generate initial light field distribution data.

[0059] Specifically, the deep learning-based Neural Radiance Field (NeRF) model consists of a multi-layer perceptron (MLP), with inputs of point coordinates and observation angle information in three-dimensional space, and outputs the volume density and radiation color of the point. The hidden layer of the model uses neurons with periodic activation functions (such as Sine activation functions) to better capture high-frequency details and complex geometric structures in the scene. At the same time, jump connections are introduced during the model training process to directly pass input information to subsequent layers, enhancing the model's ability to learn detailed features.

[0060] Specifically, the system combines the multi-scale feature map extracted by CNN with the neural radiation field model to perform light field integral calculations in three-dimensional space coordinates and viewing directions. Specifically, for each point in space, the NeRF model is used to calculate the volume density and radiation color of the point based on its feature map information at different viewing angles, and then integrated along the light path to obtain the final initial light field distribution data. This data describes the propagation, scattering and absorption of light in the three-dimensional space in the painting scene, as well as the color information observed at different viewing angles, providing a comprehensive light field foundation for subsequent deformation gradient calculations.

[0061] S313: Calculate the partial derivative of the initial light field distribution data based on the deformation field function to generate initial deformation gradient data covering the three-dimensional geometric deformation gradient.

[0062] Specifically, the system defines a deformation field function, which describes the geometric deformation of the virtual pen stroke in three-dimensional space due to factors such as changes in the user's observation angle and changes in the curvature of the medium surface. The deformation field function can usually be expressed as a vector field, whose input is the coordinates of the point in the original three-dimensional space and related deformation parameters (such as observation angle, medium curvature, etc.), and the output is the new coordinate position of the point after deformation. The specific form of the function can be derived according to the principles of geometric optics and the theory of elasticity, taking into account the curvature of the light propagation path, the refractive index distribution of the medium, and the effects of user interaction actions on the stretching, compression and rotation of the virtual pen stroke. Among them, it is assumed that the deformation field function is ,in represents the coordinates of a point in three-dimensional space, Represents deformation parameters, which may include rotation angle, scaling factor, translation vector, etc. The deformation field function can be defined as: ; in, is the deformation vector, representing the point p In the deformation parameters For example, consider a simple deformation field function where the deformation consists of a rotation and a translation: ; here, is the rotation matrix, t is the translation vector. For rotations in three-dimensional space, the rotation matrix It can be expressed as: ; This deformation field function describes the point p In rotation angle θ and translation vectors t The new position under the action.

[0063] In practical applications, the deformation field function may be more complex and needs to consider a combination of multiple deformation factors, such as nonlinear deformation, local deformation, etc. For example, in holographic projection, the deformation field function may need to consider the impact of factors such as changes in user viewing angle and changes in medium surface curvature on the position of the virtual pen stroke.

[0064] Based on the partial derivative of the initial light field distribution data of the deformation field function, the initial deformation gradient data covering the three-dimensional geometric deformation gradient is calculated. Specifically, by calculating the partial derivative of the deformation field function with respect to the coordinate position at each point in the three-dimensional space, the deformation rate and direction information of the point in different directions, namely the deformation gradient, are obtained. These deformation gradient data reflect the geometric deformation of the virtual brushstrokes in the three-dimensional space due to various factors, and provide basic data support for the subsequent nonlinear correction and phase compensation parameter calculation. Among them, the deformation gradient is obtained by the following formula: ; in, represents the deformation gradient, represents the gradient operator, which is used to calculate partial derivatives, is the deformation field function, which is used to describe the geometric deformation. is the initial light field distribution data. The formula calculates the gradient of the initial light field distribution and the deformation field function to obtain the deformation gradient of each point in the three-dimensional space, which is used for subsequent nonlinear correction and phase compensation parameter calculation, thereby achieving a high-precision immersive 3D painting experience.

[0065] S32: Perform nonlinear correction on the initial deformation gradient data based on the deformation field function to generate correction gradient data.

[0066] Specifically, after generating the initial deformation gradient data, the system further performs nonlinear correction on it to generate more accurate correction gradient data. This process is based on the deformation field function and takes into account nonlinear factors in the actual optical system, such as the non-uniformity of the medium and the distortion of the projection device.

[0067] Specifically, the system inputs the initial deformation gradient data into a nonlinear correction model. The model adjusts the gradient data based on the definition of the deformation field function, combined with the actual environmental parameters and user interaction signals. For example, when there is a curvature change on the surface of the medium, the correction model will compensate the gradient data accordingly according to the size and direction of the curvature to eliminate the error caused by the deformation of the medium. The corrected gradient data after nonlinear correction is more in line with the actual optical projection conditions, and can more accurately reflect the real displacement of the virtual pen strokes under multiple perspectives, providing more reliable data support for subsequent phase compensation.

[0068] S33: Calculate the phase modulation parameters of the correction gradient data to generate phase compensation parameters.

[0069] Specifically, the system calculates the phase modulation parameters based on the correction gradient data and generates the phase compensation parameters. The purpose of this process is to adjust the phase of the light wave to compensate for the change in optical path difference caused by deformation, thereby ensuring the precise alignment of the virtual pen strokes under multi-view projection.

[0070] During specific implementation, the system converts the correction gradient data into phase modulation parameters. This step is implemented through a physical optical model, taking into account the propagation characteristics of light waves in the medium, interference effects, etc. For example, the system calculates the required phase delay for each point based on the gradient data to offset the optical path difference caused by deformation. These phase modulation parameters are further encoded as the phase information of the hologram to guide the holographic projection equipment to accurately control the phase of the light wave. The generated phase compensation parameters will be used for subsequent holographic layer energy optimization processing to ensure that under different viewing angles and medium conditions, the virtual brushstrokes can be accurately presented in the expected position, providing a high-precision immersive painting experience.

[0071] The above-mentioned intelligent painting method based on optical processing uses technologies such as neural radiation field model and convolutional neural network to deeply process the holographic image data, generate initial deformation gradient data, and perform nonlinear correction and phase compensation parameter calculation on it. This series of steps can effectively improve the accuracy and quality of holographic projection, and provide high-quality data support for subsequent holographic projection output. At the same time, this process has significant advantages in processing complex scenes and multi-view projections, and can meet the needs of high-demand application scenarios such as real-time 3D painting interactive display.

[0072] In one embodiment, if Figure 3 As shown, S4 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S41: performing polarization allocation processing on each holographic layer of phase compensation parameters to allocate non-overlapping polarization direction references based on an orthogonal polarization algorithm to generate orthogonal polarization references for each holographic layer.

[0073] Specifically, the orthogonal polarization algorithm assigns non-overlapping polarization directions to different holographic layers, making each layer independent during projection and avoiding optical interference. Based on the polarization characteristics of light, the oscillation direction of polarized light is consistent with the transmission axis of the polarizer. This characteristic can be used to achieve independent regulation of holographic layers at different depths.

[0074] The system divides the holographic space into multiple holographic layers according to the holographic image data and the depth information of the painting scene, and each holographic layer corresponds to a specific depth range and painting details. The polarization direction allocation scheme of each holographic layer is determined to ensure that the polarization directions of adjacent layers are orthogonal. Preferably, optical elements such as polarizers or liquid crystal polarization modulators can be used to modulate the polarization state of each holographic layer according to a predetermined polarization direction. By precisely controlling the angles and characteristics of these elements, it is ensured that the polarization direction of each holographic layer is accurately allocated according to the orthogonal principle.

[0075] By performing polarization distribution processing on each holographic layer of the phase compensation parameters, orthogonal polarization references of each holographic layer are generated. These orthogonal polarization references will serve as reference standards for polarization state control in the subsequent holographic projection process, ensuring that the optical interference between different holographic layers is effectively suppressed under multi-view projection, thereby improving the clarity and contrast of the holographic image.

[0076] S42: Dynamically adjust the spectrum width of the laser light source based on orthogonal polarization based on the dynamic coherence control algorithm to generate optimized laser parameters.

[0077] Specifically, the dynamic coherence control algorithm optimizes the coherent superposition effect between holographic layers by adjusting the coherence length and coherence phase of the holographic projection light source in real time. The coherence of the light source determines the degree of interference between holographic layers at different depths. Dynamic adjustment can reduce unnecessary interference and improve image quality.

[0078] During the holographic projection process, the system uses coherence detection equipment to monitor the coherence length and coherence phase of the laser light source in real time, and feeds back the monitoring results to the control system. At the same time, according to the distribution of the holographic layer and the changes in the painting scene, the required coherence adjustment trend is predicted to provide a basis for dynamic adjustment. According to the feedback information and prediction results, the spectrum width of the laser light source is dynamically adjusted by controlling the pump current, cavity length or temperature of the laser, narrowing the spectrum width to increase the coherence length, or appropriately widening the spectrum width to reduce coherence, thereby optimizing the coherence superposition effect between the holographic layers, improving the stability and image quality of the holographic projection, and generating optimized laser parameters through dynamic adjustment of the spectrum width of the laser light source. These parameters, including the adjusted laser wavelength, spectrum width, coherence length and coherence phase, will be applied in the subsequent holographic projection process to ensure that the holographic image can maintain high definition, high contrast and stable display effects in different painting scenes and viewing angles.

[0079] S43: Optimizing the energy matching of the optimized laser parameters based on Jones vector calculation to generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

[0080] Preferably, S43 of the intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S431: Perform polarization state matching processing on the optimized laser parameters based on Jones matrix operation to generate polarization matching parameters.

[0081] Specifically, the system uses Jones matrix operations to describe the propagation and changes of light in different polarization elements and holographic layers. According to the optimized laser parameters and the orthogonal polarization reference of the holographic layer, the corresponding Jones matrix model is established to simulate the changes in the polarization state of light in the holographic system.

[0082] By solving the Jones matrix equation, we find the parameter combination that makes the polarization state of the holographic layer and the polarization state of the laser light source best match, namely the polarization matching parameters. These parameters include the angle of the polarizer, the phase delay amount, and the polarization direction adjustment amount of the holographic layer, which are used to guide the subsequent energy weight distribution processing to ensure that the light energy is efficiently utilized and accurately distributed in the holographic layer.

[0083] S432: Perform energy weight distribution processing on the polarization matching parameters based on the energy distribution optimization algorithm to generate optimized energy distribution.

[0084] Specifically, based on the polarization matching parameters and the geometric and optical properties of the holographic layer, the system uses an energy distribution optimization algorithm to calculate the energy weight that should be allocated to each holographic layer at different depths and positions. The algorithm comprehensively considers factors such as the thickness, refractive index, absorption coefficient of the holographic layer, and the lighting requirements of the painting scene to ensure that the energy distribution not only meets the brightness and contrast requirements of the image display, but also effectively suppresses the crosstalk of the non-holographic layer. Based on the energy weight distribution results, the system generates optimized energy distribution data. These data describe in detail the energy intensity distribution of each holographic layer in space, and provide energy guidance for the subsequent reconstruction of the phase and amplitude distribution of the holographic layer, so that the holographic image can present the best visual effects and optical properties when projected.

[0085] S433: Reconstructing the phase and amplitude distribution of the holographic layer according to the optimized energy distribution to generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

[0086] Specifically, the system combines the optimized energy distribution data with the holographic image information of the painting scene, and uses the holographic reconstruction algorithm to reconstruct the phase and amplitude distribution of each holographic layer. According to the guidance of the energy distribution, the phase value and amplitude value of each pixel in the holographic layer are adjusted so that it can accurately reproduce the three-dimensional structure and optical properties of the painting scene while meeting the energy distribution requirements. The reconstructed holographic layer phase and amplitude distribution data are integrated to generate the final optimized holographic layer distribution. This optimized distribution can not only achieve reasonable energy distribution, but also effectively suppress the optical crosstalk between non-holographic layers through precise control of phase and amplitude, improve the clarity, contrast and color accuracy of the holographic image, and provide users with a more realistic and immersive painting experience.

[0087] The above-mentioned intelligent painting method based on optical processing uses technical means such as orthogonal polarization algorithm, dynamic coherence control algorithm and Jones vector operation to finely control the optical parameters in holographic projection. First, the orthogonal polarization algorithm is used to assign non-overlapping polarization direction references to each holographic layer to avoid energy crosstalk. Next, the dynamic coherence control algorithm is used to optimize the spectrum width of the laser light source to improve the contrast and resolution of the holographic projection. Finally, the laser parameters are optimized for energy matching with the help of Jones vector operation to generate an optimized holographic layer distribution that can suppress non-holographic layer crosstalk. This series of steps effectively improves the quality and efficiency of holographic projection, providing users with a clearer and more accurate 3D painting interactive display experience.

[0088] In one embodiment, S5 of an intelligent painting method based on optical processing provided by the present invention specifically includes the following steps: S51: Establish a unified clock for optical and electronic devices based on the optimized holographic layer distribution, and perform clock alignment processing on the unified clock to generate a unified time reference.

[0089] Specifically, in the intelligent painting system, optical devices (such as holographic projectors, optical sensors) and electronic devices (such as stylus pens, computers) have their own clock systems, and there are deviations and asynchronisms between these clocks. In order to ensure that the various devices in the system work together and achieve low-latency holographic projection output, a unified clock needs to be established. In the specific implementation, a high-performance FPGA chip is used as the clock control core, combined with a high-speed clock signal transmission line and a precise clock synchronization algorithm, to integrate and align the clock signals of optical devices and electronic devices. Through clock alignment processing, the clock deviation between different devices is compensated, and a unified time base is generated, so that all devices in the system work under a unified time frame, providing an accurate time reference for subsequent trajectory prediction processing and rendering processing.

[0090] S52: Perform trajectory prediction processing on the acceleration and velocity data of the stylus of the user interaction signal based on the unified time reference to generate future phase distribution parameters.

[0091] Specifically, the acceleration and speed data of the stylus are collected in real time by the inertial measurement unit (IMU) inside the stylus, and these data reflect the dynamic characteristics of the user's drawing movements. The system uses prediction algorithms such as the Kalman filter algorithm and the long short-term memory network (LSTM), combined with a unified time base, to predict the motion trajectory of the stylus in the short future. During the prediction process, factors such as the user's drawing habits, current drawing speed, and acceleration change trends are fully considered to generate future phase distribution parameters. These parameters predict the user's demand for the phase distribution of the holographic layer at future moments, providing forward-looking data support for subsequent rendering processing, enabling the system to prepare rendering content in advance and reduce delays.

[0092] S53: Based on the future phase distribution parameters, 4K resolution rendering processing is performed on the user focus area of ​​the optimized holographic layer to generate a low-latency holographic projection output.

[0093] First, according to the future phase distribution parameters, the position and range of the user's focus area in the optimized holographic layer are determined. The user's focus area refers to the part of the holographic layer that the user is currently focusing on and performing painting operations, which usually corresponds to the spatial area near the stylus. Then, a high-performance graphics rendering engine is used to render the user's focus area with high precision at 4K resolution. During the rendering process, parallel computing technology and multi-threaded optimization algorithms are used to increase the rendering speed and ensure that high-quality images are generated in a short time. At the same time, combined with the optimized holographic layer distribution and phase compensation parameters generated in the previous steps, the rendered image is finally optically processed to ensure that the image output by the holographic projection has high clarity, high contrast and accurate color reproduction. The final low-latency holographic projection output can respond to the user's painting operations in real time, provide a smooth, natural and immersive creation experience, and meet the strict requirements of high-precision scenes such as medical visualization and industrial simulation for real-time 3D painting.

[0094] In summary, the present invention provides an intelligent painting method based on optical processing, which ensures the time synchronization of the two by establishing a unified clock for optical and electronic devices and performing clock alignment processing. Trajectory prediction processing of user interaction signals based on a unified time reference can generate future phase distribution parameters in advance, providing support for real-time updates of holographic projections. Finally, 4K resolution rendering processing is performed on the user focus area of ​​the optimized holographic layer to generate a low-latency holographic projection output, which not only improves the quality and detail expression of the projected image, but also greatly reduces the latency of the system, allowing users to perform 3D painting interaction operations more naturally and smoothly, improving the user experience and the overall performance of the system.

[0095] Preferably, if Figure 4 As shown, the present invention provides an intelligent painting system 600 based on optical processing, and the intelligent painting system is configured with the following modules: A data acquisition module 610 is used to acquire holographic light field data, user interaction signals and environmental parameters based on a multi-dimensional sensor; The data preprocessing module 620 is used to perform time-frequency alignment and noise filtering on the holographic light field data, the user interaction signal and the environmental parameters to generate holographic image data, which is used to indicate the geometric deformation gradient of the multi-view projection and the user interaction intention; A holographic gradient calculation module 630 is used to perform multi-view projection deformation gradient calculation on the holographic image data based on the neural radiation field model and the deformation field function to generate phase compensation parameters; The holographic distribution optimization module 640 is used to perform holographic layer energy optimization processing on the phase compensation parameters based on the orthogonal polarization encoding algorithm and the dynamic coherence control algorithm to generate an optimized holographic layer distribution; The holographic rendering output module 650 is used to perform photon-electron clock alignment on the optimized holographic layer distribution based on the predictive rendering algorithm, combined with generating low-latency holographic projection output, and the holographic projection output is used for interactive display of real-time 3D painting.

[0096] In summary, the intelligent painting system based on optical processing provided by the present invention obtains comprehensive data information through multi-dimensional sensors, and ensures the accuracy and reliability of the data through time-frequency alignment and noise filtering. The holographic image data is deeply processed using the neural radiation field model and the deformation field function to generate phase compensation parameters, which further optimizes the energy distribution of the holographic layer. Finally, through the predictive rendering algorithm and photon-electron clock alignment technology, low-latency holographic projection output can be achieved, providing users with a high-quality real-time 3D painting interactive display experience. This process has significant beneficial effects in improving the accuracy of holographic projection, enhancing user interactivity, and reducing system latency, and has promoted the application and development of holographic technology in the field of real-time interactive display.

[0097] Preferably, the data preprocessing module 620 is configured with the following units: A spectral filtering unit 621, for spectrally filtering the ambient light of the environmental parameters based on an optical filter to generate an effective projection wavelength range; A synchronization processing unit 622, configured to perform time synchronization alignment processing based on the stylus pen posture signal of the user interaction signal and the holographic light field data through a time reference signal generated by an optical crystal oscillator to generate synchronization data; The holographic image generation unit 623 is used to process the synchronization data and the effective projection wavelength range based on the data dimension reduction algorithm, compress the four-dimensional holographic light field data into a 256-dimensional low-dimensional feature vector, and generate holographic image data.

[0098] Preferably, the holographic gradient calculation module 630 is configured with the following units: A gradient generating unit 631 is used to construct a three-dimensional geometric deformation field through a neural radiation field model based on the multi-view images of the holographic image data, and generate initial deformation gradient data; Preferably, the gradient generating unit 631 is configured with the following subunits: The feature map generation subunit 6311 is used to perform convolutional neural network feature extraction on the multi-view images in the holographic image data to generate a multi-scale feature map; A light field data generation subunit 6312 is used to perform light field integral calculations on the three-dimensional spatial coordinates and the viewing direction of the multi-scale feature map based on the neural radiation field model to generate initial light field distribution data; The deformation gradient calculation subunit 6313 is used to calculate the partial derivative of the initial light field distribution data based on the deformation field function to generate initial deformation gradient data covering the three-dimensional geometric deformation gradient.

[0099] A gradient correction unit 632 is used to perform nonlinear correction on the initial deformation gradient data based on the deformation field function to generate correction gradient data; The phase parameter calculation unit 633 is used to calculate the phase modulation parameters of the correction gradient data to generate phase compensation parameters.

[0100] Preferably, the holographic distribution optimization module 640 is configured with the following units: The polarization allocation unit 641 is used to allocate non-overlapping polarization direction references to each holographic layer of the phase compensation parameter based on an orthogonal polarization algorithm to perform polarization allocation processing to generate orthogonal polarization references for each holographic layer; A parameter optimization unit 642 is used to dynamically adjust the spectrum width of the laser light source based on the orthogonal polarization reference based on a dynamic coherence control algorithm to generate optimized laser parameters; The distribution optimization unit 643 is used to perform energy matching optimization processing on the optimized laser parameters based on Jones vector calculation to generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

[0101] Preferably, the distribution optimization unit 643 is configured with the following subunits: The polarization matching subunit 6431 is used to perform polarization state matching processing on the optimized laser parameters based on Jones matrix operation to generate polarization matching parameters; The energy distribution subunit 6432 is used to perform energy weight distribution processing on the polarization matching parameters based on the energy distribution optimization algorithm to generate an optimized energy distribution; The holographic layer reconstruction subunit 6433 is used to reconstruct the holographic layer phase and amplitude distribution according to the optimized energy distribution, and generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

[0102] Preferably, the holographic rendering output module 650 is configured with the following units: The time reference establishing unit 651 is used to establish a unified clock for optical and electronic devices based on the optimized holographic layer distribution, and perform clock alignment processing on the unified clock to generate a unified time reference; A trajectory prediction unit 652 is used to perform trajectory prediction processing on the acceleration and velocity data of the stylus of the user interaction signal based on a unified time reference to generate future phase distribution parameters; The holographic projection output unit 653 is used to perform 4K resolution rendering processing on the user focus area of ​​the optimized holographic layer based on the future phase distribution parameters to generate a low-latency holographic projection output.

[0103] In one embodiment, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the above-mentioned intelligent painting method based on optical processing is implemented.

[0104] In one embodiment, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned intelligent painting method based on optical processing is implemented.

[0105] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0106] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial descriptions of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0107] As mentioned above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various changes or substitutions, and these should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An intelligent painting method based on optical processing, characterized in that: The following steps are involved: S1: Acquire holographic light field data, user interaction signals and environmental parameters based on multi-dimensional sensors; S2: performing time-frequency alignment and noise filtering processing on the holographic light field data, the user interaction signal and the environmental parameter to generate holographic image data, where the holographic image data is used to indicate the geometric deformation gradient of the multi-view projection and the user interaction intention; S3: performing multi-view projection deformation gradient calculation on the holographic image data based on the neural radiation field model and the deformation field function to generate phase compensation parameters; S4: performing holographic layer energy optimization processing on the phase compensation parameters based on an orthogonal polarization encoding algorithm and a dynamic coherence control algorithm to generate an optimized holographic layer distribution; S5: Performing photon-electron clock alignment on the optimized holographic layer distribution based on a predictive rendering algorithm, and combining it to generate a low-latency holographic projection output, wherein the holographic projection output is used for interactive display of real-time 3D paintings.

2. The intelligent painting method according to claim 1, characterized in that: The S2 includes: S21: spectrally filter the ambient light of the environmental parameters based on the optical filter to generate an effective projection wavelength range; S22: Based on the stylus pen posture signal of the user interaction signal and the holographic light field data, a time synchronization alignment process is performed through a time reference signal generated by an optical crystal oscillator to generate synchronization data; S23: Processing the synchronization data and the effective projection wavelength range based on a data dimension reduction algorithm, compressing the four-dimensional holographic light field data into a 256-dimensional low-dimensional feature vector, and generating holographic image data.

3. The intelligent painting method according to claim 1, characterized in that: The S3 includes: S31: Based on the multi-view images of the holographic image data, construct a three-dimensional geometric deformation field through a neural radiation field model to generate initial deformation gradient data; S32: performing nonlinear correction on the initial deformation gradient data based on the deformation field function to generate correction gradient data; S33: Calculate phase modulation parameters on the correction gradient data to generate phase compensation parameters.

4. The intelligent painting method according to claim 3, characterized in that: The S31 includes: S311: performing convolutional neural network feature extraction on the multi-view images in the holographic image data to generate a multi-scale feature map; S312: performing light field integral calculation of the three-dimensional spatial coordinates and the viewing direction of the multi-scale feature map based on the neural radiation field model to generate initial light field distribution data; S313: Calculating partial derivatives of the initial light field distribution data based on the deformation field function to generate initial deformation gradient data covering three-dimensional geometric deformation gradients.

5. The intelligent painting method according to claim 1, characterized in that: The S4 includes: S41: performing polarization allocation processing on each holographic layer of the phase compensation parameter to allocate non-overlapping polarization direction references based on an orthogonal polarization algorithm to generate orthogonal polarization references for each holographic layer; S42: dynamically adjusting the spectrum width of the laser light source based on the orthogonal polarization based on a dynamic coherence control algorithm to generate optimized laser parameters; S43: Optimizing the energy matching of the optimized laser parameters based on Jones vector calculation to generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

6. The intelligent painting method according to claim 1, characterized in that: The S43 includes: S431: performing polarization state matching processing on the optimized laser parameters based on Jones matrix operation to generate polarization matching parameters; S432: performing energy weight distribution processing on the polarization matching parameters based on an energy distribution optimization algorithm to generate an optimized energy distribution; S433: Reconstructing the phase and amplitude distribution of the holographic layer according to the optimized energy distribution to generate an optimized holographic layer distribution that suppresses non-holographic layer crosstalk.

7. The intelligent painting method according to any one of claims 1 to 6, characterized in that: The S5 includes: S51: establishing a unified clock for optical and electronic devices based on the optimized holographic layer distribution, and performing clock alignment processing on the unified clock to generate a unified time reference; S52: performing trajectory prediction processing on the acceleration and velocity data of the stylus of the user interaction signal based on the unified time reference to generate future phase distribution parameters; S53: Based on the future phase distribution parameters, 4K resolution rendering processing is performed on the user focus area of ​​the optimized holographic layer to generate the low-latency holographic projection output.

8. An intelligent painting system based on optical processing, characterized in that: The system comprises: A data acquisition module, used to acquire holographic light field data, user interaction signals and environmental parameters based on a multi-dimensional sensor; A data preprocessing module, used to perform time-frequency alignment and noise filtering on the holographic light field data, the user interaction signal and the environmental parameters to generate holographic image data, wherein the holographic image data is used to indicate the geometric deformation gradient of the multi-view projection and the user interaction intention; A holographic gradient calculation module, used to perform multi-view projection deformation gradient calculation on the holographic image data based on a neural radiation field model and a deformation field function, and generate phase compensation parameters; A holographic distribution optimization module, used to perform holographic layer energy optimization processing on the phase compensation parameters based on an orthogonal polarization encoding algorithm and a dynamic coherence control algorithm to generate an optimized holographic layer distribution; A holographic rendering output module is used to perform photon-electron clock alignment on the optimized holographic layer distribution based on a predictive rendering algorithm, combined with generating a low-latency holographic projection output, wherein the holographic projection output is used for interactive display of real-time 3D paintings.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images