Image geometric distortion intelligent control method based on machine vision and image projector

Abstract

The invention relates to the technical field of projectors, in particular to an intelligent picture geometric distortion control method based on machine vision and an image projector, and the method comprises the steps: obtaining image data of a to-be-corrected projection picture, and carrying out the feature analysis of the image data, so as to obtain a distortion feature value; matching a correction optimization model corresponding to the distortion characteristic value, so as to call a correction parameter from a preset correction parameter library through the correction optimization model; simulating a correction process corresponding to the projection picture to be corrected according to the correction parameter; the correction parameters are adjusted according to the correction process, and the projection picture to be corrected is corrected according to the adjusted correction parameters, the corresponding correction optimization model is matched based on the characteristic values, the adaptive correction parameters can be dynamically called from the preset parameter library, the limitation of a traditional fixed algorithm is changed, and the correction efficiency is improved. The correction strategy is highly coupled with the actual distortion scene, and the correction efficiency and pertinence are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of projectors, in particular to a picture geometric distortion intelligent control method based on machine vision and an image projector. BACKGROUND

[0002] In the scene where the projection display technology is widely applied, such as intelligent conference system, immersive virtual reality (VR), digital exhibition hall and home theater, etc., the diversity of installation position, projection angle and screen form of the projection device often leads to geometric distortion of the picture, which seriously affects the visual effect. The traditional geometric distortion correction method mainly depends on manual adjustment or automatic correction based on fixed algorithm. The former requires users to have certain operation experience and is time-consuming and laborious, and the latter is difficult to adapt to complex and variable distortion scenes. For example, in the scene of different projection distances, curved screens or multi-projector fusion, the fixed correction model and parameter library cannot accurately match the real-time distortion characteristics, resulting in insufficient correction accuracy or over-correction. Moreover, there is a lack of dynamic simulation and parameter optimization mechanism for the correction process, making it difficult to achieve adaptive high-precision correction in diversified environments.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a picture geometric distortion intelligent control method based on machine vision and an image projector, which aims to solve the technical problem that the traditional geometric distortion correction method mainly depends on manual adjustment or automatic correction based on fixed algorithm. The former requires users to have certain operation experience and is time-consuming and laborious, and the latter is difficult to adapt to complex and variable distortion scenes.

[0005] To achieve the above purpose, the present application provides a picture geometric distortion intelligent control method based on machine vision, which comprises: obtaining image data of a projection picture to be corrected, and performing feature analysis on the image data to obtain distortion characteristic values; matching a correction optimization model corresponding to the distortion characteristic values to call correction parameters from a preset correction parameter library through the correction optimization model; simulating a correction process corresponding to the projection picture to be corrected according to the correction parameters; adjusting the correction parameters according to the correction process, and correcting the projection picture to be corrected according to the adjusted correction parameters.

[0006] Optionally, the image data includes edge feature parameters, and the obtaining of the image data of the projection picture to be corrected and the feature analysis on the image data to obtain the distortion characteristic values comprise: obtaining an edge feature parameter of the to-be-corrected projection picture, wherein the edge feature parameter comprises an edge curvature parameter and a pixel offset parameter; extracting a key point coordinate from the edge feature parameter by using a Hough transform, and constructing a projection plane equation and a distortion mapping equation according to the key point coordinate, so as to obtain a distortion coefficient parameter; obtaining a distortion vector according to the distortion coefficient parameter, and performing normalization processing on the distortion vector to obtain a distortion feature value, wherein the distortion vector comprises a bending degree, an offset amount and a rotation angle.

[0007] Optionally, the calling of the correction parameter from the preset correction parameter library through the correction optimization model comprises: obtaining a parameter feature corresponding to each parameter combination in the correction parameter library, and calculating a similarity between each parameter feature and the distortion feature value through a mapping relationship preset in the correction optimization model; when the similarity exceeds a preset threshold, optimizing the current parameter combination according to a preset correction optimization rule, and generating a correction parameter corresponding to the optimized parameter combination.

[0008] Optionally, the simulating of the correction process corresponding to the to-be-corrected projection picture according to the correction parameter comprises: constructing a virtual projection geometric model according to a discrete control point of the correction parameter, and performing mesh division on the virtual projection geometric model through a quadrilateral mesh unit to obtain a mesh division model; establishing a pixel mapping equation by using a bilinear interpolation function, and calculating deformation field data and luminance field data of the mesh division model through the pixel mapping equation and a ray tracing solver; performing feature extraction on the deformation field data and the luminance field data through a three-layer convolutional neural network to obtain a feature mapping relationship of the deformation field data and the luminance field data; processing real-time picture data collected by a camera according to the feature mapping relationship, so as to obtain coordinate offset data and luminance change data of the to-be-corrected projection picture in the correction process; fusing the coordinate offset data and the luminance change data through a particle filter to obtain an actual distortion curve and an actual luminance distribution curve; verifying the reliability of the correction parameter according to the actual distortion curve, the actual luminance distribution curve, the deformation field data and the luminance field data.

[0009] Optionally, the adjusting of the correction parameter according to the correction process, and the correction of the to-be-corrected projection picture according to the adjusted correction parameter comprise: When the correction parameter is verified to be reliable, a fuzzy evaluator based on a confidence interval is used to calculate a deformation evaluation parameter and a brightness evaluation parameter corresponding to the correction parameter, and a to-be-adjusted parameter sequence is generated according to comparison results of the deformation evaluation parameter and the brightness evaluation parameter with preset threshold values respectively; A gradient boosting tree regressor is trained according to the to-be-adjusted parameter sequence, a parameter adjustment amount is predicted by the gradient boosting tree regressor, and a parameter value of the correction parameter is adjusted according to the parameter adjustment amount, so as to obtain discrete control point coordinates of an updated correction parameter; The discrete control point coordinates of the updated correction parameter are fitted by using a NURBS curve, and a speed planning curve is obtained by performing dynamic speed planning on the NURBS curve under constraints of maximum pixel displacement and brightness change rate; Curvature values at each discrete point of the speed planning curve are calculated, a transition point is inserted in a curvature mutation area by using spline interpolation, an adjusted correction parameter is obtained, and the to-be-corrected projection picture is corrected according to the adjusted correction parameter.

[0010] Optionally, before the image data of the to-be-corrected projection picture is obtained, the method further includes: Material parameters of a projection screen are obtained to construct a geometric distortion statics model corresponding to the projection screen and perform correction analysis on the geometric distortion statics model, so as to obtain a screen deformation cloud picture and a brightness distribution cloud picture corresponding to the projection picture under each initial correction parameter. Discrete points of each initial correction parameter in the screen deformation cloud picture and edge gradients of each initial correction parameter in the brightness distribution cloud picture are extracted respectively, and correction speeds and brightness change rates corresponding to each initial correction parameter at each discrete point are calculated, so as to obtain screen deformation variable change curves in a correction process corresponding to the projection picture under different initial correction parameters. A plurality of first correction parameters are selected from the initial correction parameters according to the screen deformation variable change curves. Each first parameter in the first correction parameters is optimized according to a preset parameter optimization algorithm, and a plurality of optimized parameter combinations are obtained and saved in a preset correction parameter library.

[0011] Optionally, the material parameters include a surface roughness parameter and a material refractive index, and the obtaining of the material parameters of the projection screen, the construction of the geometric distortion statics model corresponding to the projection screen, and the correction analysis on the geometric distortion statics model to obtain the screen deformation cloud picture and the brightness distribution cloud picture corresponding to the projection picture under each initial correction parameter include: Obtaining surface roughness parameters and material refractive index of a projection screen to calculate a distortion stiffness matrix corresponding to the projection screen, wherein the surface roughness parameters include microstructure depth and distribution density, and the distortion stiffness matrix is determined by the material refractive index, microstructure depth and distribution density; Obtaining pixel node distortion values through the distortion stiffness matrix and mapping to grid nodes to establish a bicubic spline interpolation equation, wherein the grid nodes are divided by quadrilateral elements and the grid size in the edge region is reduced to one third of the basic grid size; Calculating edge curvature values according to the bicubic spline interpolation equation to obtain a distortion field calculation equation, i.e., the geometric distortion statics model, wherein the edge curvature values are determined by pixel gradient vectors and normal vectors; Establishing a nonlinear pixel displacement equation set according to the distortion field calculation equation to obtain screen deformation nephograms and brightness distribution nephograms corresponding to the projection picture under each initial correction parameter; wherein the nonlinear pixel displacement equation set is solved by using the quasi-Newton method until the residual error is less than a preset value.

[0012] Optionally, the correction speed and brightness change rate corresponding to each initial correction parameter at each discrete point are calculated to obtain screen deformation variable change curves in the correction process of the projection picture under different initial correction parameters, including: Obtaining a parameter curve corresponding to each initial correction parameter and obtaining coordinate data sets of discrete points on the parameter curve; wherein different initial correction parameters correspond to different curve shapes, curve lengths and curve curvatures; Fitting the discrete points by using a Bezier curve to obtain brightness distribution values; wherein the brightness distribution values are obtained by calculating the mapping relationship between curve parameters and brightness; Establishing an optical kinematics equation according to the brightness distribution values and solving to obtain brightness change rate values of the discrete points by using Euler integral formula; wherein the brightness change rate values are obtained by integrating brightness with respect to time; Constructing a state equation according to the brightness change rate values of the discrete points and the deformation variables on the screen deformation nephogram, and solving to obtain a deformation time function by using the least square method, and obtaining a screen deformation variable change curve according to the deformation time function.

[0013] Optionally, the plurality of first correction parameters are selected from the plurality of initial correction parameters according to the screen deformation variable change curve, including: Determining a current screen deformation variable corresponding to each initial correction parameter through the screen deformation variable change curve, constructing a constraint equation according to a preset maximum deformation variable of the projection picture and the current screen deformation variable corresponding to each initial correction parameter, and calculating a deformation variable constraint optimization parameter by using a conjugate gradient iteration. According to the shape variable constraint optimization parameter, a multi-objective optimization function of a shape variable target function and a brightness target function is established, and a mapping relationship between a parameter vector and a constraint response vector corresponding to each initial correction parameter is obtained through a support vector machine of a radial basis kernel function; According to the mapping relationship, a constraint optimization equation group is constructed by using a KKT condition, and an optimal correction parameter and a Lagrange multiplier corresponding to each initial correction parameter are obtained by solving through a sequential quadratic programming method; If a norm difference value of the optimal correction parameter and an initial parameter corresponding to the initial correction parameter is less than a preset convergence threshold, the optimal correction parameter is stored as a first correction parameter.

[0014] In addition, to achieve the above object, the application further provides an image projector, which comprises a memory, a processor, and a machine vision-based picture geometric distortion intelligent control program stored in the memory and capable of running on the processor, and the machine vision-based picture geometric distortion intelligent control program is configured to implement the steps of the machine vision-based picture geometric distortion intelligent control method according to any one of the above.

[0015] The application provides a machine vision-based picture geometric distortion intelligent control method, which acquires image data of a picture to be corrected and extracts distortion characteristic values in real time through machine vision, avoids tedious manual adjustment, and realizes accurate quantification of distortion characteristics. Based on matching of the characteristic values and a corresponding correction optimization model, adaptive correction parameters can be dynamically called from a preset parameter library, the limitation of a traditional fixed algorithm is changed, the correction strategy is highly coupled with an actual distortion scene, correction efficiency and pertinence are significantly improved, a correction process simulation mechanism is introduced, a correction effect is simulated in advance through the called correction parameters, the rationality of a correction scheme can be visually evaluated before actual adjustment, and loss caused by multiple trial and error due to direct hardware adjustment is avoided. Meanwhile, the correction parameters are dynamically optimized based on simulation results, a closed-loop control is formed, nonlinear distortion problems in a complex scene can be adaptively solved, defects of parameter fixation and lack of feedback optimization in traditional methods are effectively solved, and correction accuracy and system robustness are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a schematic structural diagram of an image projector of a hardware running environment related to an embodiment scheme of the application; Figure 2 is a flowchart of an embodiment of the machine vision-based picture geometric distortion intelligent control method of the application; Figure 3 is a structural block diagram of an embodiment of the machine vision-based picture geometric distortion intelligent control system of the application.

[0017] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of an image projector in the hardware operating environment involved in the embodiments of the present invention.

[0020] like Figure 1 As shown, the image projector may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0021] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the image projector and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0022] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a machine vision-based intelligent control program for image geometric distortion.

[0023] exist Figure 1The network interface 1004 is mainly used for connecting a background server and communicating data with the background server, and the user interface 1003 is mainly used for connecting an external device. The image projector calls a machine vision-based picture geometric distortion intelligent control program stored in the memory 1005 through the processor 1001, and executes the machine vision-based picture geometric distortion intelligent control method provided by the embodiment of the application.

[0024] Based on the above hardware structure, the embodiment of the machine vision-based picture geometric distortion intelligent control method is provided.

[0025] Reference Figure 2 , Figure 2 The flowchart of the embodiment of the machine vision-based picture geometric distortion intelligent control method is provided.

[0026] In an embodiment, the machine vision-based picture geometric distortion intelligent control method comprises the following steps. Step S100, acquiring image data of a projection picture to be corrected, and performing feature analysis on the image data to obtain distortion feature values.

[0027] The image data refers to a set of visual information of a picture projected on a screen by a projection device, which is usually stored in the form of a pixel matrix and contains information such as brightness, color and geometric shape, and is used to provide original data support for subsequent distortion analysis. The image data can be obtained in real time by a camera, a sensor or a visual acquisition module built in the projection device. For example, the actual display effect on the screen can be captured at high resolution by using a double-camera stereo imaging or a structured light projection technology. The feature analysis is a technical process of extracting and quantifying key geometric or visual features in the image data by using an algorithm. For example, the feature analysis includes edge detection, corner recognition, shape matching and other technologies. The distortion feature values are a set of parameters for quantitatively describing the degree of picture geometric distortion, such as perspective deformation coefficients, nonlinear distortion amplitudes and edge distortion rates, which are used to represent the stretching, compression or bending degree of the projection picture in different directions. The image data is optimized by preprocessing, and then the key geometric features in the picture are located by using a feature extraction algorithm, such as recognizing the bending degree of a calibration grid line or detecting the deviation amount of a picture edge from an ideal geometric shape. This process realizes automatic quantification of distortion features and avoids errors caused by manual measurement.

[0028] Step S200, matching a correction optimization model corresponding to the distortion feature values to call correction parameters from a preset correction parameter library through the correction optimization model.

[0029] The correction optimization model is a mathematical or machine learning based algorithm framework that maps distortion feature values to optimal correction parameters, exemplary including support vector machines, random forests or neural network models such as convolutional neural networks (CNN). The pre-set correction parameter library is a collection of verified effective correction parameters under different distortion scenarios, which can include projection lens focal length adjustment, pixel coordinate mapping table, multi-projector fusion overlap area compensation value, etc. By inputting the extracted distortion feature values into the correction optimization model, the model retrieves the closest parameter combination in the parameter library through feature matching algorithms. For example, if the feature values show that the picture has significant barrel distortion, the model will call the correction parameters designed for such distortion. This process replaces fixed algorithms with dynamic retrieval, enhancing adaptability to complex scenarios.

[0030] Step S300, simulating a correction process corresponding to the to-be-corrected projection picture according to the correction parameters.

[0031] Correction process simulation refers to simulating the actual effect of correction parameters on the picture in a virtual environment through computer simulation technology. Exemplary, including simulating the projection light path using a three-dimensional modeling engine, or generating a corrected virtual picture through image processing algorithms. Visual evaluation is to present the simulation results in an intuitive form for the system or user to verify the rationality of the correction scheme. When building the simulation environment, a virtual scene needs to be established based on the physical parameters of the projection device and the screen geometric model, for example, if the screen is spherical, it needs to include the light tracing calculation of curved surface reflection. The corrected geometric shape is simulated by applying the correction parameters, and the corrected virtual picture is output. After generating the simulation results, the correction effect is evaluated through quantitative indicators, and if it does not meet the standard, the subsequent parameter optimization process is triggered.

[0032] Step S400, adjusting the correction parameters according to the correction process, and correcting the to-be-corrected projection picture according to the adjusted correction parameters.

[0033] Parameter adjustment refers to iterative optimization of the original correction parameters based on the simulation results, exemplary including parameter value fine-tuning through gradient descent method, genetic algorithm or artificial neural network. Dynamic optimization closed loop refers to a cyclic mechanism of simulation evaluation and adjustment, continuously correcting parameters until the pre-set precision threshold is met. By comparing the simulation results with the ideal correction target, the correction residual of the current parameters is calculated, for example, locating the local distortion area of the picture. The parameter value is adjusted according to the error distribution, for example, enhancing the pixel mapping coefficient of a specific area or adjusting the local trapezoidal correction intensity of the projection device. Finally, the optimized parameters are sent to the projection device, and the geometric distortion of the actual projection picture is corrected in real time by adjusting the lens displacement, digital image processing algorithm or multi-device synchronous control, for example, compensating barrel distortion through DLP chip pixel offset or eliminating the splicing seam through edge fusion algorithm.

[0034] The embodiment provides a picture geometric distortion intelligent control method based on machine vision, which captures image data of a projection picture in real time, extracts geometric distortion characteristic values, dynamically matches correction parameters in a preset parameter library by combining a correction optimization model, and further realizes accurate correction through simulation verification and parameter iteration optimization, so that the following technical effects can be achieved: the geometric distortion degree is directly quantified through machine vision technology, so that subjectivity and low efficiency of manual measurement are avoided; a parameter dynamic calling mechanism based on the characteristic values improves the adaptability to complex scenes such as curved screens and multi-projection fusion; a closed-loop mechanism of simulation and parameter optimization reduces the trial-and-error cost and avoids the cumulative error of direct hardware adjustment through virtual verification; the dynamic closed-loop control can cope with environmental factors such as projection distance changes or screen deformation, and maintain the long-term stability of the correction effect; the whole automatic process reduces the user operation burden, and at the same time, the visual simulation provides an intuitive correction effect preview, improving the interactive friendliness. Through the combination of machine vision and intelligent optimization, the method realizes high precision, self-adaptation and high efficiency of geometric distortion correction, and solves the problems of insufficient precision and lack of flexibility of traditional methods in complex environments.

[0035] In one of the embodiments, the image data includes edge feature parameters, the image data of the projection picture to be corrected is acquired, and feature analysis is performed on the image data to obtain distortion characteristic values, including: The edge feature parameters of the projection picture to be corrected are acquired, wherein the edge feature parameters include edge curvature parameters and pixel offset parameters; The edge feature parameters can be geometric and position information of the edge region in the projection picture, and are used to accurately capture the distortion form of the picture edge. The edge curvature parameters can be numerical values describing the bending degree of the edge, such as the curvature radius or the curvature direction, which can be extracted by an image processing algorithm to extract the curvature variation characteristics of the edge contour. The pixel offset parameters can be displacement amounts of the edge pixels relative to ideal positions, such as recording the pixel coordinate deviation values by comparing the edge contour of the calibration pattern with an ideal coordinate system. Illustratively, the edge curvature parameters can include a numerical distribution of local curvature radii, and the pixel offset parameters can include displacement vectors in the X-axis or Y-axis directions.

[0036] Key point coordinates are extracted from the edge feature parameters by using a Hough transform, and a projection plane equation and a distortion mapping equation are constructed according to the key point coordinates to obtain distortion coefficient parameters; The Hough transform can be an image processing algorithm based on a voting mechanism, which is used to locate feature points with geometric rules. For example, the Hough transform can convert the image space into a parameter space by inputting the edge feature parameters into the algorithm, such as mapping the pixel points into a curve in the parameter space when detecting straight lines, and determining the geometric parameters by the intersection points of the curve. The key point coordinates can be a set of coordinates of edge inflection points or structural intersection points, such as the intersection points of the calibration straight lines or the center positions of the circles. The projection plane equation can be a plane equation in a three-dimensional coordinate system or a two-dimensional parameterized equation, which is used to establish an ideal distortion-free reference model. The distortion mapping equation can be a mathematical model describing the relationship between the actual distortion and the ideal plane, such as a polynomial function containing radial distortion parameters. In one specific embodiment, the key point coordinates are fitted by the least squares method to determine the projection plane equation parameters, and then the distortion mapping equation is established by comparing the deviation of the actual coordinates from the ideal plane, so as to obtain the distortion coefficient parameters.

[0037] According to the distortion coefficient parameters, a distortion vector is obtained, and a distortion feature value is obtained by normalizing the distortion vector, wherein the distortion vector includes a curvature, a displacement, and a rotation angle.

[0038] The distortion coefficient parameters can be a set of numerical values quantifying the distortion type, such as radial distortion coefficients or perspective distortion parameters. The distortion vector can be a set of vector forms integrated from the distortion coefficient parameters, which is used to facilitate subsequent calculation and model matching. The curvature can be a numerical value describing the curvature change of the edge or the overall picture, the displacement can be the displacement of the pixel position relative to the ideal position, and the rotation angle can be the rotation deviation angle of the overall or local picture. The normalization processing can be to unify the numerical value range of the distortion vector to a fixed interval by a standardization algorithm. For example, if the original range of the curvature is [0, 2] and the range of the displacement is [5, 5] pixels, then after normalization, each parameter is converted to a numerical value in the interval [0, 1], thereby eliminating the influence of dimension difference on subsequent model matching.

[0039] This embodiment can capture the curvature and displacement information of the edge region by obtaining the edge feature parameters, extract the key points by using the Hough transform, construct the reference plane and distortion mapping model to quantify the distortion coefficient parameters, and finally integrate the distortion parameters into the standardized vector to eliminate the dimension difference, which can achieve the technical effects of improving the accuracy and comprehensiveness of distortion feature extraction and enhancing the adaptive correction ability of complex distortion scenes. This method can represent the perspective distortion, nonlinear distortion, and rotation deviation at the same time through multi-dimensional distortion modeling and parameter standardization processing, and reduce noise interference and model matching errors through robust key point positioning and normalization algorithm, thereby optimizing the dynamic matching efficiency and geometric restoration accuracy of the subsequent correction model.

[0040] In one of the embodiments, the correction parameters are called from the preset correction parameter library by the correction optimization model, including: obtaining the parameter characteristics corresponding to each parameter combination in the correction parameter library, and calculating the similarity between each parameter characteristic and the distortion characteristic value through the mapping relationship preset in the correction optimization model; when the similarity exceeds a preset threshold, the preset correction optimization rule is called to optimize the current parameter combination, and the correction parameters corresponding to the optimized parameter combination are generated.

[0041] The parameter characteristics can be a set of metadata describing the characteristics of the correction parameter combination, which is used to provide a quantifiable comparison basis for the matching of the parameters and the distortion characteristics. The parameter characteristics can be obtained by quickly retrieving the description information of the parameters through the index mechanism of the parameter library. For example, the parameter characteristics can include the distortion type applicable to the parameters, the projection distance range in which the parameters take effect, the screen curvature radius matching value, etc. The mapping relationship can be a mathematical or logical rule preset in the correction optimization model, which is used to quantitatively express the association between the distortion characteristic value and the parameter characteristics. For example, the mapping relationship can be a nonlinear mapping established by the fully connected layer of a neural network, or a matching priority of the parameter characteristics and the distortion characteristics defined by a rule engine.

[0042] The calculation of the similarity can be realized by numerically or logically comparing the distortion characteristic value and the parameter characteristics based on the mapping relationship. For example, when the distortion characteristic value shows that there is 15% barrel distortion in the current picture, the model can evaluate the matching degree of the parameter combination marked as “applicable to barrel distortion 10% to 20%” in the parameter characteristics through cosine similarity calculation. The technical effect of this technical operation is to realize the quantitative matching of the parameters and the distortion scene through multi-dimensional weighted calculation (such as assigning weights to the distortion type, amplitude, screen shape, etc. and then synthesizing the scores), thereby avoiding the inefficiency of full traversal.

[0043] When the similarity exceeds a preset threshold, the preset correction optimization rule is called to optimize the current parameter combination. The threshold can be a preset numerical benchmark (such as 0.8), which is used to determine whether the matching degree of the parameter combination and the current distortion scene meets the condition for triggering optimization. The correction optimization rule can be a preset parameter adjustment algorithm or strategy, which can include gradient descent iteration, parameter weight weighted fusion, parameter boundary constraint, etc. The technical effect of this technical operation is to refine the parameter combination through the parameter fine-tuning algorithm. For example, when the focal length adjustment amount of the original parameter combination is 2mm but the simulation still shows edge distortion, the optimization rule can increase the focal length adjustment amount to 2.5mm and recalculate the pixel mapping table, thereby improving the matching accuracy.

[0044] The generating of the optimized parameter combination corresponding to the correction parameter can be realized through a process of ensuring the compatibility between parameters. For example, if the original parameter only includes a lens shift parameter, the optimized parameter can supplement a brightness compensation coefficient in the multi-projector fusion. The technical effect of this technical operation is to form a composite correction scheme, for example, to solve the problems of geometric alignment and color consistency in the multi-projector fusion scenario, while avoiding conflicts in other dimensions caused by single parameter adjustment.

[0045] This embodiment realizes rapid screening of potential candidate parameters through quantitative matching of parameter features and distortion features, combines threshold judgment to trigger correction optimization rules for fine adjustment of parameter combination, and generates final correction parameters through parameter compatibility verification, which can achieve the technical effect of improving parameter calling accuracy and efficiency. This scheme uses a hierarchical strategy of mapping relationship and similarity calculation to ensure processing speed while enhancing parameter adaptation accuracy. The structured storage of parameter features and dynamic calculation characteristics support flexible expansion of the system, which can adapt to future new distortion types or hardware configuration needs.

[0046] In one of the embodiments, a correction process corresponding to a to-be-corrected projection picture is simulated according to a correction parameter, including: A virtual projection geometric model is constructed according to discrete control points of the correction parameter, and the virtual projection geometric model is meshed by quadrilateral mesh units to obtain a meshed model; The discrete control points can be a set of coordinates defining key deformation nodes of the projection picture, such as four corner points or grid intersection points of the projection picture, which can be obtained through a preset lens offset or pixel mapping coordinate in the correction parameter. The quadrilateral mesh unit can be a grid structure that divides the projection picture into multiple quadrilateral regions, each grid unit being composed of four vertices. For example, the uniform grid division or adaptive grid generation algorithm can be used. The meshed model can be a digital expression of the continuous geometric model after being divided into discrete quadrilateral units, and the generation process is completed by a geometric modeling tool such as the mesh generation library of OpenGL or MATLAB. By dynamically associating the discrete control points with the vertex coordinates of the grid units, the uniformity or adaptability of the meshing can directly affect the subsequent calculation accuracy and efficiency. The technical operation realizes meshing through a geometric modeling tool. For example, in a projection picture with barrel distortion, the algorithm can adaptively increase the grid density in the distortion area to improve the local modeling accuracy, thereby providing a structured data basis for the calculation of the deformation field and the brightness field.

[0047] A pixel mapping equation is established using a bilinear interpolation function, and the deformation field data and the brightness field data of the meshed model are calculated by the pixel mapping equation and a ray tracing solver; The bilinear interpolation function can be an estimation method based on weighted average of two-dimensional grid vertex values, for example, for calculating the gray scale or color value of a non-integer coordinate pixel. The ray tracing solver can be a calculation tool simulating the propagation path of light from the projection device to the screen, which can be implemented by a path tracking algorithm, for example. The deformation field data can be a set of displacement amounts of the grid cell vertex coordinates, such as X / Y axis displacement amounts. The brightness field data can be distribution data recording the changes in light intensity of the grid cells, including the light attenuation or superposition effect caused by correction. The technical operation establishes a pixel coordinate mapping relationship through the bilinear interpolation function, for example, when the grid cell vertices are offset, the deformation mapping of the internal pixels is calculated by using the weighted vertex coordinates. In combination with the ray tracing solver, the algorithm can simulate the interaction of the light path and the grid model, and quantize the deformation field and brightness field data of each grid cell through numerical integration or ray tracing technology, so as to accurately predict the influence of the correction parameters on the geometric shape and brightness distribution.

[0048] The deformation field data and the brightness field data are feature extracted through a three-layer convolutional neural network to obtain a feature mapping relationship of the deformation field data and the brightness field data. The three-layer convolutional neural network can be a neural network architecture including an input layer, two hidden convolutional layers, and an output layer, which can be a lightweight network such as LeNet5, for example. The feature mapping relationship can be a correlation description between the deformation field and the brightness field learned by the neural network, for example, the coupling degree of brightness change and geometric deformation. The technical operation inputs the deformation field and brightness field data into the network, extracts local features using convolution kernels, for example, the first layer of convolution extracts the micro deformation mode of the grid cell, the second layer of convolution fuses the joint features of deformation and brightness, and the final output layer generates the feature mapping relationship. This process optimizes the network parameters through the back propagation algorithm, so that the model can capture the nonlinear correlation between deformation and brightness change, for example, the edge stretching caused by correction can be accompanied by local overexposure or underexposure, thereby providing a feature correlation model for subsequent real-time data processing.

[0049] The real-time image data captured by the camera is processed according to the feature mapping relationship to obtain coordinate offset data and brightness change data of the projection image to be corrected during the correction process. The real-time picture data can be current state information of a projection picture continuously collected by a camera, containing uncorrected distortion characteristics. The coordinate offset data can be a numerical set quantifying the coordinate difference of pixels or grid cells before and after geometric correction, and the brightness change data can be a numerical set describing the brightness gain or attenuation distribution caused by correction. The technical operation predicts the coordinate offset and brightness change in the current distortion scene by inputting the real-time picture into the trained feature mapping model using the forward propagation algorithm. For example, if there is a shadow in the lower left corner of the real-time picture display screen, the model can infer the brightness attenuation trend that may be caused by the correction parameters in this area in combination with the feature mapping relationship, and output adjustment suggestions to balance the geometric correction and brightness uniformity, thereby providing data support for subsequent fusion.

[0050] The actual distortion curve and the actual brightness distribution curve are obtained by fusing the coordinate offset data and the brightness change data through the particle filter. The particle filter can be a state estimation technique based on the Monte Carlo method, and the sequential Monte Carlo method can be used as an example. The actual distortion curve can be a quantitative description reflecting the degree of residual distortion of the corrected picture varying with position, and the actual brightness distribution curve can be a regular expression describing the variation of brightness after correction. The technical operation generates a large number of particles, each particle representing a possible combination of coordinate offset and brightness change, and calculates the particle weight according to the real-time data and the simulation data. For example, if the camera shows that the brightness of a certain area is abnormal and the simulation predicts that the distortion is reduced, the algorithm can fuse the two types of data through weighted average, and finally output the actual distortion and brightness distribution curve with higher confidence, thereby effectively reducing the noise interference of a single data source.

[0051] The reliability of the correction parameters is verified according to the actual distortion curve, the actual brightness distribution curve, the deformation field data and the brightness field data. The reliability verification can be to evaluate the balance effect of the correction parameters between geometric correction and brightness fidelity by comparing the simulation prediction and the actual measurement data. The technical operation compares the actual distortion curve output by the particle filter with the deformation field data, and if the difference is within the preset error range, it indicates that the geometric correction is effective; at the same time, the actual brightness distribution curve is compared with the brightness field data, and if the fluctuation meets the human eye comfort standard, it indicates that the color fidelity meets the standard. If the verification fails, the system can adjust the grid subdivision density or the feature extraction parameters, reiterate the simulation and verification process, for example, by increasing the grid density of the distortion area or optimizing the convolution kernel parameters of the CNN to improve the model accuracy.

[0052] The embodiment realizes high-precision deformation modeling by constructing a virtual projection geometric model and performing mesh partitioning, combines bilinear interpolation and ray tracing to calculate deformation field and brightness field data, extracts feature mapping relationship using a convolutional neural network and processes real-time picture data, generates actual distortion and brightness distribution curves by fusing multi-source data through a particle filter, and finally verifies the reliability of the correction parameters based on simulation and measurement data, so that the technical effects of improving the simulation accuracy of correction parameters in complex scenes, reducing data island effects, and supporting dynamic parameter optimization can be achieved. Through the cooperation of multiple technologies, the method solves the shortcomings of traditional methods in local distortion modeling, multi-modal data fusion, and real-time verification, and realizes the dynamic balance between geometric correction and brightness fidelity.

[0053] In one of the embodiments, the correction parameters are adjusted according to the correction process, and the to-be-corrected projection picture is corrected according to the adjusted correction parameters, including: When the correction parameters are verified to be reliable, a fuzzy evaluator based on a confidence interval is used to calculate deformation evaluation parameters and brightness evaluation parameters corresponding to the correction parameters, and a to-be-adjusted parameter sequence is generated according to the comparison results of the deformation evaluation parameters, the brightness evaluation parameters, and a preset threshold value.

[0054] The fuzzy evaluator based on the confidence interval can be an evaluation tool combining statistical confidence intervals and fuzzy logic, which is used to quantify the reliability of the parameters. The tool generates multiple possible values of the parameters through the Monte Carlo method, calculates the statistical distribution, and then reflects the uncertainty range of the parameter estimation. The deformation evaluation parameters can be indicators for quantifying the geometric correction effect, such as distortion residual standard deviation or edge alignment error, which are mapped to fuzzy sets such as high distortion through the statistical characteristics of the parameter confidence interval and the fuzzy membership function. The brightness evaluation parameters can be indicators for evaluating the brightness uniformity, such as the inter-region brightness difference coefficient or the proportion of overexposed / underexposed pixels, which are mapped to fuzzy sets such as low brightness through a similar method. The preset threshold value can be a correction target value set according to the visual comfort of the human eye, such as a maximum allowed distortion residual of 0.5 pixels or a brightness difference of no more than 10%. The technical operation can be realized through fuzzy inference rules, such as "if the brightness difference is high and the confidence is low, then the gain parameter is adjusted first". By comparing whether the lower limit of the confidence interval of the deformation evaluation parameters and the brightness evaluation parameters exceeds the threshold value, the to-be-adjusted parameter sequence is generated. This process prioritizes parameters that have the greatest impact on correction effect, such as lens focal length rather than edge fine-tuning, thereby determining the parameter adjustment order and weight.

[0055] The gradient boosting tree regressor is trained according to the to-be-adjusted parameter sequence, the parameter adjustment amount is predicted through the gradient boosting tree regressor, and the parameter value of the correction parameter is adjusted according to the parameter adjustment amount, so as to obtain the discrete control point coordinates of the updated correction parameters.

[0056] The gradient boosting tree regressor can be an ensemble learning model, such as a regression model implemented by the XGBoost or LightGBM framework, which iteratively stacks decision tree weak learners to gradually reduce prediction errors. The parameter adjustment amount can be an increment or a ratio to be applied to the original correction parameter, such as adjusting the lens focal length parameter from 5mm to 5.2mm. The discrete control point coordinates can be a set of coordinates in the correction parameter that define key deformation nodes of the projected image, such as the positions of grid vertices or control handles. The technical operation can be performed by constructing a training set containing historical correction data (such as distortion residual and brightness difference corresponding to different parameter values), and using a forward step-by-step algorithm to fit the residual between the current model and the target value. For example, if the current brightness difference is 15%, the model predicts that the brightness gain parameter needs to be reduced by 8%. The adjustment amount is updated by the parameter value through the backpropagation gradient descent method, and finally the updated discrete control point coordinates, such as the new coordinate positions of the grid vertices, are output. This process ensures the priority adjustment of key parameters (such as master control point coordinates) through model iteration and feature importance analysis.

[0057] The discrete control point coordinates of the updated correction parameter are fitted using a NURBS curve, and dynamic speed planning is performed based on the NURBS curve under the constraints of maximum pixel displacement and brightness change rate to obtain a speed planning curve.

[0058] The NURBS curve can be a non-uniform rational B-spline curve that accurately describes complex geometric shapes by controlling vertices, node vectors, and weight parameters. The maximum pixel displacement constraint can be a limit on the maximum movement distance of a single pixel during the correction process, such as not exceeding 3 pixels to prevent image tearing. The brightness change rate constraint can be a control of the brightness change slope caused by the correction, such as not exceeding 20% per second to avoid visual flicker. Dynamic speed planning can be a speed curve generated by parameter adjustment based on path constraints, such as reducing adjustment speed in sensitive areas to improve accuracy. The technical operation can be performed by fitting a smooth path using the discrete control point coordinates as the control vertices of the NURBS curve, combined with non-uniform node vectors. For example, if a certain area needs to move 5 pixels but is constrained by 3 pixels / second, a speed curve is generated by a quadratic programming algorithm in stages. This curve takes into account the motion capabilities of physical hardware to ensure that the adjustment process is reliably executed within the mechanical performance boundary.

[0059] The curvature values at each discrete point are calculated for the speed planning curve, transition points are inserted in the curvature mutation area through spline interpolation, the adjusted correction parameters are obtained, and the projected image to be corrected is corrected according to the adjusted correction parameters.

[0060] The curvature value can be a geometric quantity describing the degree of bending of a curve at a point, for example, in units of pixel-1. The spline interpolation can be a smooth transition curve generated by fitting discrete data points with a piecewise polynomial function, for example, cubic spline interpolation ensures the continuity of first and second derivatives. The transition point can be a control point added at the point of curvature mutation, for example, an intermediate control point is inserted at the peak of curvature to alleviate the dramatic change of the path. The technical operation can be to calculate the curvature of each point on the curve, if the curvature of a certain region exceeds the threshold value, such as 1 / pixel, then a transition point is generated near the mutation point by cubic spline interpolation. For example, if the NURBS curve indicates that a certain region needs to be moved up by 2 pixels, the driving system plans the curve step by step according to the speed, and finally sends the coordinate set containing the original control point and the transition point to the projection device to execute the correction.

[0061] The embodiment quantifies the reliability of the parameters and generates an adjustment sequence by using a fuzzy evaluator based on the confidence interval, combines a gradient boosting tree regressor to predict the parameter adjustment amount to optimize the discrete control point coordinates, uses NURBS curve and dynamic speed planning to ensure that the adjustment path is smooth and continuous under physical constraints, and finally eliminates path mutations through curvature analysis and spline interpolation, which can achieve the technical effects of significantly improving the parameter adjustment accuracy and execution stability. This method changes the parameter adjustment from extensive to precise and controllable through multi-level optimization technology, reduces visual residue and hardware loss in complex scenarios, shortens the convergence time, and realizes real-time correction with high precision and low energy consumption.

[0062] In one of the embodiments, before acquiring the image data of the projection picture to be corrected, the method further comprises: Obtaining the material parameters of the projection screen to construct a geometric distortion statics model corresponding to the projection screen and perform correction analysis on the geometric distortion statics model to obtain screen deformation cloud maps and brightness distribution cloud maps corresponding to the projection picture under each initial correction parameter. The material parameters of the projection screen can be a set of numerical values describing the physical properties of the screen, which can be obtained by sensor measurement or manual input. For example, the parameters include the bending stiffness of a flexible screen, the radius of curvature of a curved screen, or the surface roughness. The geometric distortion statics model can be a mathematical model based on material mechanics and structural analysis, which is used to quantify the influence of different correction parameters on screen deformation. For example, a meshed mechanics model can be established by finite element analysis (FEA). The screen deformation cloud map can be a two-dimensional heat map showing the displacement of different regions of a curved screen or the local concave-convex degree of a flat screen. The brightness distribution cloud map can be a distribution map describing the brightness gradient of the projection picture, which reflects the brightness attenuation characteristics of the edge region caused by material absorption or diffuse reflection.

[0063] A geometric distortion statics model is constructed and correction analysis is performed. Exemplarily, the influence of different initial correction parameters (such as lens offset or digital correction coefficients) on screen deformation and brightness can be simulated by inputting material parameters into the model. For example, a deformation cloud map is generated by calculating the stress distribution under a specific focal length adjustment through finite element simulation, or a brightness distribution cloud map is generated by simulating the reflection path through a ray tracing algorithm. A multi-parameter traversal process is then performed on each initial correction parameter set to generate the deformation and brightness cloud maps corresponding to the parameters. This process is completed offline and provides basic data for subsequent screening.

[0064] Discrete points in the screen deformation cloud map and edge gradients in the brightness distribution cloud map of each initial correction parameter are extracted respectively, and the correction speed and brightness change rate corresponding to each initial correction parameter at each discrete point are calculated to obtain the screen deformation variation curve in the correction process of the projection picture under different initial correction parameters. The discrete points can be key sampling points in the deformation cloud map for quantifying local deformation. Exemplarily, the discrete points are distributed at the screen edges, corners, or regions prone to distortion. The edge gradient can be the brightness change slope of the edge region in the brightness distribution cloud map. Exemplarily, the brightness difference of the edge pixels is obtained by a Sobel operator to detect the edge. The correction speed can be the variation rate of the screen deformation under a unit parameter adjustment. Exemplarily, the time derivative or parameter derivative of the deformation data is calculated. The brightness change rate can be the ratio of the edge gradient to the correction parameter adjustment. Exemplarily, the brightness attenuation sensitivity to parameter change is reflected. The deformation variation curve can be a visual model of the parameter deformation relationship. Exemplarily, the correction speed and deformation data under different initial correction parameters are fitted to generate the deformation variation curve.

[0065] The discrete points and edge gradients are extracted and dynamic parameters are calculated. Exemplarily, the deformation values of the pre-defined discrete points in the deformation cloud map are recorded, and the gradient values of the edge regions in the brightness cloud map are calculated. The dynamic parameter calculation combines the deformation and brightness data, for example, the correction speed and brightness change rate are obtained by derivative operation. The deformation variation curve is generated by fitting the deformation data of different initial parameters into a curve to form a visual model of the parameter and deformation relationship. For example, the curve peak value can indicate a correction critical point.

[0066] A plurality of first correction parameters are selected from the initial correction parameters according to the screen deformation variation curve. The first correction parameter can be a candidate parameter set that meets the deformation convergence speed or residual distortion rate indicators. The screening process exemplarily includes setting a convergence threshold of the deformation variable change curve, such as a deformation rate lower than 0.1 mm / s or a residual distortion less than 1%, and eliminating initial parameters that do not meet the conditions. Multi-objective optimization is achieved by Pareto frontier analysis or weight scoring method, which comprehensively considers indicators such as deformation variable and brightness change rate, to select parameters that balance deformation correction and brightness preservation. For example, a parameter that causes a sudden drop in edge brightness can be excluded.

[0067] According to a preset parameter optimization algorithm, the first parameter in each first correction parameter is optimized, and a plurality of optimized parameter combinations are obtained and saved to a preset correction parameter library.

[0068] The parameter optimization algorithm can be an algorithm for iteratively improving the performance of candidate parameters. Exemplarily, it includes particle swarm optimization (PSO), genetic algorithm or gradient descent method. The parameter combination can be a vector containing multi-dimensional correction parameters, exemplarily including focal length, trapezoidal correction angle or edge compensation coefficient. The correction parameter library can be a database for storing optimized parameter sets, exemplarily supporting the coverage or supplement of original parameters to improve adaptability.

[0069] The process of optimizing the parameter combination exemplarily includes dynamically adjusting the sub-parameters such as lens displacement or digital correction coefficient weight by the algorithm to minimize the deformation residual or maximize the brightness uniformity. For example, the genetic algorithm generates new parameter combinations through crossover and mutation, and verifies their performance through simulation. The generation of parameter combinations needs to ensure the synergy between sub-parameters, for example, adjusting the focal length needs to cooperate with the pixel mapping parameter to avoid edge distortion. The parameter library update stores the optimized parameter combinations in the database to form a self-evolving parameter library to adapt to new screen materials or complex scene requirements.

[0070] The embodiment obtains the material parameters of the projection screen and constructs a geometric distortion statics model to generate deformation and brightness cloud maps corresponding to different initial correction parameters; extracts key points and gradient data to calculate dynamic parameters and generate a deformation variable change curve; screens candidate parameters based on the curve and performs multi-objective optimization; finally generates parameter combinations through a parameter optimization algorithm and updates the correction parameter library, which can achieve the following technical effects: through the model construction driven by material characteristics, the correction schemes in the parameter library are deeply bound with the physical properties of the screen, reducing the correction deviation caused by material differences; through the screening of parameters such as deformation variable change curve and brightness gradient, the robustness and environmental adaptability of the parameter combination are improved; the pre-generated parameter library reduces the calculation load of real-time correction, enabling the subsequent steps to quickly respond to environmental changes; the dynamic parameter library updating mechanism supports the adaptability extension of new screen materials or complex scenes, improving the flexibility and efficiency of the overall scheme.

[0071] In one embodiment, the material parameters include a surface roughness parameter and a material refractive index, the material parameters of the projection screen are obtained, a geometric distortion statics model corresponding to the projection screen is constructed and corrected, and the screen deformation nephogram and the brightness distribution nephogram corresponding to the projection screen under each initial correction parameter are obtained. The surface roughness parameter and the material refractive index of the projection screen are obtained to calculate the distortion stiffness matrix corresponding to the projection screen, wherein the surface roughness parameter includes a microstructure depth and a distribution density, and the distortion stiffness matrix is determined by the material refractive index, the microstructure depth and the distribution density. The surface roughness parameter can be a quantitative index for describing the microstructure characteristics of the screen surface, and can be obtained by three-dimensional microscopic imaging or a white light interferometer. For example, the microstructure depth can be the vertical height of the fiber depression of the fabric screen or the depth of the etched groove of the metal screen, and the distribution density can be the number or area ratio of the microstructure per unit area. The material refractive index can be the relative change of the screen material to the light wave propagation speed, which can be obtained by a spectrophotometer. The distortion stiffness matrix can be a coupled mathematical expression of mechanical and optical characteristics, and its element value is determined by the above parameters, which is used to represent the deformation response characteristics of the screen under different correction parameters. In the technical operation, the refractive index, the microstructure depth and the distribution density are substituted into the mechanical-optical coupling model, and the distortion stiffness matrix reflecting the material characteristics is established through the parameter quantization and matrix construction process, such as increasing the local stiffness coefficient by increasing the microstructure depth, or adjusting the light path by using high refractive index material to affect the deformation distribution.

[0072] The pixel node distortion value is obtained by the distortion stiffness matrix and is mapped to the grid node to establish a bicubic spline interpolation equation, wherein the grid node is divided by a quadrilateral element and the grid size in the edge area is reduced to one third of the basic grid size. The pixel node distortion value can be the deformation displacement of the discrete points on the screen surface, which can be calculated by mechanical analysis of the stiffness matrix. The quadrilateral element division can be a discretization method for dividing the screen surface into rectangular grid elements, and the basic grid size can be set to 5mm, for example. The grid size in the edge area can be reduced to one third of the basic size, for example, the grid spacing in the edge area is adjusted to 1.67mm. The bicubic spline interpolation equation can be a high-order continuous curved surface fitting model based on a polynomial function, and its expression can be expressed as wherein represents the spatial position coordinates in the two-dimensional coordinate system of the projection screen surface, usually taking the lower left corner of the screen as the origin, x is the horizontal direction, and y is the vertical direction. represents the amount of deformation displacement at the screen surface point (x, y), i.e., the position offset of the point in the projection picture correction process caused by material characteristics, is the coefficient of the bicubic polynomial, a total of 16 (4x4), determined by least squares fitting of the distortion values of the discrete grid nodes, reflecting the curved shape of the screen surface deformation. In technical operations, the screen surface is discretized by grid division, the pixel node distortion value is mapped to the nearest grid node to form a discrete data set, and the equation coefficients are determined by least squares method, thereby establishing a smooth and continuous deformation field function. For example, a finer grid in the edge area can improve the local deformation simulation accuracy, and the bicubic interpolation can more accurately describe the distortion shape of the curved transition area than the linear interpolation.

[0073] The edge curvature value is calculated according to the bicubic spline interpolation equation to obtain the distortion field calculation equation, i.e., the geometric distortion statics model; wherein the edge curvature value is determined by the pixel gradient vector and the normal vector; The edge curvature value can be a bending degree parameter of the edge area of the screen surface, and its value can be obtained by solving the second derivative of the interpolation equation. The pixel gradient vector can be a vector representation of the direction and amplitude of the pixel gray change, and an example can be calculated by the Sobel operator. The normal vector can be a vertical direction vector of a point on the screen surface, and its direction is determined by the surface curvature. In technical operations, the curvature radius of the edge area is calculated by combining the angle relationship between the pixel gradient vector and the normal vector, for example, when the gradient vector is nearly perpendicular to the normal vector, it indicates that there is significant bending. Then the curvature value is substituted into the distortion field model to form a comprehensive equation containing deformation, curvature and optical characteristics, thereby establishing a geometric distortion statics model that can reflect the optical and mechanical characteristics of the edge area.

[0074] A set of nonlinear pixel displacement equations is established according to the distortion field calculation equation to obtain the screen deformation cloud picture and the brightness distribution cloud picture corresponding to each initial correction parameter of the projection picture; wherein the set of nonlinear pixel displacement equations is solved by using the quasi-Newton method until the residual error is less than the preset value.

[0075] The set of nonlinear pixel displacement equations can be a set of equations describing the spatial displacement relationship of the pixel points, which contains nonlinear terms to reflect material plastic deformation or large deformation effect. The quasi-Newton method can be an iterative optimization algorithm that accelerates the convergence process by approximating the Hessian matrix. In technical operations, the distortion field model is converted into a pixel-level displacement equation, for example, wherein, is the original coordinate of the pixel point in the projection picture, corresponding to the ideal position when the screen is not deformed, , is the actual coordinate of the pixel point after the screen deformation, reflecting the position offset caused by material mechanics deformation and optical characteristics, , is the displacement of the pixel in the x and y directions, is the relative change of the propagation speed of light in the screen material, is measured by a spectrophotometer, affects the light path and reflection / scattering characteristics, and further affects the calculation of the brightness distribution cloud, is the vertical height of the microstructure on the screen surface, such as the depth of the fabric fiber depression or the depth of the metal etching groove, which belongs to the surface roughness parameter, affects the local stiffness of the material, the greater the depth, the higher the local stiffness, and are the deformation functions in the x and y directions, respectively, are continuous deformation fields generated based on bicubic interpolation equations, describe the displacement distribution of the screen surface, and the equation set is solved by iterative methods such as the Broyden update formula, with a deformation error threshold (such as less than 0.01 pixels) as the termination condition. At the same time, the refractive index and the curvature value are combined to simulate the light reflection path, and the brightness distribution cloud is generated, for example, the high-curvature area causes the brightness to attenuate due to light scattering.

[0076] The embodiment quantifies the surface microstructure parameters and optical characteristics to construct a distortion stiffness matrix, uses adaptive grid division and high-order interpolation algorithms to establish a deformation field model, combines edge curvature analysis and nonlinear equation solving to generate high-precision deformation and brightness cloud maps, which can achieve the technical effects of improving the deformation prediction accuracy of flexible screens, optimizing the distortion compensation effect of edge regions, and reducing the complexity of large-scale parameter traversal calculations. The method solves the problems of insufficient simulation accuracy and low computational efficiency of traditional methods in complex deformation and optical coupling scenarios through the synergistic effect of micro-macro parameter coupling modeling, local high-density grid optimization, and high-efficiency numerical algorithms.

[0077] In one embodiment, the corresponding correction speed and brightness change rate of each initial correction parameter at each discrete point are calculated to obtain the screen deformation variation curve of the projection picture in the correction process under different initial correction parameters, including: obtain the parameter curve corresponding to each initial correction parameter, and obtain the coordinate data set of the discrete points on the parameter curve; wherein different initial correction parameters correspond to different curve shapes, curve lengths, and curve curvatures; A parameter curve is a mathematical function describing the mapping between a correction parameter and the screen response, whose geometric characteristics are determined by the physical properties of the initial correction parameter. For example, a focal length adjustment parameter can correspond to a parabolic curve, while a trapezoidal correction parameter can exhibit a linear or exponential curve. This curve can be constructed through experimental measurements or numerical simulations, for example by adjusting the lens displacement or digital correction coefficients, and recording the screen deformation or luminance response data at different parameter values. A discrete point dataset is a set of coordinates obtained by uniformly or non-uniformly sampling along the parameter curve, containing paired data of parameter values and corresponding response values. The difference in curves for different initial correction parameters reflects the nonlinear characteristics of their influence on the screen, for example, a high-curvature curve can indicate that the parameter has a higher sensitivity to deformation.

[0078] fitting the discrete points with a Bezier curve to obtain a luminance distribution value; wherein the luminance distribution value is calculated through a curve parameter and a luminance mapping relationship; A Bezier curve is a parametric curve fitting method that defines a smooth curve shape through control points, suitable for interpolating discrete data points and generating a continuous function. The luminance distribution value is a continuous function value calculated through the parameter of the fitted Bezier curve and the luminance mapping relationship, which may include optical parameters such as reflectivity and diffuse reflectance coefficient. For example, a cubic Bezier curve can fit a natural transition in luminance variation by adjusting the positions of four control points. The calculation of the luminance value needs to be combined with the parameters of the fitted curve and the optical model, for example, when the curve parameter represents the lens offset, the influence of the change in incident angle caused by the offset on the luminance distribution needs to be considered.

[0079] establishing an optical kinematics equation according to the luminance distribution value, and solving the luminance change rate value of the discrete points using Euler integral formula; wherein the luminance change rate value is calculated by integrating the luminance with respect to time; The optical kinematics equation is a dynamic model describing the change of luminance over time, containing parameters such as luminance gradient and diffusion coefficient, used to simulate the evolution of the luminance distribution during the correction process. For example, if the luminance change is affected by heat conduction, the equation can take the form of a heat equation. By substituting the luminance distribution value into the equation, a differential equation of the luminance change over time is established. The Euler integral formula is a numerical solution method that estimates the function value at the next time step using the derivative value at the current time step, for example, if the current luminance gradient is ΔL / Δt, then the luminance value at the next time step L(t+Δt)=L(t)+ΔL. This process converts the static luminance distribution into a dynamic change rate, providing a time dimension parameter for deformation analysis.

[0080] constructing a state equation according to the luminance change rate value of the discrete points and the deformation amount on the screen deformation cloud map, and solving a deformation time function through least squares method to obtain a screen deformation amount change curve according to the deformation time function.

[0081] The state equation is a mathematical equation set of the joint theory brightness change rate and the deformation variable, which describes the coupling relationship between the two in the physical process. For example, the deformation rate can be proportional to the brightness gradient. The least squares method solves the parameters of the deformation time function by minimizing the sum of squares of errors between the theoretical prediction value and the actual measured value. The final generated deformation time function is visualized to form a screen deformation variable change curve, which shows the evolution law of the deformation variable with time under different initial correction parameters, such as exponential decay or oscillation convergence trend.

[0082] The embodiment obtains the geometric characteristics of the parameter curve and samples discrete point data, generates a continuous brightness distribution model by combining the Bezier curve fitting, quantifies the brightness dynamic change rate by using the optical kinematics equation and the Euler integral, further couples the deformation and the brightness parameters by using the state equation, and optimizes the deformation time function by using the least squares method. Finally, the deformation variable change curve is generated. This method combines the depth of mathematical modeling and numerical calculation, converts the discrete and static parameter analysis into continuous and dynamic multi-physical field coupling modeling, significantly improves the generation accuracy of the deformation variable change curve and the scientificity of the parameter screening, and enhances the quality of the correction parameter library by eliminating the interpolation error of discrete data, quantifying the dynamic response characteristics, integrating the multi-field coupling effect and optimizing the solving efficiency. It provides a reliable prediction model for real-time correction in complex dynamic scenarios.

[0083] In one of the embodiments, a plurality of first correction parameters are selected from a plurality of initial correction parameters according to the screen deformation variable change curve, comprising: The screen current deformation variable corresponding to each initial correction parameter is determined by the screen deformation variable change curve, and a constraint equation is constructed according to the preset maximum deformation variable of the projection picture and the screen current deformation variable corresponding to each initial correction parameter, so as to obtain the deformation variable constraint optimization parameter by conjugate gradient iteration calculation; The screen deformation variable change curve can be a quantitative relationship model describing the degree of screen physical deformation over time or parameter adjustment amplitude under the action of different initial correction parameters, can be generated by collecting screen surface displacement data through a machine vision sensor and data fitting, and exemplarily, the curve can include deformation displacement time series data obtained based on strain gauge measurement or optical tracking technology. The preset maximum deformation variable can be a deformation allowable upper limit value set by a projection device manufacturer according to screen material characteristics and service life requirements, and exemplarily, the value can include a safe deformation variable threshold of a flexible screen under high-frequency vibration. The constraint equation can be a mathematical expression comparing the current deformation variable with the maximum allowable value, for example, deformation variable ≤ preset maximum deformation variable, and the conjugate gradient iteration can be a numerical optimization method for gradually approaching the optimal solution by constructing a conjugate direction sequence, which is different from the traditional gradient descent method, and accelerates the convergence process by reducing the correlation of the search direction. In one specific embodiment, when the current deformation variable corresponding to a certain initial correction parameter exceeds the threshold value, the conjugate gradient iteration adjusts the parameter value in the opposite direction until the deformation variable converges within the constraint range, thereby screening out the candidate parameters that meet the physical deformation limit.

[0084] A multi-objective optimization function of the deformation variable target function and the brightness target function is established according to the deformation variable constraint optimization parameter, and a mapping relationship between the parameter vector and the constraint response vector corresponding to each initial correction parameter is obtained through a support vector machine of a radial basis kernel function; The deformation variable target function can be a mathematical expression quantifying the degree of screen residual deformation, for example, a minimization function defined as the sum of squares of the deformation variables, and the brightness target function can be an index representing the uniformity of the projection picture, for example, a minimization function of the standard deviation of the edge gradient. The multi-objective optimization function can be a comprehensive optimization target obtained by combining the above two target functions through weighting coefficients, and exemplarily, the weight coefficients can be dynamically adjusted according to application scene requirements. The radial basis kernel function can be a nonlinear mapping tool in the form of a Gaussian function, used to map the complex relationship between the initial correction parameter vector and the constraint response vector to a high-dimensional feature space. The support vector machine can be a supervised learning model that maximizes the classification interval or minimizes the regression error, which can effectively capture the nonlinear relationship between the parameters and the responses, for example, the nonlinear response relationship between the material deformation characteristics and the correction parameters in the flexible screen scenario.

[0085] A constraint optimization equation set is constructed by using the KKT condition according to the mapping relationship, and the optimal correction parameter and the Lagrange multiplier corresponding to each initial correction parameter are obtained by solving through a sequential quadratic programming method. The KKT condition can be a necessary condition to ensure that the optimization solution meets the feasibility and optimality at the same time, including the satisfaction requirements of inequality constraints such as deformation variable constraints ≤ maximum allowed value, brightness gradient constraints ≥ minimum threshold value, etc. The constraint optimization equation set can be a set of equations that converts a multi-objective optimization problem into a set of equations containing a target function, constraint conditions and Lagrange multipliers, for example, the deformation constraint and the brightness constraint are simultaneously included in the equation system as hard constraints. The sequential quadratic programming method can be an iterative solution method by decomposing the original optimization problem into a series of quadratic programming sub-problems, each sub-problem contains a linearized target function and constraint condition, for example, this method can gradually approach the global optimal solution by alternately optimizing the parameter vector and the Lagrange multiplier. In one specific embodiment, when the deformation constraint and the brightness constraint conflict, the sequential quadratic programming method can achieve dynamic balance between multiple objectives by adjusting the weight coefficient or priority parameter.

[0086] If the norm difference of the initial parameter corresponding to the optimal correction parameter and the initial correction parameter is less than the preset convergence threshold, the optimal correction parameter is stored as the first correction parameter.

[0087] The norm difference can be a mathematical index for measuring the difference between the optimal correction parameter and the initial parameter, for example, the difference between the two vectors is calculated by using the Euclidean distance or L2 norm. The preset convergence threshold can be a termination condition set according to the parameter adjustment accuracy and the calculation efficiency, for example, the threshold can include numerical parameters such as 0.01. The convergence judgment condition can effectively avoid infinite iteration of the algorithm in the local optimal solution or the oscillation state, and ensure that the screening process is terminated and a stable solution is output under limited computing resources. When the difference is less than the threshold, it indicates that the parameter adjustment has converged to a stable state that meets the constraints and is close to the initial parameter, and at this time, storing the optimal parameter as the first correction parameter can ensure the real-time and reliability of the subsequent correction process.

[0088] In this embodiment, real-time deformation data is extracted through the screen deformation variable change curve and constraint equations are constructed, and parameter convergence under constraint conditions is realized by conjugate gradient iteration; the contradiction between deformation correction and picture quality is balanced by using multi-objective optimization function and nonlinear mapping modeling of support vector machine; the multi-objective problem is converted into a solvable constraint optimization equation set by using KKT condition and sequential quadratic programming method, and the iteration process is terminated by judging the norm difference; finally, a complete closed loop from initial parameter screening to optimal parameter storage is formed. This method deeply integrates mathematical optimization algorithm and machine learning model, which not only ensures the feasibility of the correction parameter under the physical deformation constraint, but also improves the adaptability of parameter selection to nonlinear scenes, and reduces the calculation redundancy by using the convergence condition control, thereby significantly improving the quality and environmental adaptability of the correction parameter library and reducing the risk of correction failure caused by improper parameter selection.

[0089] In addition, the embodiment of the present application also provides a storage medium, wherein the storage medium stores a machine vision-based picture geometric distortion intelligent control program, and the machine vision-based picture geometric distortion intelligent control program is executed by a processor to realize the steps of the machine vision-based picture geometric distortion intelligent control method.

[0090] In addition, with reference to Figure 3 , the embodiment of the present application also provides a machine vision-based picture geometric distortion intelligent control system, which comprises: a data acquisition module 10, configured to acquire image data of a to-be-corrected projection picture, and perform feature analysis on the image data to obtain a distortion feature value; a model matching module 20, configured to match a correction optimization model corresponding to the distortion feature value, so as to call a correction parameter from a preset correction parameter library through the correction optimization model; a correction simulation module 30, configured to simulate a correction process corresponding to the to-be-corrected projection picture according to the correction parameter; a parameter adjustment module 40, configured to adjust the correction parameter according to the correction process, and correct the to-be-corrected projection picture according to the adjusted correction parameter.

[0091] Other embodiments or specific implementation manners of the machine vision-based picture geometric distortion intelligent control system can refer to the above-mentioned method embodiments, which will not be described here.

[0092] It should be noted that, in this document, the terms “comprising”, “containing” or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement “comprising a” does not exclude the presence of another identical element in the process, method, article or device including the element.

[0093] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. In the system module claims in which several systems are enumerated, several of these systems can be embodied by the same hardware item. The use of the words first, second, and third does not represent any order, and these words can be interpreted as names.

[0094] Those skilled in the art can clearly understand the above-mentioned example method can be realized by means of software and the necessary general hardware platform, of course, also can be through hardware, but in many cases the former is a better implementation. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art contribution can be embodied in the form of software products, the computer software product is stored in a storage medium (such as read only memory image (Read Only Memory image, ROM) / random access memory (Random Access Memory, RAM), disk, optical disk), including a number of instructions to make a terminal user equipment (may be a mobile phone, computer, server, air conditioner, or network user equipment, etc.) executes the method described in various embodiments of the present application.

[0095] The above is only the preferred embodiment of the present application, not therefore limit the patent scope of the present application, any equivalent structure or equivalent flow transformation using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A machine vision-based intelligent control method for image geometric distortion, characterized in that, The method includes: Acquire image data of the projected image to be corrected, and perform feature analysis on the image data to obtain distortion feature values; Match the correction optimization model corresponding to the distortion feature value, so as to call the correction parameters from the preset correction parameter library through the correction optimization model; Simulate the correction process corresponding to the projected image to be corrected based on the correction parameters; The correction parameters are adjusted according to the correction process, and the projected image to be corrected is corrected according to the adjusted correction parameters.

2. The intelligent control method for image geometric distortion based on machine vision as described in claim 1, characterized in that, The image data includes edge feature parameters. The process of acquiring image data of the projected image to be corrected and performing feature analysis on the image data to obtain distortion feature values ​​includes: Obtain the edge feature parameters of the projection image to be corrected, wherein the edge feature parameters include edge curvature parameters and pixel offset parameters; The Hough transform is used to extract the coordinates of key points from the edge feature parameters, and the projection plane equation and distortion mapping equation are constructed based on the coordinates of the key points to obtain the distortion coefficient parameters. The distortion vector is obtained based on the distortion coefficient parameters, and the distortion vector is normalized to obtain the distortion feature value, wherein the distortion vector includes curvature, offset and rotation angle.

3. The intelligent control method for image geometric distortion based on machine vision as described in claim 1, characterized in that, The step of calling correction parameters from a preset correction parameter library through the correction optimization model includes: Obtain the parameter features corresponding to each parameter combination in the correction parameter library, and calculate the similarity between each parameter feature and the distortion feature value through the mapping relationship preset in the correction optimization model; When the similarity exceeds a preset threshold, a preset correction and optimization rule is invoked to optimize the current parameter combination and generate the correction parameters corresponding to the optimized parameter combination.

4. The intelligent control method for image geometric distortion based on machine vision as described in claim 1, characterized in that, The step of simulating the correction process corresponding to the projected image to be corrected based on the correction parameters includes: A virtual projection geometric model is constructed based on the discrete control points of the correction parameters, and the virtual projection geometric model is meshed using quadrilateral mesh elements to obtain a meshed model. A pixel mapping equation is established using a bilinear interpolation function, and the deformation field data and brightness field data of the mesh subdivision model are calculated using the pixel mapping equation and a ray tracing solver. The feature mapping relationship between the deformation field data and the brightness field data is obtained by extracting features from the deformation field data and the brightness field data through a three-layer convolutional neural network. The real-time image data captured by the camera is processed according to the feature mapping relationship to obtain the coordinate offset data and brightness change data of the projected image to be corrected during the correction process. By fusing the coordinate offset data and the brightness change data using a particle filter, the actual distortion curve and the actual brightness distribution curve are obtained. The reliability of the correction parameters is verified based on the actual distortion curve, the actual brightness distribution curve, the deformation field data, and the brightness field data.

5. The intelligent control method for image geometric distortion based on machine vision as described in claim 4, characterized in that, The step of adjusting the correction parameters according to the correction process and correcting the projected image to be corrected according to the adjusted correction parameters includes: When the reliability of the correction parameters is verified, a fuzzy evaluator based on confidence intervals is used to calculate the deformation evaluation parameters and brightness evaluation parameters corresponding to the correction parameters, and a sequence of parameters to be adjusted is generated based on the comparison results of the deformation evaluation parameters and the brightness evaluation parameters with preset thresholds respectively. A gradient boosting tree regressor is trained based on the sequence of parameters to be adjusted. The parameter adjustment amount is predicted by the gradient boosting tree regressor. The parameter value of the correction parameter is adjusted according to the parameter adjustment amount to obtain the discrete control point coordinates of the updated correction parameter. The coordinates of the discrete control points of the updated correction parameters are fitted with NURBS curves, and dynamic speed planning is performed based on the NURBS curves under the constraints of maximum pixel displacement and brightness change rate to obtain the speed planning curve. The curvature values ​​at each discrete point of the velocity planning curve are calculated. Transition points are inserted in the curvature abrupt change region by spline interpolation to obtain the adjusted correction parameters. The projected image to be corrected is then corrected according to the adjusted correction parameters.

6. The intelligent control method for image geometric distortion based on machine vision as described in claim 1, characterized in that, Before acquiring the image data of the projected image to be corrected, the process also includes: The material parameters of the projection screen are obtained to construct a geometric distortion static model corresponding to the projection screen and to perform correction analysis on the geometric distortion static model to obtain the screen deformation cloud map and brightness distribution cloud map corresponding to the projection image under each preset initial correction parameter. Extract the discrete points of each initial correction parameter in the screen deformation cloud map and the edge gradient in the brightness distribution cloud map, and calculate the correction speed and brightness change rate corresponding to each initial correction parameter at each discrete point to obtain the screen deformation change curve of the projected image during the correction process under different initial correction parameters. Based on the screen deformation change curve, select multiple first correction parameters from a plurality of initial correction parameters; The first parameter in each of the first correction parameters is optimized according to the preset parameter optimization algorithm to obtain several optimized parameter combinations and save them to the preset correction parameter library.

7. The intelligent control method for image geometric distortion based on machine vision as described in claim 6, characterized in that, The material parameters include surface roughness parameters and material refractive index. The process of obtaining the material parameters of the projection screen, constructing a geometric distortion static model corresponding to the projection screen, and performing correction analysis on the geometric distortion static model to obtain the screen deformation cloud map and brightness distribution cloud map corresponding to the projected image under each preset initial correction parameter includes: The surface roughness parameters and material refractive index of the projection screen are obtained to calculate the distortion stiffness matrix corresponding to the projection screen. The surface roughness parameters include microstructure depth and distribution density, and the distortion stiffness matrix is ​​determined by the material refractive index, microstructure depth and distribution density. The distortion values ​​of pixel nodes are obtained through the distortion stiffness matrix and mapped to the mesh nodes to establish a bicubic spline interpolation equation. The mesh nodes are divided into quadrilateral elements and the mesh size is reduced to one-third of the basic mesh size in the edge region. The edge curvature value is calculated based on the bicubic spline interpolation equation to obtain the distortion field calculation equation, i.e., the geometric distortion static model; wherein, the edge curvature value is determined by the pixel gradient vector and the normal vector; A set of nonlinear pixel displacement equations is established based on the distortion field calculation equations to obtain the screen deformation cloud map and brightness distribution cloud map corresponding to each preset initial correction parameter of the projected image; wherein, the set of nonlinear pixel displacement equations is solved using the quasi-Newton method until the residual is less than the preset value.

8. The intelligent control method for image geometric distortion based on machine vision as described in claim 6, characterized in that, The calculation of the correction speed and brightness change rate corresponding to each of the initial correction parameters at each discrete point, to obtain the screen deformation change curve of the projected image during the correction process under different initial correction parameters, includes: Obtain the parameter curve corresponding to each of the initial correction parameters, and obtain the coordinate dataset of discrete points on the parameter curve; wherein, different initial correction parameters correspond to different curve shapes, curve lengths and curve curvatures; The brightness distribution value is obtained by fitting the discrete points with a Bézier curve; wherein, the brightness distribution value is calculated through the mapping relationship between curve parameters and brightness. An optical kinematic equation is established based on the brightness distribution value, and the brightness change rate at discrete points is obtained by solving the Euler integral formula; wherein the brightness change rate is calculated by integrating brightness over time. A state equation is constructed based on the brightness change rate of the discrete points and the deformation on the screen deformation cloud map, and the deformation time function is obtained by solving it using the least squares method. The screen deformation change curve is obtained based on the deformation time function.

9. The intelligent control method for image geometric distortion based on machine vision as described in claim 6, characterized in that, The step of selecting multiple first correction parameters from a plurality of initial correction parameters based on the screen deformation change curve includes: The current screen deformation corresponding to each initial correction parameter is determined by the screen deformation change curve. Constraint equations are constructed based on the preset maximum deformation of the projected image and the current screen deformation corresponding to each initial correction parameter, so as to obtain the deformation constraint optimization parameters through conjugate gradient iteration calculation. Based on the deformation constraint optimization parameters, a multi-objective optimization function is established for the deformation objective function and the brightness objective function, and the mapping relationship between the parameter vector and the constraint response vector corresponding to each initial correction parameter is obtained through a support vector machine with radial basis function kernel. Based on the mapping relationship, a set of constrained optimization equations is constructed using KKT conditions, and the optimal correction parameters and Lagrange multipliers corresponding to each initial correction parameter are obtained by solving the sequential quadratic programming method. If the norm difference between the optimal correction parameter and the initial parameter corresponding to the initial correction parameter is less than a preset convergence threshold, then the optimal correction parameter is stored as the first correction parameter.

10. An image projector, characterized in that, The image projector includes: a memory, a processor, and a machine vision-based intelligent control program for image geometric distortion stored in the memory and executable on the processor, wherein the machine vision-based intelligent control program for image geometric distortion is configured to implement the steps of the machine vision-based intelligent control method for image geometric distortion as described in any one of claims 1 to 9.

Images