High-resolution image calculation and display integrated method and system based on photoelectric calculation hybrid architecture

Abstract

The invention discloses a high-resolution image calculation and display integrated method and system based on a photoelectric calculation hybrid architecture, and belongs to the technical field of image data processing, and the method comprises the steps: carrying out the combined modeling of the time domain gray scale response characteristics of a display panel and a space pixel structure, and constructing a display end prior template; an optical modulation mask is generated based on a template, physical modulation and coding sampling are carried out on an incident light field, a photoelectric mapping relation is utilized to establish a cross-domain correlation tensor as regularization constraint to reconstruct to-be-displayed image data adaptive to panel characteristics, and a display end prior template is combined to carry out consistency residual calculation to generate driving compensation correction. And finally synthesizing an optimized display driving sequence. A display end prior template is combined with optical coding to construct a cross-domain association tensor, and driving data is reconstructed by regularization constraint, so that compensation and optimization of space-time coupling characteristics of a display panel can be realized.

Description

Technical Field

[0002] This invention relates to the field of image data processing technology, and in particular to a high-resolution image processing and display method and system based on a hybrid optoelectronic architecture. Background Technology

[0004] High-resolution image display technology has core application value in consumer electronics, professional surveillance and virtual reality. Its final presentation quality depends not only on the original image data, but also on the physical characteristics of the display panel itself, including the temporal response speed of pixel units and the spatial structural relationship between pixels.

[0005] To improve display quality, existing technologies typically employ independent image processing algorithms or drive compensation techniques. For example, overdrive technology accelerates the flipping of liquid crystal molecules by querying a preset overshoot voltage meter to alleviate motion blur. Another common approach is to use motion estimation and compensation algorithms based on image content analysis to preprocess the input image sequence to predict and correct display lag.

[0006] However, existing technologies mostly focus on processing electrical signals or digital images, failing to establish a precise mapping model between the optoelectronic physical properties of the display panel and the final imaging effect. Existing methods typically treat temporal response and spatial structure as independent factors, ignoring their coupling effect in actual light output. This leads to discrepancies between the compensation model and the actual behavior of the panel, resulting in defects such as ghosting and edge blurring in high-speed dynamic scenes or fine texture displays. Summary of the Invention

[0008] To address the aforementioned issues, this invention provides a high-resolution image computing and display integrated method and system based on a hybrid optical-electronic computing architecture. It employs a priori templates at the display end combined with optical coding to construct a cross-domain correlation tensor, and reconstructs driving data with regularization constraints, thereby enabling compensation and optimization of the spatiotemporal coupling characteristics of the display panel.

[0009] The above objectives can be achieved through the following approach:

[0010] A high-resolution image computing and display integration method based on a hybrid optoelectronic computing architecture includes: jointly modeling the temporal grayscale response characteristics and spatial pixel structure of the display panel to construct a priori template for the display end; constructing a corresponding optical coding metric matrix based on the priori template, and generating an optical modulation mask loaded onto an optical wavefront modulator according to the optical coding metric matrix; controlling the optical wavefront modulator to physically modulate the incident light field through the optical modulation mask, and projecting the modulated incident light field onto a preset imaging sensor array to obtain optically coded coupled sampling data; establishing a cross-domain correlation tensor using the optoelectronic mapping relationship between the priori template for the display end and the optical modulation mask; solving the coupled sampling data using the cross-domain correlation tensor as a regularization constraint to reconstruct image data to be displayed that is adapted to the characteristics of the display panel; calculating the consistency residual by combining the image data to be displayed and the priori template for the display end to generate a driving compensation correction amount; and superimposing the driving compensation correction amount onto the image data to be displayed to generate a display driving sequence optimized for the physical characteristics of the display panel.

[0011] Optionally, the step of jointly modeling the temporal grayscale response characteristics and spatial pixel structure of the display panel to construct a priori template for the display end includes: obtaining the time-domain sequence of light intensity output of the display panel under different driving signals, and measuring the brightness establishment and decay process data of the pixel unit; analyzing the brightness establishment and decay process data of the pixel unit, and extracting dynamic response feature parameters characterizing the response delay and overshoot characteristics of the display panel; obtaining the spatial pixel arrangement geometric parameters of the display panel, and combining the dynamic response feature parameters to construct a priori template for the display end containing spatiotemporal coupling characteristics.

[0012] Optionally, the step of constructing a corresponding optical coding metric matrix based on the prior template of the display end, and generating an optical modulation mask loaded on the optical wavefront modulator according to the optical coding metric matrix includes: determining the spatial frequency modulation range and modulation depth constraints required for optical coding based on the spatiotemporal coupling characteristics of the prior template of the display end; constructing an optical coding metric matrix for modulating the incident light field in a specific mode according to the spatial frequency modulation range and modulation depth constraints; quantizing the optical coding metric matrix into phase or amplitude distribution data that conforms to the physical modulation range of the optical wavefront modulator, and generating an optical modulation mask.

[0013] Optionally, acquiring the optically encoded coupled sampling data includes: loading the optical modulation mask onto the control port of the optical wavefront modulator to form a physical modulation pattern; guiding the incident light field from the target scene or light source through the optical wavefront modulator loaded with the physical modulation pattern to generate a modulated light field with encoded information; and guiding the modulated light field to the photosensitive plane of a preset imaging sensor array for integration sampling to acquire coupled sampling data.

[0014] Optionally, establishing a cross-domain correlation tensor using the photoelectric mapping relationship between the display end prior template and the optical modulation mask includes: establishing a linear projection relationship from the actual display light output domain to the image sensing domain based on the optical modulation mask to obtain a first mapping relationship; establishing a temporal transformation relationship from the desired display image domain to the image sensing domain based on the display end prior template to obtain a second mapping relationship; and forming a cross-domain correlation tensor by concatenating the first mapping relationship and the second mapping relationship.

[0015] Optionally, the step of using the cross-domain correlation tensor as a regularization constraint to solve the coupled sampling data and reconstruct the image data to be displayed that is adapted to the characteristics of the display panel includes: constructing a target optimization function with the difference between the reconstructed image and the coupled sampling data in the image sensing domain as the fidelity term and the smoothness of the display effect predicted by the cross-domain correlation tensor of the reconstructed image as the regularization term; solving the target optimization function using a preset iterative optimization algorithm to update the reconstructed image; and outputting the image data to be displayed when the target optimization function converges to a preset threshold or reaches a preset number of iterations.

[0016] Optionally, the output of the image data to be displayed includes: acquiring optical sensing data of the external environment of the display terminal; analyzing the optical sensing data to extract environmental interference features characterizing changes in ambient light intensity and color temperature; using the environmental interference features as dynamic regularization constraints, and outputting the image data to be displayed when the objective optimization function converges to a preset threshold or reaches a preset number of iterations.

[0017] Optionally, the step of combining the image data to be displayed with the prior template of the display end to perform consistency residual calculation and generate the drive compensation correction amount includes: using the image data to be displayed and the prior template of the display end to simulate the actual light output sequence of the display panel; calculating the difference between the ideal display light output corresponding to the image data to be displayed and the actual light output sequence of the display panel to generate the original residual signal; and performing time-domain filtering and amplitude scaling on the original residual signal to generate the drive compensation correction amount.

[0018] Optionally, the step of superimposing the drive compensation correction amount onto the image data to be displayed to generate a display drive sequence optimized for the physical characteristics of the display panel includes: performing calculations on the drive compensation correction amount and the corresponding image data to be displayed to obtain pre-compensated drive data; performing digital-to-analog conversion and level matching on the pre-compensated drive data to generate an analog voltage or current sequence; organizing the analog voltage or current sequence according to the horizontal and vertical timing requirements of the display panel, and outputting the display drive sequence.

[0019] Based on the same inventive concept, this invention also provides a high-resolution image computing and display integrated system based on a hybrid optoelectronic computing architecture. The system includes: a display response modeling module, used to jointly model the temporal grayscale response characteristics and spatial pixel structure of the display panel to construct a priori template for the display end; an optical encoding module, used to construct a corresponding optical encoding metric matrix based on the priori template for the display end, and generate an optical modulation mask loaded onto an optical wavefront modulator according to the optical encoding metric matrix; and an imaging acquisition module, used to control the optical wavefront modulator to physically modulate the incident light field through the optical modulation mask, and project the modulated incident light field onto a preset imaging sensor array to acquire the image. The system comprises: a coupled sampling data module for optical encoding; an association mapping module for establishing a cross-domain association tensor using the photoelectric mapping relationship between the display end prior template and the optical modulation mask; a constraint solving calculation module for solving the coupled sampling data using the cross-domain association tensor as a regularization constraint to reconstruct the image data to be displayed that is adapted to the characteristics of the display panel; a consistency evaluation module for calculating the consistency residual by combining the image data to be displayed with the display end prior template to generate a driving compensation correction amount; and a display driving module for superimposing the driving compensation correction amount onto the image data to be displayed to generate a display driving sequence optimized for the physical characteristics of the display panel.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] This invention constructs a priori template for the display end by jointly modeling the temporal grayscale response characteristics and spatial pixel structure of the display panel, and designs an optical coding and computational reconstruction process based on this template. This can accurately compensate for the inherent dynamic response delay and spatial crosstalk of the display panel from the physical source, thereby improving the display clarity and contour fidelity of high-speed moving images.

[0022] This invention utilizes an optical wavefront modulation device to physically encode the incident light field, and combines cross-domain correlation tensors to correlate display characteristics with imaging sensing domain data. The image is reconstructed by solving regularization constraints, which enables the simultaneous perception of the spatiotemporal characteristics of the display panel and target scene information in a single sampling, thereby improving the system's accuracy and efficiency in correcting display defects.

[0023] This invention generates a drive compensation correction amount by calculating the consistency residual and feeds it back to the drive sequence to form a closed-loop optimization. This can eliminate the drift of display characteristics caused by panel aging and changes in ambient temperature, thereby ensuring that the display system maintains stable and consistent image quality output throughout its entire life cycle.

[0024] This invention deeply integrates optical modulation, imaging sensing, and computational reconstruction into the same architecture. The resulting display driving sequence is deeply adapted to the physical characteristics of the panel, which can suppress motion artifacts, improve grayscale transition smoothness, and improve color reproduction accuracy without modifying the display panel hardware.

[0025] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating the high-resolution image computing and display integration method based on a photoelectric computing hybrid architecture according to an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of the spatial pixel structure according to an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the physical process of optical encoding and imaging acquisition in an embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of the structure of a high-resolution image computing and display integrated system based on a photoelectric computing hybrid architecture according to an embodiment of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Reference Figure 1One embodiment of the present invention proposes a high-resolution image computing and display integration method based on a hybrid optical-electric computing architecture. It uses a priori template of the display end and optical coding to construct a cross-domain correlation tensor, and reconstructs the driving data with regularization constraints, which can achieve compensation and optimization of the spatiotemporal coupling characteristics of the display panel.

[0035] The method described in this embodiment specifically includes:

[0036] S1. Jointly model the temporal grayscale response characteristics and spatial pixel structure of the display panel to construct a priori template for the display end;

[0037] In one embodiment of the present invention, step S1 includes the following steps:

[0038] The time-domain sequence of light intensity output of the display panel under different driving signals is obtained, and the brightness establishment and decay process data of the pixel unit are measured.

[0039] Analyze the brightness build-up and decay process data of pixel units to extract dynamic response characteristic parameters that characterize the response delay and overshoot characteristics of the display panel;

[0040] Obtain the spatial pixel arrangement geometric parameters of the display panel, and combine them with the dynamic response feature parameters to construct a priori template for the display end that includes spatiotemporal coupling characteristics.

[0041] Specifically, step S1 first acquires the time-domain sequence of light intensity output from the display panel under different driving signals. This process is completed by measuring the light intensity at the center point of a single pixel unit on the display panel using a photodetector. The driving signal is a standardized digital driving value step change signal, for example, a sudden change from 0 to 255. The resulting continuous waveform of light intensity changing over time is measured. The measurement is based on actual measurements using 200 sets of industrial sensors at a sampling frequency of 10 kHz to ensure complete capture of the brightness build-up and decay process data of the pixel unit. This data is a discrete sequence of light intensity changing over time.

[0042] This study analyzes the brightness build-up and decay process data of pixel units to extract dynamic response characteristic parameters that characterize the response delay and overshoot properties of the display panel. The brightness build-up process is fitted using a first-order exponential rising model, with the following formula:

[0043] ,

[0044] in This represents the measured brightness value at time t, expressed in candela per square meter. This represents the steady-state maximum brightness value. Let be the brightness rise time constant, in seconds. The brightness decay process is fitted using a first-order exponential descent model, with the following formula:

[0045] ,

[0046] in This represents the measured brightness value at time t. To attenuate the initial brightness value, This represents the brightness decay time constant. If the measured data exhibits overshoot, i.e., the brightness momentarily exceeds the steady-state value and then drops back, then the overshoot ratio is introduced. This parameter is characterized and is defined as the overshoot peak brightness. With steady-state brightness The formula is: Subtract 1 from the ratio.

[0047] ,

[0048] Overshoot ratio It is a dimensionless parameter. Therefore, the set of dynamic response characteristic parameters includes the brightness rise time constant. Brightness decay time constant and overshoot ratio .

[0049] Finally, obtain the spatial pixel arrangement geometry parameters of the display panel, such as... Figure 2 As shown, these parameters include the pixel lateral period. Pixel vertical period Pixel effective light-emitting area width and height All units are in micrometers. A priori template for the display terminal, incorporating spatiotemporal coupling characteristics, is constructed. The template is a three-dimensional data structure. Its two-dimensional spatial layers generate a pixel mesh matrix based on geometric parameters. Each element in the matrix represents the duty cycle of a pixel at its corresponding spatial location. At the time dimension, a unit step response vector is generated based on the dynamic response feature parameters. This vector calculates the normalized luminance response values ​​at different times by combining a luminance establishment and attenuation model. (Display prior template) By calculating the grid matrix With response vector The outer product is obtained as follows:

[0050] ,

[0051] in This indicates an outer product operation. This template quantifies the light output characteristics of any pixel on the display panel under any timing drive.

[0052] For example, data on the brightness build-up and decay process of a pixel unit in an organic light-emitting diode display panel as the driving value jumps from 0 to 255 are obtained, and the steady-state maximum brightness is measured. The brightness rise time constant was obtained by fitting a value of 500 candela per square meter. The brightness decay time constant is 2.5 milliseconds. The peak overshoot brightness is 1.8 milliseconds. The overshoot ratio is calculated to be 550 candela per square meter. The value is 0.1. Obtain the pixel arrangement geometry parameters of this panel space, including the pixel horizontal period. With longitudinal period Both are 50 micrometers wide, representing the effective light-emitting area width of the pixel. With height All pixels are 45 micrometers in size, from which the pixel duty cycle is calculated and a grid matrix is ​​generated. Utilizing dynamic response characteristic parameters , and Generate unit step response vector Finally, the prior template for the display end is obtained through outer product operation. The template fully includes the dynamic response feature parameters and spatial pixel arrangement geometric parameters extracted from the aforementioned measured brightness establishment and decay process data.

[0053] S2. Based on the prior template of the display end, construct the corresponding optical coding metric matrix, and generate an optical modulation mask loaded on the optical wavefront modulation device according to the optical coding metric matrix.

[0054] In one embodiment of the present invention, step S2 includes the following steps:

[0055] Based on the spatiotemporal coupling characteristics of the prior template of the display end, the spatial frequency modulation range and modulation depth constraints required for optical coding are determined.

[0056] Based on the constraints of spatial frequency modulation range and modulation depth, an optical coding metric matrix is ​​constructed for modulating the incident light field in a specific mode.

[0057] The optical coding metric matrix is ​​quantized into phase or amplitude distribution data that conforms to the physical modulation range of the optical wavefront modulator, thereby generating an optical modulation mask.

[0058] Specifically, step S2 is implemented based on the spatiotemporal coupling characteristics of the display terminal's prior template. This display terminal's prior template is denoted as... It includes the pixel spatial geometric arrangement information obtained from step S1 and the temporal dynamic response characteristics of the pixel units. When determining the spatial frequency modulation range required for optical encoding, it mainly relies on the lateral periodicity of the pixels characterizing the spatial structure in the prior template of the display. With longitudinal period To avoid spectral aliasing during imaging, the upper limit of the spatial frequency range that the encoding must cover is... Determined by the Nyquist sampling theorem, its calculation formula is as follows:

[0059] ,

[0060] in and The unit is micrometer. The unit is line pairs per micrometer. The modulation depth constraint is mainly related to the brightness rise time constant, which characterizes the temporal properties, in the prior template of the display. With brightness decay time constant To ensure that the optical encoding can effectively capture the dynamic process of the display response, a minimum modulation depth is set. It needs to match the overall time resolution of the system. One setting method is:

[0061] ,

[0062] in To display the frame period, This is an empirical coefficient determined based on 200 sets of timing simulation tests, typically ranging from 0.5 to 2.0. The formula normalizes the time constant by comparing it to the frame period, making... This is a dimensionless ratio that represents the minimum contrast required for the modulated signal.

[0063] Based on the determined spatial frequency modulation range With modulation depth constraints Construct an optically encoded metric matrix for modulating the incident light field in a specific mode. The matrix is ​​constructed in the following way: in the frequency domain Within, at a preset discrete interval Uniform sampling yields a set of base frequencies For each spatial frequency basis This generates a two-dimensional pattern in the spatial domain that is sinusoidally modulated at that frequency:

[0064] ,

[0065] in and It is frequency Components in the x and y directions, It is in Phase shifts are randomly initialized within the module to increase mode diversity. Subsequently, each two-dimensional modulation pattern is... Expanding the matrix in spatial order into a row vector, and stacking the row vectors corresponding to all M frequency bases, we obtain the optical coding metric matrix. Its dimensions are , where N is the total number of pixels in the imaging sensor array. Matrix Each element's value ranges from 0 to 1, and its overall modulation depth is scalar-scaled to ensure that the minimum contrast of all modes is not less than [value missing]. .

[0066] Optical coding measure matrix The phase or amplitude distribution data is quantized to conform to the physical modulation range of the optical wavefront modulator, generating an optical modulation mask. For spatial light modulators using pure phase modulation, the quantization process will convert the matrix... Each element Linear mapping to The phase interval is calculated using the following formula:

[0067] ,

[0068] The resulting matrix This refers to an optical modulation mask. For devices employing amplitude modulation, the mask is directly applied... As an optical modulation mask The final generated optical modulation mask data format must be compatible with the control port drive protocol of the target optical wavefront modulator.

[0069] For example, taking the prior template parameters of the display end obtained in step S1, the pixel horizontal period... With longitudinal period Both are 50 micrometers. Calculate the upper limit of the spatial frequency modulation range. Brightness rise time constant in dynamic response characteristic parameters The brightness decay time constant is 2.5 milliseconds. The display frame period is 1.8 milliseconds. It is 16.7 milliseconds, empirical coefficient. Taking 1.0, the minimum modulation depth constraint is calculated. Based on spatial frequency range With a minimum modulation depth of 0.108, in the frequency domain Interval sampling yields 21 frequency bases. A random-phase sinusoidal modulation two-dimensional pattern is generated for each frequency base and unfolded to construct an optical coded metric matrix. It has 21 rows and columns, corresponding to the assumed 1 million pixels of the imaging sensor in the example. Using a pure phase spatial light modulator, the matrix... Each element multiplied by Quantization is performed to obtain the value range within Optical modulation mask between The mask data can be directly loaded into the control port of the spatial light modulator, and its parameters are entirely derived from the spatial frequency modulation range and modulation depth constraints calculated from the spatiotemporal coupling characteristics of the aforementioned display end a priori template.

[0070] S3. Control the optical wavefront modulation device to physically modulate the incident light field through the optical modulation mask, and project the modulated incident light field onto a preset imaging sensor array to obtain optically encoded coupled sampling data.

[0071] In one embodiment of the present invention, step S3 includes the following steps:

[0072] The optical modulation mask is loaded onto the control port of the optical wavefront modulator to form a physical modulation pattern;

[0073] The incident light field from the target scene or light source is guided through the optical wavefront modulator loaded with the physical modulation pattern to generate a modulated light field with encoded information.

[0074] The modulated light field is guided to the photosensitive plane of the preset imaging sensor array for integration sampling to obtain coupled sampling data.

[0075] Specifically, such as Figure 3 As shown, step S3 uses the optical modulation mask generated in step S2 to physically modulate the light field and acquire data. This process begins by loading the optical modulation mask onto the control port of the optical wavefront modulator. (Optical modulation mask) Its data format is a two-dimensional matrix conforming to the device addressing format. For example, for a digital micromirror device, it is a binarized amplitude distribution, while for a liquid crystal spatial light modulator, it is a normalized phase or amplitude distribution matrix. The matrix is ​​transmitted through a dedicated driving circuit and data transmission interface. Each element value is written into the physical control unit corresponding to the optical wavefront modulator, thereby forming a matrix within the effective modulation region of the device. The corresponding physical modulation pattern. This pattern is an optically encoded metric matrix. The spatial distribution is reflected after being realized by physical devices.

[0076] Subsequently, the incident light field from the target scene or a stable light source is guided through an optical wavefront modulator loaded with this physical modulation pattern. The incident light field can be characterized as a function of its spatial and temporal distribution. The unit is watts per square meter. As it passes through the device, the device adjusts its power according to each spatial point. Value of physical modulation pattern Modulation alters specific properties of light waves. For amplitude modulation, the transmitted or reflected light intensity is directly weighted; for phase modulation, the wavefront phase of the light wave is changed, typically translated into intensity changes during subsequent propagation or interference. The modulation process can be described by a transfer function, generating a modulated light field with encoded information. Taking amplitude modulation as an example, the relationship can be expressed as:

[0077] ,

[0078] in It is the overall optical efficiency coefficient of the device, a dimensionless constant less than 1. The value of is between 0 and 1. Both ends of the formula have the same dimension, which is the unit of light intensity (watts per square meter).

[0079] Finally, the modulated light field will be The light is guided and focused onto the photosensitive plane of a pre-defined imaging sensor array by an imaging lens group. The photosensitive plane is composed of a large number of independent photosensitive units, or pixels, arranged in a regular pattern. Each pixel has a fixed exposure integration time. Inside, its photosensitive area The total optical energy received is collected and converted into an electrical signal. This integral sampling process is for the first... For each pixel, its output raw signal value It can be represented as:

[0080] ,

[0081] in It is the response coefficient of the imaging sensor array, which can be measured in volts per joule or digital counts per joule. It converts the collected light energy into voltage or digital value. This is the offset introduced by dark current or background noise. Integration time. The settings need to take into account the dynamic response time constant in the prior template of the display, and are usually set to . and Several times that, for example, based on 200 sets of time-series matching tests. Can be set to This is to ensure that a complete brightness build-up or decay process of the displayed pixel is fully captured. After performing this integral sampling operation on all pixels, the resulting set... That is, optically encoded coupled sampling data Its dimensions It equals the total number of pixels in the imaging sensor array.

[0082] For example, following the example of step S2, the optical modulation mask has been... The light is applied to a pure phase-type liquid crystal spatial light modulator. The incident light field comes from a surface light source emitting uniform white light from a light-emitting diode, and its intensity... It is spatially uniform and stable over the integration time. Phase modulation pattern. The resulting phase change in the light wavefront, after propagating through free space, is converted into a specific intensity distribution by a filter placed on the Fourier surface of a 4-f system, thus effectively achieving amplitude modulation. The imaging sensor array uses a global shutter complementary metal-oxide-semiconductor sensor with a pixel size of 2.2 micrometers by 2.2 micrometers, therefore the photosensitive area of ​​each pixel is... for Square meters. Based on the maximum time constant obtained in the example of step S1. Milliseconds, setting the integration time Sensor response coefficient for Volts per joule. Substituting the above parameters into the integration formula, calculations are performed for each pixel on the sensor, ultimately obtaining a voltage value vector with a dimension of 1 million. This vector is the optically encoded coupled sampling data obtained in this step. The generation of this data is strictly dependent on the loaded optical modulation mask. Set integration time The sensor's physical parameters were also fully reproduced, showing the entire process from physical modulation of the optical field to integral sampling.

[0083] S4. Establish a cross-domain correlation tensor by utilizing the photoelectric mapping relationship between the prior template of the display end and the optical modulation mask;

[0084] In one embodiment of the present invention, step S4 includes the following steps:

[0085] Based on the optical modulation mask, a linear projection relationship from the actual display light output domain to the image sensing domain is established to obtain the first mapping relationship;

[0086] Based on the prior template of the display end, a time-domain transformation relationship from the desired display image domain to the image sensing domain is established to obtain the second mapping relationship;

[0087] By concatenating the first mapping relationship with the second mapping relationship, a cross-domain correlation tensor is formed.

[0088] Specifically, step S4 uses the prior template of the display end and the optical modulation mask to establish a cross-domain correlation tensor. This process begins with the optical modulation mask, which is the two-dimensional phase or amplitude distribution data generated in step S2, denoted as... Its spatial dimensions correspond to the number of controllable units in the optical wavefront modulator, denoted as A linear projection relationship, i.e., the first mapping relationship, is established based on an optical modulation mask from the actual display light output domain to the image sensing domain. The actual display light output domain consists of the light intensity sequence emitted by all pixels of the display panel within a specific time period, which can be discretized into a three-dimensional tensor. ,in and The number of pixel rows and columns of the display panel. The number of discrete points in time. The image sensing domain consists of data acquired by a single integral sampling of the imaging sensor array, and can be represented as a vector. , This represents the total number of sensor pixels. First mapping relationship. It is a size of A matrix whose each element Characterized the first The first display light output unit (corresponding to a specific spatial location and time) for the first... The contribution weight of each sensor pixel reading. This matrix is ​​obtained through optical modeling: assuming the display panel's... Each pixel emits light with a unit intensity at the corresponding moment. The emitted light propagates through the optical system and is modulated by an optical mask. After modulation, an optical field distribution is formed on the sensor plane. Then the weight This is equal to the distribution of the light field at the th Each sensor pixel photosensitive area The integral on, that is Calculation in progress It can be obtained using ray tracing or point spread function models, which include optical modulation masks. The multiplication factor modulates the operation at the corresponding position. The matrix is ​​constructed in this way. Vectorized actual display light output Linear mapping to sensor readings ,satisfy .

[0089] Subsequently, based on the prior template of the display end, a time-domain transformation relationship from the desired display image domain to the image sensing domain is established, namely the second mapping relationship. The desired display image domain consists of a sequence of digital driving values ​​to be displayed, denoted as . ,in For driving frame rate. Display-side prior template. It is a three-dimensional tensor with spatiotemporal coupling properties, where its two-dimensional spatial component represents pixel geometry and its temporal component represents the unit step response. Second mapping relationship. It is a size of The matrix will vectorize the expected display image. Mapped to vectorized actual display light output ,Right now .matrix The construction is based on the prior template of the display end. The described discrete-time spatiotemporal convolution relationship. Specifically, it will... Considered as a three-dimensional convolution kernel, its spatial dimensions are aligned with the display pixel grid, and its time length is... So for any desired image sequence... Its actual light output In position and time The value is given by the following convolution:

[0090] ,

[0091] in For spatial neighborhood, This is the time scaling factor. Expanding this convolution relation across all spatial locations and time points yields the matrix. Its elements are composed of The values ​​are filled in a block-cycle manner. This mapping is essentially linear, and its setting is based on the fact that the grayscale response characteristics of the display panel can be approximated as a linear system within the normal operating range.

[0092] Finally, by concatenating the first and second mapping relationships, a cross-domain correlation tensor is formed. Concatenation operations are equivalent to matrix multiplication, which involves cross-domain correlation of tensors. Its dimensions are This tensor establishes a complete linear mapping directly from the desired display image domain to the image sensing domain: In the subsequent solution process, This will serve as the fundamental operator for regularization constraints, and its construction depends entirely on known optical modulation masks. and display prior template It does not require additional calibration data.

[0093] For example, suppose the display panel has a pixel row and column count for Number of discrete points in time Set to 5, drive frame rate The total number of pixels in the imaging sensor array is 3. 100. Optical modulation mask. Size is A simplified optical model is adopted, assuming that after each display pixel emits light through an ideal lens, it forms a uniform light spot of the same size and corresponding to a sensor pixel on the sensor plane. However, the light intensity is weighted by the positional value of the optical modulation mask. Based on this model, the first mapping matrix is ​​calculated. Its size is Display-side prior template The space part is The pixel duty cycle matrix, the time part is of length [missing information]. unit step response vector The second mapping matrix is ​​obtained by expanding based on the convolution relationship. Its size is .Will and Multiply to obtain the cross-domain association tensor Size is This tensor Each element value is calculated by the specific model described above, verifying the effectiveness of constructing the mapping relationship from the optical modulation mask and the prior template of the display end.

[0094] S5. Using the cross-domain correlation tensor as a regularization constraint, solve the coupled sampling data to reconstruct the display image data adapted to the characteristics of the display panel.

[0095] In one embodiment of the present invention, step S5 includes the following steps:

[0096] Construct an objective optimization function with the difference between the reconstructed image and the coupled sampled data in the image sensing domain as the fidelity term and the smoothness of the display effect predicted by the reconstructed image through the cross-domain correlation tensor as the regularization term;

[0097] The target optimization function is solved using a preset iterative optimization algorithm to update and reconstruct the image;

[0098] When the target optimization function converges to a preset threshold or reaches a preset number of iterations, the image data to be displayed is output.

[0099] Optionally, the output of the image data to be displayed includes:

[0100] Acquire optical sensing data of the external environment in which the display terminal is located;

[0101] Analyze the optical sensing data to extract environmental interference features that characterize changes in ambient light intensity and color temperature;

[0102] Using the environmental interference features as dynamic regularization constraints, when the objective optimization function converges to a preset threshold or reaches a preset number of iterations, the image data to be displayed is output.

[0103] Specifically, step S5 uses the cross-domain correlation tensor as a regularization constraint to solve for the coupled sampled data. Cross-domain correlation tensor Established by step S4, its dimension is ,in This represents the total number of pixels in the imaging sensor array. and This refers to the number of pixel rows and columns of the display panel. The number of frames in the image sequence to be displayed. Coupled sampling data. Obtained from step S3, which is the dimension. The vector. Image data to be displayed. It is a three-dimensional tensor to be solved, with dimension . Its vectorized form is denoted as .

[0104] Construct the objective optimization function This function includes a fidelity term and a regularization term. The fidelity term measures how well the reconstructed image correlates with tensors across domains. Predicted image sensing domain data and actual coupled sampling data The differences between them are adopted Norm square form: Regularization terms are used to improve the smoothness of the display effect, which is achieved through the display light output. It is measured using the total variation of spacetime. Display light output. From the image data to be displayed Through the second mapping matrix Calculated, i.e. ,in It is the time-domain transformation matrix established in step S4, from the desired display image domain to the actual display light output domain, with dimensions of... , To display the number of discrete points in time for the light output. Display light output. Spacetime Total Variation Defined as the sum of the absolute values ​​of the spatial gradient and the temporal gradient:

[0105] ,

[0106] in , , Let represent the first-order finite differences in the spatial horizontal, spatial vertical, and time directions, respectively, with the difference value being zero at the boundary. This is the time smoothing weighting coefficient, used to balance spatial and temporal smoothness. Its setting is based on the dynamic response characteristics of the display panel, typically based on 200 sets of dynamic sequence tests, and is set to... ,in This represents the average of the brightness rise and fall time constants. To display the frame period. Regularization parameters. The strength of the smoothness constraint is controlled. The final form of the objective function is:

[0107] ,

[0108] All terms in the formula are dimensionless quantities, because , , , All have been normalized.

[0109] The objective function is solved using an iterative optimization algorithm based on the alternating direction multiplier method. Auxiliary variables are introduced. By separating the total variational terms, the problem is transformed into a constrained optimization form:

[0110] ,

[0111] in This means reshaping the vector into dimensions. The tensor of . The corresponding augmented Lagrangian function is:

[0112] ,

[0113] Here For dual variable tensors, the dimension is... same, The penalty parameter is initially set to 0.1. Let Frobenius norm be represented. The iterative update process is as follows: First, update the original variable. This is achieved by solving a system of linear equations:

[0114] ,

[0115] superscript Indicates matrix transpose. For the current iteration number, the conjugate gradient method is used to solve the problem. The second step is to update the auxiliary variables. This can be achieved by solving the following formula:

[0116] ,

[0117] This problem is equivalent to performing total variation denoising on a noisy image, and is solved using the Chambolle projection algorithm with 5 inner iterations. The third step updates the dual variables. :

[0118] ,

[0119] During the iteration process, the penalty parameter Multiply by a coefficient of 1.1 every 10 iterations to accelerate convergence.

[0120] When the objective optimization function value Compared with the previous iteration value The relative change is less than the preset threshold. The iteration terminates when the number of iterations reaches a preset maximum value of 100. The final vector is then... Reorganization into Dimensions The tensor is output as the image data to be displayed. .

[0121] Optionally, when outputting the image data to be displayed, environmental interference features are introduced as a dynamic regularization constraint. Optical sensing data of the external environment in which the display terminal is located, including ambient light intensity, is acquired through a light sensor. The unit is lux, and the color temperature is... The unit is Kelvin. Analyzing these optical sensing data, environmental interference characteristics are extracted, specifically the ambient light intensity influencing factor. and color temperature deviation factor ,in Lux and Kelvin is used as the baseline value. Environmental disturbance characteristics are incorporated into the regularization parameter, which is then dynamically adjusted to... ,in Based on the regularization parameters, and The weighting coefficients were set to 0.2 and 0.1 respectively, based on 100 sets of environmental adaptability tests. This dynamic regularization constraint is updated in each iteration. Effective immediately, through adjustment The value is used to adapt to environmental changes.

[0122] For example, suppose the display panel has a pixel row and column count for Drive frame count Total number of pixels in the imaging sensor array Display the number of discrete points in the time display of light output. Cross-domain association tensor Dimensions Coupled sampling data Dimensions The time smoothing weighting coefficients are based on the example parameters. millisecond, Milliseconds, calculated Basic regularization parameters The value is set to 0.05. The alternating direction multiplier method is used for solving the problem. Initially... After 37 iterations, the relative change in the objective function is less than [a certain value]. The iteration terminates. Output the image data to be displayed. , dimension If the light sensor measures the ambient light intensity at this time... Lux, color temperature Kelvin, then calculate , Dynamic regularization parameters This parameter was used as a total variation constraint in the solution process, verifying the effectiveness of environmental disturbance characteristics as a dynamic regularization constraint.

[0123] S6. Combine the image data to be displayed with the prior template of the display terminal to perform consistency residual calculation and generate the driving compensation correction amount;

[0124] In one embodiment of the present invention, step S6 includes the following steps:

[0125] Using the image data to be displayed and the prior template of the display end, the actual light output sequence of the display panel is simulated and obtained;

[0126] Calculate the difference between the ideal display light output corresponding to the image data to be displayed and the actual light output sequence of the display panel to generate the original residual signal;

[0127] The original residual signal is subjected to time-domain filtering and amplitude scaling to generate a driving compensation correction amount.

[0128] Specifically, step S6 combines the image data to be displayed with the prior template of the display terminal to perform consistency residual calculation. Image data to be displayed The output of step S5 is a dimension of The three-dimensional tensor, in which and This refers to the number of pixel rows and columns of the display panel. For driving frame rate. Display-side prior template. Constructed by step S1, it is a three-dimensional tensor with spatiotemporal coupling characteristics. Its two-dimensional spatial part represents the pixel geometry, and its temporal part represents the unit step response vector. .

[0129] Using the image data to be displayed Prior template for display Simulation yields the actual light output sequence of the display panel. This simulation process is achieved through three-dimensional discrete convolution:

[0130] ,

[0131] in These are the spatial coordinates on the display panel. For discrete-time indexing, To map the driving frame interval to the display's prior template temporal resolution scaling factor, it is typically set to 1. , To display the frame period, The temporal resolution of the prior template for the display. When the index is out of range... The value is zero. This convolution operation is equivalent to the second mapping relationship established in step S4. Matrix multiplication, i.e. ,in It is by Constructed block Toplitz matrix. Actual optical output sequence. Dimensions and They are the same in space, but expand to [other forms] in time. One point, , For the display terminal prior template The length of time.

[0132] Calculate the image data to be displayed Corresponding ideal display light output With respect to the actual light output sequence of the display panel The difference between them generates the original residual signal. Ideal display light output This is obtained under the ideal condition of ignoring all dynamic delays and spatial crosstalk of the display panel. The calculation method is as follows: Each frame is spatially determined by the pixel duty cycle matrix. Modulation is performed, and a transient response is assumed in time. Specifically, It can be represented as:

[0133] ,

[0134] in It is the display end prior template The pixel duty cycle matrix extracted from the spatial portion It is the discrete Dirac function, that is, at the corresponding time point The upper value is 1, and the rest are 0. Original residual signal Calculated by point-by-point difference:

[0135] ,

[0136] in, Dimensions and Same, for .

[0137] For the original residual signal Time-domain filtering and amplitude scaling are performed to generate the drive compensation correction amount. Time-domain filtering aims to suppress high-frequency disturbances introduced by measurement noise or model inaccuracies, while preserving systematic residual components caused by the dynamic characteristics of the display panel. A time-domain filter with a length of... A linear-phase finite-impulse response low-pass filter with the transfer function is given by... Filter coefficients According to the cutoff frequency Design, cutoff frequency The settings are based on the dynamic response feature parameters extracted from the prior template of the display terminal, specifically: ,in Brightness rise time constant With brightness decay time constant The average value. Based on 200 sets of dynamic response tests, the filter length... Set as And round down. Time-domain filtering along the time dimension. For each spatial location residual sequence Performed independently, the filtered residual signal is obtained. .

[0138] Amplitude scaling maps the filtered residual signal back to the driving range, taking into account the driving nonlinearity of the display panel. The scaling formula is:

[0139] ,

[0140] in This is the global scaling factor, typically set between 0.5 and 1.0, determined based on closed-loop calibration experiments; It is the driving value Light output The derivative of represents the electro-optical conversion characteristics of the display panel. This function can be obtained by difference from a pre-calibrated drive-brightness lookup table. The summation operation is performed for each drive frame cycle. The filtered residuals are accumulated to generate image data to be displayed. Dimensionally identical driving compensation correction amount ,Right now .

[0141] For example, suppose the image data to be displayed Dimensions Display end prior template The space part is Duty cycle matrix The time component is the unit step response vector. Length is Time resolution Milliseconds, display frame period Milliseconds, hence the scaling factor The actual optical output sequence was obtained through convolution simulation. Its duration Calculate the ideal display light output. And obtain the original residual signal , dimension Average value of dynamic response characteristic parameters Milliseconds, calculation cutoff frequency Design a finite impulse response low-pass filter with a cutoff frequency of 74.1 Hz and a length of... The residual sequence of each pixel is filtered to obtain... Global scaling factor Taking a value of 0.8, amplitude scaling is performed using the derivative of the calibrated electro-optic conversion function, and the values ​​at 33 time points within each driving frame period are summed to finally generate the driving compensation correction amount. , dimension .

[0142] S7. The driving compensation correction amount is superimposed on the image data to be displayed to generate a display driving sequence optimized for the physical characteristics of the display panel.

[0143] In one embodiment of the present invention, step S7 includes the following steps:

[0144] The drive compensation correction amount is calculated with the corresponding image data to be displayed to obtain the drive data after preliminary compensation;

[0145] The pre-compensated drive data is converted from digital to analog and matched to levels to generate an analog voltage or current sequence.

[0146] According to the horizontal and vertical timing requirements of the display panel, organize the analog voltage or current sequence and output the display drive sequence.

[0147] Specifically, step S7 superimposes the drive compensation correction amount onto the image data to be displayed, generating a display drive sequence. Drive compensation correction amount Generated by step S6, it is a dimension of The three-dimensional tensor, in which and This refers to the number of pixel rows and columns of the display panel. This is the driving frame rate. Image data to be displayed. The output of step S5 is a three-dimensional tensor of the same dimension.

[0148] Drive compensation correction amount With the corresponding image data to be displayed Perform calculations to obtain the pre-compensated driving data. This operation is an element-wise addition:

[0149] ,

[0150] in , , These represent the spatial row index, spatial column index, and temporal frame index, respectively. The addition operation is performed based on the same coordinate positions, ensuring that each display pixel receives its corresponding compensation value in each driving frame. Because... and These are all normalized numeric driving values, and their value range is typically... or The sum of integers The value may exceed the original range, therefore limiting is required. The limiting function is defined as follows:

[0151] ,

[0152] here and These are the minimum and maximum digital drive values ​​supported by the display panel driver circuit, respectively. For example, for an 8-bit drive, , The data after the bandwidth limit will still be recorded as follows. .

[0153] For the driving data after preliminary compensation The process involves digital-to-analog conversion and level matching to generate an analog voltage or current sequence. The digital-to-analog conversion process converts each digital drive value... Mapped to the corresponding analog level This mapping relationship is determined by the characteristics of the display panel's driver chip and is typically a piecewise linear function or a lookup table. Taking a voltage-driven display panel as an example, its mapping formula can be expressed as:

[0154] ,

[0155] in It corresponds to the digital drive value The analog voltage, It corresponds to the digital drive value The analog voltages. These two voltage values ​​are based on the display panel's datasheet settings, for example... volt, Volts. Ensuring that when Time output ,when Time output , .

[0156] Organize the analog voltage or current sequence according to the horizontal and vertical timing requirements of the display panel. The output displays the driving sequence. The timing requirements of the display panel include the row validity time. Line disappearance time Valid time of the venue Field disappearance time The organization process is performed frame by frame; for the first frame... Frame data This is converted into a one-dimensional sequence by scanning the rows. Specifically, for the first row... OK( From 1 to The pixel data for this row is The corresponding analog voltage value Valid time of the line The output is sequential, and the holding time for each pixel is... After a row of data is output, insert the row blanking time. During this period, the output blanking level is 100%. , usually equal to Repeat this process until all lines of a frame have been output, then insert the field blanking time. The final output is a continuous analog level waveform that conforms to the panel timing specifications, i.e., the display drive sequence. The sequence in time value Determined by the current frame, line, and pixel position, its mathematical description is a piecewise function, but it is actually generated in real time by the driving circuit hardware according to the above rules.

[0157] For example, suppose the image data to be displayed and drive compensation correction amount All The tensor uses 8-bit driving, therefore , Take spatial position Taking the first frame of data as an example, , Calculated The value is within the range and no limiting is required. Voltage drive parameters volt, volts, calculate the corresponding analog voltage: The volt timing parameters are set as follows: microseconds microseconds Therefore, the retention time of each pixel Microseconds. When generating the display drive sequence, the 20th pixel in the 10th row of the first frame will output approximately 2.47 volts at a specific moment and hold for 0.3125 microseconds. The entire sequence strictly follows the drive compensation correction amount. With the image data to be displayed The process begins with addition, followed by analog-to-digital conversion and timing organization to generate the final drive waveform. All parameters are set based on the output of the aforementioned steps and the physical characteristics of the display panel.

[0158] Based on the same inventive concept, such as Figure 4 As shown, the present invention also provides a high-resolution image computing and display integrated system based on a photoelectric computing hybrid architecture, the system comprising:

[0159] The display response modeling module is used to jointly model the temporal grayscale response characteristics and spatial pixel structure of the display panel to construct a priori template for the display end.

[0160] An optical coding module is used to construct a corresponding optical coding metric matrix based on the prior template of the display end, and to generate an optical modulation mask loaded on the optical wavefront modulation device according to the optical coding metric matrix.

[0161] The imaging acquisition module is used to control the optical wavefront modulation device to physically modulate the incident light field through the optical modulation mask, and project the modulated incident light field onto a preset imaging sensor array to acquire optically encoded coupled sampling data.

[0162] The correlation mapping module is used to establish a cross-domain correlation tensor by utilizing the photoelectric mapping relationship between the display end prior template and the optical modulation mask;

[0163] The constraint solving calculation module is used to solve the coupled sampling data using the cross-domain correlation tensor as a regularization constraint, and reconstruct the display image data adapted to the characteristics of the display panel.

[0164] The consistency evaluation module is used to calculate the consistency residual by combining the image data to be displayed with the prior template of the display end, and generate the driving compensation correction amount;

[0165] The display driver module is used to superimpose the drive compensation correction amount onto the image data to be displayed, thereby generating a display driver sequence optimized for the physical characteristics of the display panel.

[0166] It should be noted that the electrical connections between the various units described above do not necessarily represent direct or indirect connections. Any indirect connection method is applicable to the embodiments of the present invention as long as it achieves the purpose of the present invention. The above descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the present invention.

[0167] All equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of this invention upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this invention that follow the general principles of this invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. A high-resolution image processing and display integrated method based on a hybrid optoelectronic computing architecture, characterized in that: The method includes: The temporal grayscale response characteristics and spatial pixel structure of the display panel are jointly modeled to construct a priori template for the display end; Based on the prior template of the display end, a corresponding optical coding metric matrix is ​​constructed, and an optical modulation mask loaded on the optical wavefront modulation device is generated according to the optical coding metric matrix. The optical wavefront modulation device is controlled to physically modulate the incident light field through the optical modulation mask, and the modulated incident light field is projected onto a preset imaging sensor array to obtain optically encoded coupled sampling data. A cross-domain correlation tensor is established by utilizing the photoelectric mapping relationship between the prior template of the display end and the optical modulation mask; Using the cross-domain correlation tensor as a regularization constraint, the coupled sampling data is solved to reconstruct the display image data adapted to the characteristics of the display panel; By combining the image data to be displayed with the prior template of the display end, a consistency residual calculation is performed to generate a driving compensation correction amount; The driving compensation correction is superimposed on the image data to be displayed to generate a display driving sequence optimized for the physical characteristics of the display panel.

2. The high-resolution image processing and display method based on a photoelectric computing hybrid architecture according to claim 1, characterized in that, The joint modeling of the temporal grayscale response characteristics and spatial pixel structure of the display panel to construct the prior template for the display end includes: The time-domain sequence of light intensity output of the display panel under different driving signals is obtained, and the brightness establishment and decay process data of the pixel unit are measured. Analyze the brightness build-up and decay process data of pixel units to extract dynamic response characteristic parameters that characterize the response delay and overshoot characteristics of the display panel; Obtain the spatial pixel arrangement geometric parameters of the display panel, and combine them with the dynamic response feature parameters to construct a priori template for the display end that includes spatiotemporal coupling characteristics.

3. The high-resolution image processing and display method based on a photoelectric computing hybrid architecture according to claim 2, characterized in that, The step of constructing a corresponding optical coding metric matrix based on the prior template of the display end, and generating an optical modulation mask loaded on the optical wavefront modulation device according to the optical coding metric matrix includes: Based on the spatiotemporal coupling characteristics of the prior template of the display end, the spatial frequency modulation range and modulation depth constraints required for optical coding are determined. Based on the constraints of spatial frequency modulation range and modulation depth, an optical coding metric matrix is ​​constructed for modulating the incident light field in a specific mode. The optical coding metric matrix is ​​quantized into phase or amplitude distribution data that conforms to the physical modulation range of the optical wavefront modulator, thereby generating an optical modulation mask.

4. The high-resolution image processing and display method based on a photoelectric computing hybrid architecture according to claim 3, characterized in that, The acquisition of optically encoded coupled sampling data includes: The optical modulation mask is loaded onto the control port of the optical wavefront modulator to form a physical modulation pattern; The incident light field from the target scene or light source is guided through the optical wavefront modulator loaded with the physical modulation pattern to generate a modulated light field with encoded information. The modulated light field is guided to the photosensitive plane of the preset imaging sensor array for integration sampling to obtain coupled sampling data.

5. The high-resolution image processing and display method based on a photoelectric computing hybrid architecture according to claim 4, characterized in that, The step of establishing a cross-domain correlation tensor by utilizing the photoelectric mapping relationship between the display terminal prior template and the optical modulation mask includes: Based on the optical modulation mask, a linear projection relationship from the actual display light output domain to the image sensing domain is established to obtain the first mapping relationship; Based on the prior template of the display end, a time-domain transformation relationship from the desired display image domain to the image sensing domain is established to obtain the second mapping relationship; By concatenating the first mapping relationship with the second mapping relationship, a cross-domain correlation tensor is formed.

6. The high-resolution image processing and display method based on a photoelectric computing hybrid architecture according to claim 5, characterized in that, The step of using the cross-domain correlation tensor as a regularization constraint to solve the coupled sampling data and reconstruct the display image data adapted to the characteristics of the display panel includes: Construct an objective optimization function with the difference between the reconstructed image and the coupled sampled data in the image sensing domain as the fidelity term and the smoothness of the display effect predicted by the reconstructed image through the cross-domain correlation tensor as the regularization term; The target optimization function is solved using a preset iterative optimization algorithm to update and reconstruct the image; When the target optimization function converges to a preset threshold or reaches a preset number of iterations, the image data to be displayed is output.

7. The high-resolution image processing and display method based on a photoelectric computing hybrid architecture according to claim 6, characterized in that, The output of the image data to be displayed includes: Acquire optical sensing data of the external environment in which the display terminal is located; Analyze the optical sensing data to extract environmental interference features that characterize changes in ambient light intensity and color temperature; Using the environmental interference features as dynamic regularization constraints, when the objective optimization function converges to a preset threshold or reaches a preset number of iterations, the image data to be displayed is output.

8. The high-resolution image processing and display method based on a photoelectric computing hybrid architecture according to claim 6, characterized in that, The step of combining the image data to be displayed with the prior template of the display terminal to perform consistency residual calculation and generate the driving compensation correction amount includes: Using the image data to be displayed and the prior template of the display end, the actual light output sequence of the display panel is simulated and obtained; Calculate the difference between the ideal display light output corresponding to the image data to be displayed and the actual light output sequence of the display panel to generate the original residual signal; The original residual signal is subjected to time-domain filtering and amplitude scaling to generate a driving compensation correction amount.

9. The high-resolution image processing and display method based on a photoelectric computing hybrid architecture according to claim 8, characterized in that, The step of superimposing the driving compensation correction amount onto the image data to be displayed to generate a display driving sequence optimized for the physical characteristics of the display panel includes: The drive compensation correction amount is calculated with the corresponding image data to be displayed to obtain the drive data after preliminary compensation; The pre-compensated drive data is converted from digital to analog and matched to levels to generate an analog voltage or current sequence. According to the horizontal and vertical timing requirements of the display panel, organize the analog voltage or current sequence and output the display drive sequence.

10. A high-resolution image computing and display integrated system based on a hybrid optoelectronic computing architecture, characterized in that: The system includes: The display response modeling module is used to jointly model the temporal grayscale response characteristics and spatial pixel structure of the display panel to construct a priori template for the display end. An optical coding module is used to construct a corresponding optical coding metric matrix based on the prior template of the display end, and to generate an optical modulation mask loaded on the optical wavefront modulation device according to the optical coding metric matrix. The imaging acquisition module is used to control the optical wavefront modulation device to physically modulate the incident light field through the optical modulation mask, and project the modulated incident light field onto a preset imaging sensor array to acquire optically encoded coupled sampling data. The correlation mapping module is used to establish a cross-domain correlation tensor by utilizing the photoelectric mapping relationship between the display end prior template and the optical modulation mask; The constraint solving calculation module is used to solve the coupled sampling data using the cross-domain correlation tensor as a regularization constraint, and reconstruct the display image data adapted to the characteristics of the display panel. The consistency evaluation module is used to calculate the consistency residual by combining the image data to be displayed with the prior template of the display end, and generate the driving compensation correction amount; The display driver module is used to superimpose the drive compensation correction amount onto the image data to be displayed, thereby generating a display driver sequence optimized for the physical characteristics of the display panel.

Images