Extended display method, device and equipment of cloud mobile phone, medium and product

Abstract

The invention discloses an extended display method and device of a cloud mobile phone, equipment, a medium and a product, and the method comprises the steps: carrying out the feature extraction of the current target display content when receiving the physical environment data uploaded by at least one extended terminal, and obtaining a content feature matrix of the target display content; generating a driving instruction corresponding to the extension terminal according to the content feature matrix and the physical environment data; and sending the driving instruction to a corresponding extension terminal, so that the corresponding extension terminal adjusts a corresponding programmable photonic crystal array according to the driving instruction to generate a specified target lattice structure, and generates corresponding light field distribution according to the target lattice structure to present corresponding target display content. A photon topological mapping technology is adopted to regulate and control the effective refractive index through an electric field to generate a directional light field, indifference superposition of display light and ambient light is avoided, the problem of contrast attenuation caused by ambient light interference is solved, and the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of cloud phone technology, and in particular to an extended display method, apparatus, device, medium, and product for cloud phones. Background Technology

[0002] Traditional cloud phone multi-screen extension solutions primarily rely on physical connections between hardware screens and software-level screen segmentation technologies to achieve multi-screen interaction. For example, they use the Miracast protocol for screen mirroring or a distributed rendering architecture to distribute computing tasks to auxiliary terminals. Therefore, the displayed images in traditional cloud phone multi-screen extension solutions need to be presented through medium reflection or direct overlay. However, because traditional cloud phone multi-screen extension solutions lack active anti-glare adjustment mechanisms, such as light field compensation or isolation technology, they are severely affected by ambient light interference. When the ambient light intensity reaches 200 lux, the effective resolution of the displayed light decreases significantly, the image contrast degrades, and the display effect deteriorates drastically, resulting in a poor user experience. Summary of the Invention

[0003] This invention provides an extended display method, device, equipment, medium, and product for cloud phones. It uses photonic topology mapping technology to generate a directional light field by controlling the equivalent refractive index through an electric field, avoiding the indiscriminate superposition of display light and ambient light, solving the contrast attenuation problem caused by ambient light interference, and improving the user experience.

[0004] To achieve the above objectives, embodiments of the present invention provide an extended display method for a cloud phone, applied to a main terminal, comprising: When physical environment data uploaded by at least one extended terminal is received, feature extraction is performed on the current target display content to obtain the content feature matrix of the target display content; The corresponding extended terminal driving instructions are generated based on the content feature matrix and the physical environment data. The driving command is sent to the corresponding extended terminal, so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array according to the driving command to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0005] As an improvement to the above solution, the step of generating the corresponding extended terminal's driving instructions based on the content feature matrix and the physical environment data includes: Calculate the virtual space coordinates of the corresponding extended terminal to adapt to the target display content based on the content feature matrix and the physical environment data; The corresponding extended terminal driving instructions are generated based on the virtual space coordinates and the content feature matrix.

[0006] As an improvement to the above solution, the step of generating the corresponding extended terminal's driving instructions based on the virtual space coordinates and the content feature matrix includes: Calculate the target phase of the corresponding extended terminal based on the virtual space coordinates and the content feature matrix; The photonic crystal driving matrix of the corresponding extended terminal is calculated based on the target phase, and the driving command of the corresponding extended terminal is generated based on the photonic crystal driving matrix.

[0007] As an improvement to the above solution, the step of sending the driving command to the corresponding extended terminal, so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array according to the driving command to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content, includes: The driving command is sent to the corresponding extended terminal so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array to generate the specified adjustment lattice structure according to the driving command. When the actual optical field data uploaded by the corresponding extended terminal after adjusting the corresponding programmable photonic crystal array is received, the photonic crystal driving matrix of the corresponding extended terminal is calibrated according to the actual optical field data, and a calibration instruction for the corresponding extended terminal is generated. The calibration command is sent to the corresponding extended terminal, so that the corresponding extended terminal calibrates the corresponding adjustment lattice structure according to the calibration command, generates the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0008] As an improvement to the above solution, the step of sending the calibration command to the corresponding extended terminal, so that the corresponding extended terminal calibrates the corresponding adjustment lattice structure according to the calibration command to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content, includes: The calibration command is sent to the corresponding extended terminal so that the corresponding extended terminal calibrates the corresponding adjustment lattice structure according to the calibration command and generates the specified calibration lattice structure. When the positioning deviation uploaded by the corresponding extended terminal after calibrating the corresponding adjusted lattice structure is received, the photonic crystal driving matrix of the corresponding extended terminal after calibration is fine-tuned according to the positioning deviation, and the fine-tuning instruction of the corresponding extended terminal is generated. The fine-tuning command is sent to the corresponding extended terminal, so that the corresponding extended terminal fine-tunes the corresponding calibration lattice structure according to the fine-tuning command to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0009] As an improvement to the above solution, before receiving physical environment data uploaded by at least one extended terminal, the method further includes: When a cloud phone is detected to have activated an extended display command, the device type code is broadcast within a preset distance. When an authentication response is received from the terminal to be extended after receiving the device type code, a capability negotiation request is generated based on the authentication response, and the capability negotiation request is sent to the terminal to be extended so that the terminal to be extended can generate a data certificate signed by the terminal to be extended based on the capability negotiation request. When the terminal to be extended receives a signed data certificate after receiving the capability negotiation request, the signature validity of the data certificate is verified, and the certificate chain root verification request of the data certificate is sent to the cloud. When the verification result returned after receiving the cloud verification certificate chain root is received, the node weight between the terminal to be expanded and the main terminal is calculated based on the verification result, as well as the device distance and bandwidth between the terminal to be expanded and the main terminal, and the device topology diagram of the main terminal is generated. The authentication pass command is sent to the terminals to be expanded in the device topology diagram, and the device capability fingerprint database of the main terminal is established based on the terminals to be expanded in the device topology diagram.

[0010] As an improvement to the above solution, if the data channel between the main terminal and the corresponding extended terminal is a distributed rendering channel, then the method for constructing the distributed rendering channel includes: Based on the WebRTC protocol extension, a FOVEC proprietary protocol is constructed as the underlying communication standard for the data channel; A hierarchical transmission architecture is constructed based on a preset transmission strategy. A direct communication link for the data channel is established based on the high-frequency channel between the main terminal and the corresponding extended terminal. A distributed rendering channel is formed based on the underlying communication standard, the hierarchical transmission architecture, and the direct communication link.

[0011] As an improvement to the above solution, after constructing the distributed rendering pipeline, the method further includes: Obtain the eye-tracking data of the target user, perform scene semantic segmentation and application type feature extraction on the target display content, and obtain the scene semantic segmentation result and application type feature of the target display content; A display weight matrix for the target display content is generated based on the user's eye-tracking data, the scene semantic segmentation results, and the application type features; The primary and secondary display areas are divided according to the display weight matrix to generate rendering instructions for the corresponding extended terminals. The rendering instructions are then sent to the corresponding extended terminals so that the corresponding extended terminals can return a rendering confirmation instruction to the primary terminal based on the rendering instructions. Upon receiving the preparation rendering confirmation instruction from the corresponding extended terminal, and after completing cross-terminal clock calibration through the quantum entanglement timestamp synchronization algorithm, the target display content is started for synchronous output.

[0012] As an improvement to the above solution, after receiving the preparation for rendering confirmation instruction returned by the corresponding extended terminal, the method further includes: When the edge node receives the rendering task allocation weight for the corresponding extended terminal, the rendering task of the target display content is allocated to the corresponding extended terminal according to the rendering task allocation weight, so that the corresponding extended terminal can complete the corresponding rendering task; wherein, the rendering task allocation weight is calculated by the edge node based on the environmental state data of the extended terminal.

[0013] As an improvement to the above solution, after presenting the corresponding target display content, the method further includes: When an offline extended terminal is detected, a state replica retrieval request is sent to the blockchain shard node, so that the blockchain shard node retrieves the most recent several state snapshot replicas of the offline extended terminal according to the state replica retrieval request and returns them to the main terminal. When receiving several state snapshot copies returned by the blockchain shard node, perform data integrity verification on the several state snapshot copies to generate the state snapshot corresponding to the offline extended terminal; The offline extended terminal generates a content reorganization instruction based on its rendering task, and sends the content reorganization instruction and the status snapshot to other online extended terminals so that the online extended terminals can complete the content reorganization according to the content reorganization instruction and the status snapshot.

[0014] As an improvement to the above solution, after constructing the distributed rendering pipeline, the method further includes: When a model increment package is received from the cloud, the updated dual-loop deep reinforcement learning model is loaded according to the model increment package; wherein, the model increment package is obtained by the cloud through user behavior data periodically uploaded by each extended terminal to the cloud, so that the cloud can train the dual-loop deep reinforcement learning model according to the user behavior data; The updated model is used to generate personalized extension instructions based on the current usage scenario of the target display content. The personalized extension instructions are then sent to the corresponding extension terminal so that the corresponding extension terminal can adjust the display layout and rendering parameters of the target display content according to the personalized extension instructions.

[0015] To achieve the above objectives, embodiments of the present invention provide another method for extending the display of a cloud phone, applied to an extended terminal, the method comprising: When the optical field configuration start command is received from the main terminal, the physical environment data of the extended terminal is obtained and uploaded to the main terminal. When the main terminal receives the driving instruction returned after receiving the physical environment data, it adjusts the corresponding programmable photonic crystal array according to the driving instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0016] As an improvement to the above solution, the step of generating a corresponding light field distribution based on the target lattice structure to present the corresponding target display content includes: By controlling the optical parameters of the incident light through the target lattice structure, the incident light propagates in the air to generate a light field distribution that coordinates with the main terminal and other extended terminals, so as to present the corresponding target display content.

[0017] As an improvement to the above solution, when the main terminal receives the driving instruction returned after receiving the physical environment data, it adjusts the corresponding programmable photonic crystal array according to the driving instruction to generate a specified target lattice structure, and generates a corresponding light field distribution according to the target lattice structure to present the corresponding target display content, including: When the main terminal receives the driving instruction returned after receiving the physical environment data, the corresponding programmable photonic crystal array is adjusted according to the driving instruction to generate the specified adjustment lattice structure. Obtain the actual light field data of the adjusted lattice structure and upload the actual light field data to the main terminal; When the main terminal receives a calibration instruction after receiving the actual light field data, it calibrates the corresponding adjustment lattice structure according to the calibration instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0018] As an improvement to the above solution, when the main terminal receives a calibration command returned after receiving the actual light field data, the corresponding adjustment lattice structure is calibrated according to the calibration command to generate a specified target lattice structure, and a corresponding light field distribution is generated according to the target lattice structure to present the corresponding target display content, including: When a calibration command is received from the main terminal after receiving the actual light field data, the corresponding adjustment lattice structure is calibrated according to the calibration command to generate the specified calibration lattice structure. The positioning deviation of the calibration lattice structure is obtained and uploaded to the main terminal; When the main terminal receives a fine-tuning instruction returned after receiving the positioning deviation, the corresponding calibration lattice structure is fine-tuned according to the fine-tuning instruction to generate the specified target lattice structure, and the corresponding light field distribution is generated according to the target lattice structure to present the corresponding target display content.

[0019] As an improvement to the above scheme, before receiving the optical field configuration start command issued by the main terminal, the method further includes: When the device type code broadcast by the main terminal is received, an authentication response is generated based on the device type code and uploaded to the main terminal so that the main terminal can generate a capability negotiation request based on the authentication response. When the main terminal receives the capability negotiation request returned after receiving the authentication response, a data certificate to be signed by the terminal to be extended is generated according to the capability negotiation request. The signed data certificate is uploaded to the main terminal so that the main terminal can verify the signature validity of the data certificate and send the certificate chain root verification request of the data certificate to the cloud. When a terminal receives an authentication pass instruction returned by the main terminal after receiving a signed data certificate, it becomes an extended terminal of the main terminal.

[0020] As an improvement to the above scheme, before receiving the optical field configuration start command issued by the main terminal, the method further includes: When a rendering instruction is received from the main terminal, a rendering preparation confirmation instruction is generated based on the rendering instruction, and the rendering preparation confirmation instruction is uploaded to the main terminal. This allows the main terminal to perform cross-terminal clock calibration using a quantum entanglement timestamp synchronization algorithm after receiving the rendering preparation confirmation instruction, and then start synchronously outputting the target display content.

[0021] As an improvement to the above solution, after uploading the preparation rendering confirmation command to the main terminal, the method further includes: When a rendering task is received from the main terminal, the rendering task is completed.

[0022] As an improvement to the above solution, after completing the rendering task, the method further includes: When the main terminal sends a content reorganization instruction and a status snapshot, the content reorganization is completed according to the content reorganization instruction and the status snapshot.

[0023] As an improvement to the above solution, after completing content reconstruction according to the content reconstruction instruction and the state snapshot, the method includes: When the main terminal is accessed, a hardware resource pool for the extended terminal is constructed. When an authentication request from a target user is received, the biometric data of the target user is obtained; the biometric data is uploaded to the FPGA so that the FPGA can authenticate the target user based on the biometric data and return the authentication result to the extended terminal. When the verification result returned by the FPGA is received, if the verification is successful, the user account instance is decrypted from the cloud using a key. The key is dynamically generated based on the biometric verification result and is automatically updated periodically.

[0024] As an improvement to the above solution, after uploading the preparation rendering confirmation command to the main terminal, the method further includes: When a personalized extension instruction is received from the main terminal, the display layout and rendering parameters of the target display content are adjusted according to the personalized extension instruction.

[0025] To achieve the above objectives, embodiments of the present invention provide an extended display device for a cloud phone, applied to a main terminal, comprising: The content feature extraction module is used to extract features from the current target display content to obtain the content feature matrix of the target display content when receiving physical environment data uploaded by at least one extended terminal; The driver instruction generation module is used to generate driver instructions for the corresponding extended terminal based on the content feature matrix and the physical environment data. The driving instruction issuing module is used to issue the driving instruction to the corresponding extended terminal, so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array according to the driving instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0026] To achieve the above objectives, embodiments of the present invention provide another extended display device for cloud phones, applied to an extended terminal, comprising: The relevant data acquisition module is used to acquire the physical environment data of the extended terminal when it receives the light field configuration start command issued by the main terminal, and upload the physical environment data to the main terminal. The driving instruction receiving module is used to adjust the corresponding programmable photonic crystal array to generate a specified target lattice structure according to the driving instruction when it receives the driving instruction returned by the main terminal after receiving the physical environment data, and to generate a corresponding light field distribution according to the target lattice structure in order to present the corresponding target display content.

[0027] To achieve the above objectives, embodiments of the present invention provide an extended display device for a cloud phone, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the extended display method for the cloud phone described above.

[0028] To achieve the above objectives, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the above-described extended display method of the cloud phone.

[0029] To achieve the above objectives, embodiments of the present invention also provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the extended display method for cloud phones described above.

[0030] Compared with existing technologies, the extended display method, apparatus, device, medium, and product for cloud phones disclosed in this invention, when receiving physical environment data uploaded by at least one extended terminal, extracts features from the current target display content to obtain a content feature matrix of the target display content; generates a corresponding driving instruction for the extended terminal based on the content feature matrix and the physical environment data; and sends the driving instruction to the corresponding extended terminal, so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array to generate a specified target lattice structure according to the driving instruction, and generates a corresponding light field distribution based on the target lattice structure to present the corresponding target display content. The photonic topology mapping technology generates a directional light field by controlling the equivalent refractive index through an electric field, avoiding the indiscriminate superposition of display light and ambient light, solving the contrast attenuation problem caused by ambient light interference, and improving the user experience. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating an extended display method for a cloud phone provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating another extended display method for cloud phones provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an extended display device for a cloud phone provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of another cloud phone extended display device provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of an extended display device for a cloud phone provided in an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] It should be noted that the terms "comprising" and "specific" in this invention, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0034] Please see Figure 1 , Figure 1 This is a flowchart illustrating an extended display method for a cloud phone according to an embodiment of the present invention. The extended display method for the cloud phone is applied to a main terminal, and the method includes: S11, when physical environment data uploaded by at least one extended terminal is received, feature extraction is performed on the current target display content to obtain the content feature matrix of the target display content; S12, Generate driving instructions for the corresponding extended terminal based on the content feature matrix and the physical environment data; S13, the driving instruction is sent to the corresponding extended terminal so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array according to the driving instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0035] For example, the main terminal is a smartphone, and the extended terminal is a tablet device. After the user enables the cloud phone's multi-terminal extended display function, the tablet collects physical environment data (its own spatial coordinates, surface curvature, ambient light intensity, etc.) through its built-in sensors (gyroscope, ambient light sensor, millimeter-wave radar) and uploads it to the phone; the phone extracts features from the target display content (8K video) and generates a content feature matrix through CNN (Convolutional Neural Network); the phone combines the content feature matrix with the physical environment data to calculate the tablet's photonic crystal driving matrix, generates driving commands, and sends them out; the tablet adjusts the programmable photonic crystal array according to the driving commands to generate the target lattice structure (lattice constant, applied bias voltage). This structure regulates the generation of incident light and the light field distribution that is continuous with the phone screen, ultimately presenting the extended part of the video on the tablet surface, achieving seamless display connection between the phone and the tablet. Breaking through the limitations of traditional physical screens, this technology achieves display medium generalization through a programmable photonic crystal array, supporting continuous image display on extended terminals of different forms (flat panels, curved devices) and materials. Based on content characteristics and physical environment data, it dynamically generates driving instructions to ensure precise matching of light field distribution with target content and terminal status, solving the problems of image fragmentation and distortion in traditional extended displays. The fast light field generation response speed meets the expansion requirements of 8K@120fps high frame rate content, ensuring the continuity of the user's visual experience. Utilizing the properties of photonic crystal materials, a programmable optical layer is constructed on the surface of the extended terminal, achieving screen shape adaptation through electric field control. When a new device is detected, a light field distribution matching the current content is automatically generated, achieving seamless physical extension.

[0036] Specifically, step S12 includes: S121, calculate the virtual space coordinates of the corresponding extended terminal to adapt to the target display content based on the content feature matrix and the physical environment data; S122, Generate driving instructions for the corresponding extended terminal based on the virtual space coordinates and the content feature matrix.

[0037] For example, based on the content feature matrix and physical environment parameters, the virtual spatial coordinates of the target display content of the corresponding extended terminal are calculated through three-dimensional affine transformation. Based on the virtual spatial coordinates and the content feature matrix, a driving instruction containing lattice structure parameters and voltage control commands is generated and sent to the extended terminal. Through virtual spatial coordinate mapping, a precise correlation between content features and the physical state of the terminal is achieved, avoiding light field misalignment caused by changes in terminal placement or orientation. With coordinate mapping and feature fusion as the core, the computational complexity of the main terminal is reduced, ensuring a driving instruction generation latency of <1ms. It adapts to heterogeneous multi-terminal scenarios; regardless of the spatial position or shape of the extended terminal, unified light field scheduling can be achieved through coordinate calculation. The calculation formula for spatial coordinate mapping is: , In the formula, To extend the virtual space coordinates of the terminal; Let be the content feature matrix, where Display content for the target; The rotation matrix (calculated from the device's gyroscope data) Indicates the extended terminal is in , , (rotation angle of the axis) To extend the actual spatial coordinates of the terminal, three-dimensional coordinates ; For displacement vectors (millimeter-level accuracy), three-dimensional vectors .

[0038] More specifically, step S122 includes: S1221, Calculate the target phase of the corresponding extended terminal based on the virtual space coordinates and the content feature matrix; S1222, calculate the photonic crystal driving matrix of the corresponding extended terminal based on the target phase, and generate the driving command of the corresponding extended terminal based on the photonic crystal driving matrix.

[0039] For example, based on the virtual space coordinates and content feature matrix, the target phase is calculated using the phase modulation calculation formula; then, using the electric field drive optimization calculation formula, the photonic crystal drive matrix (containing the voltage control values ​​of each lattice unit) is calculated; this drive matrix is ​​encapsulated into drive commands and sent to the extended terminal. The drive matrix is ​​accurately derived based on the target phase, ensuring that the light field generated by the photonic crystal matches the phase of the target content, with brightness uniformity >92%; voltage constraints are introduced (…). This avoids damage to the photonic crystal due to overvoltage, improving the stability of equipment operation; the phase calculation and driving matrix generation processes are highly efficient, and with hardware acceleration, they meet the low-latency requirements of real-time extended displays. The phase modulation calculation formula is as follows: , , The calculation formula for electric field-driven optimization is as follows: st , In the formula, The phase distribution of the target light field; This is the integration region; Display content for the target The grayscale / brightness value of a pixel; Let be the optical field transfer function; The imaginary unit; The incident light wavelength is in nanometers; The refractive index of air; This is the distance light travels in the air; For photonic crystals in Refractive index deviation at position; The thickness of the photonic crystal; This represents the driving voltage matrix of the photonic crystal; For voltage The corresponding photonic crystal driving matrix; The target phase matrix; The regularization coefficient is denoted by ; For the first Line 1 The applied voltage of the column lattice unit.

[0040] Specifically, step S13 includes: S131, the driving instruction is sent to the corresponding extended terminal so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array to generate the specified adjustment lattice structure according to the driving instruction; S132, when receiving the actual optical field data uploaded by the corresponding extended terminal after adjusting the corresponding programmable photonic crystal array, calibrating the photonic crystal driving matrix of the corresponding extended terminal according to the actual optical field data, and generating the calibration command of the corresponding extended terminal; S133, the calibration command is sent to the corresponding extended terminal so that the corresponding extended terminal calibrates the corresponding adjustment lattice structure according to the calibration command, generates the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0041] For example, after receiving the driving command, the extended terminal adjusts the programmable photonic crystal array to generate an adjustable lattice structure and collects actual light field data through a miniature spectral sensor. ;Will Uploaded to the phone, the phone calculates the phase deviation using residuals, and then uses the learning rate... Update driver matrix The calibration command is generated and issued; the tablet adjusts its lattice structure according to the calibration command, fine-tunes the voltage to 13.2V, and generates the target lattice structure. The final light field distribution has a splicing error of less than 0.1 pixels with the mobile phone screen. An actual light field feedback calibration mechanism is introduced to solve the problem of light field distortion caused by environmental interference (such as light reflection and temperature changes), improving display accuracy. The closed-loop calibration process is short-time and does not affect the real-time display of high frame rate content, ensuring a continuous user experience. It reduces the dependence on the consistency of extended terminal hardware; even if there are manufacturing errors in the photonic crystals of different terminals, a unified light field effect can be achieved through calibration. The residual calculation formula is as follows: , In the formula, This refers to the phase deviation (phase residual). For Fast Fourier Transform; For inverse Fast Fourier Transform; The target phase matrix.

[0042] More specifically, step S133 includes: S1331, The calibration command is sent to the corresponding extended terminal so that the corresponding extended terminal calibrates the corresponding adjustment lattice structure according to the calibration command and generates the specified calibration lattice structure. S1332, when the positioning deviation uploaded by the corresponding extended terminal after calibrating the corresponding adjusted lattice structure is received, the photonic crystal driving matrix after calibration of the corresponding extended terminal is fine-tuned according to the positioning deviation, and the fine-tuning instruction of the corresponding extended terminal is generated. S1333, the fine-tuning instruction is sent to the corresponding extended terminal so that the corresponding extended terminal fine-tunes the corresponding calibration lattice structure according to the fine-tuning instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0043] For example, after the flat plate generates a calibration lattice structure according to the calibration instructions, the positioning deviation is collected by the embedded CdSe / ZnS (cadmium selenide / zinc sulfide) quantum dot array. The data is then uploaded to the mobile phone; the mobile phone fine-tunes the driving matrix based on the positioning deviation and generates fine-tuning instructions; after receiving the fine-tuning instructions, the tablet fine-tunes the calibration lattice structure to generate the target lattice structure, ultimately achieving a light field positioning accuracy of [missing information]. The visual disconnect between different devices has completely disappeared. For example, a mobile phone (main device) can extend its video display to a tablet (extended device), using millimeter-wave radar to identify the curvature (radius) of the tablet's surface. =300mm), based on the dynamic characteristics of the video content (motion vector) (15 pixels / frame) Calculate the required refractive index gradient A voltage of V=12.7V is applied to the corresponding region to drive the generation of an asymmetric lattice. This achieves a 120° viewing angle extension and brightness uniformity >92% (compared to <75% in traditional solutions). The nonlinear lattice control algorithm addresses the light field distortion problem of curved media by introducing fractional Fourier transform. Quantum dot-assisted calibration embeds a CdSe / ZnS quantum dot array in the photonic crystal layer, achieving sub-pixel-level calibration through its excitation spectrum (FWHM <25nm). This solution, through material innovation and algorithmic breakthroughs, frees screen expansion from physical screen limitations, achieving a display medium transition delay <2.1ms and a continuous display effect imperceptible to the human eye.

[0044] , The formula for calculating the fractional Fourier transform is: , The calculation formula for quantum dot-assisted calibration is as follows: / / Positioning accuracy reaches , In the formula, For function of The result of the fractional Fourier transform; The spatial distribution function of the input light field; For spatial coordinate variables; For coordinate variables in the transformation domain; This is due to optical field positioning deviation; The peak wavelength of the excitation spectrum of the quantum dot; The reference wavelength for the target light field; This represents the Rayleigh scattering wavelength width. Sub-pixel-level fine-tuning assisted by quantum dots further enhances the optical field positioning accuracy, resolving the splicing gap problem in traditional extended displays; the positioning deviation feedback and fine-tuning process are automated, requiring no manual user intervention and improving operational convenience.

[0045] Furthermore, before receiving physical environment data uploaded by at least one extended terminal, the method further includes: S101, When the cloud phone is detected to have enabled extended display, the device type code is broadcast within a preset distance; S102, when receiving the authentication response returned by the terminal to be extended after receiving the device type code, a capability negotiation request is generated according to the authentication response, and the capability negotiation request is sent to the terminal to be extended so that the terminal to be extended can generate a data certificate signed by the terminal to be extended according to the capability negotiation request. S103, when the terminal to be extended returns a signed data certificate after receiving the capability negotiation request, verify the signature validity of the data certificate and send the certificate chain root verification request of the data certificate to the cloud. S104, when the verification result returned after receiving the cloud verification certificate chain root, calculate the node weight between the terminal to be expanded and the main terminal based on the verification result, as well as the device distance and bandwidth between the terminal to be expanded and the main terminal, and generate the device topology diagram of the main terminal. S105, the authentication pass instruction is sent to the terminal to be expanded in the device topology diagram, and the device capability fingerprint database of the main terminal is established according to the terminal to be expanded in the device topology diagram.

[0046] For example, when the main terminal (mobile phone) enables the extended display function of the cloud phone, a dual-band hybrid discovery protocol is used. The mobile phone triggers device wake-up by broadcasting the device type code (TypeCode=0x8F3A) via Bluetooth Mesh. For instance, after receiving the code value, the terminal to be extended (tablet, flexible display device) within a preset distance of 10 meters activates its own Bluetooth Mesh and WiFi P2P modules and returns a pending authentication response to the main terminal, including basic device identifiers (such as device model). After receiving the response, the main terminal actively establishes a 5GHz high-frequency channel for WiFi P2P (WiFi Peer-to-Peer, WiFi point-to-point (communication)) (802.11ax standard) and sends capability negotiation requests to each terminal to be extended, requesting the submission of device capability parameters. Each terminal to be extended needs to submit an X.509v3 digital certificate containing parameters such as device ID, GPU (Central Processing Unit) computing power (TFLOPS (Tera Floating-Point Operations Per Second) value), and screen PPI. The certificate chain is signed using the national cryptographic SM2-SM3 algorithm. After successful authentication, the node weights of the main terminal and each extended terminal are calculated, generating a dynamic device topology map. This process can complete cross-brand device interoperability authentication within 200ms. Heterogeneous terminal devices are discovered and authenticated using a low-power Bluetooth Mesh + WiFi P2P dual-channel protocol, enabling automatic discovery and secure authentication of cross-brand devices and establishing a device capability fingerprint database. A dual-band hybrid discovery protocol is adopted to achieve rapid automatic discovery of cross-brand devices, overcoming the device compatibility barriers of traditional multi-terminal collaboration. Certificate authentication and node weight calculation based on the national cryptographic SM2-SM3 algorithm ensure the security of device authentication and the rationality of connection priority, improving user operation efficiency. The formula for calculating node weight is as follows: , In the formula, Main terminal With extended terminal Node weights between; For distance attenuation factor, ; This is the bandwidth weighting coefficient. ; Main terminal With extended terminal The distance between devices (meter-level accuracy); Main terminal WiFi bandwidth (Mbps); Main terminal WiFi bandwidth.

[0047] Specifically, if the data channel between the main terminal and the corresponding extended terminal is a distributed rendering channel, then the method for constructing the distributed rendering channel includes: Based on the WebRTC protocol extension, a FOVEC proprietary protocol is constructed as the underlying communication standard for the data channel; A hierarchical transmission architecture is constructed based on a preset transmission strategy. A direct communication link for the data channel is established based on the high-frequency channel between the main terminal and the corresponding extended terminal. A distributed rendering channel is formed based on the underlying communication standard, the hierarchical transmission architecture, and the direct communication link.

[0048] For example, a hierarchical video stream transmission channel is established based on the FOVEC (Field of View Encoding Control) protocol, an improvement on WebRTC, supporting dynamic bitrate adaptation for 8K@120fps. The 8K image is divided into 144 dynamic tiles, each allocated a bitrate of 2-45Mbps based on the user's gaze point prediction (e.g., a high bitrate (45Mbps) is allocated to the center tile and a low bitrate (2Mbps) to the edge tiles). Motion compensation transmission using an extended version of H.266 / VVC (Versatile Video Coding) inter-frame prediction is employed to complete the encoding, improving motion vector accuracy to 1 / 128 pixel. The FEC (Forward Error Correction) redundancy packet ratio (5%-40%) is dynamically adjusted based on the Channel Quality Indicator (CQI) (levels 0-15). Actual testing shows that end-to-end latency of <8ms can be achieved on a 100Mbps network. For example, a distributed rendering channel is constructed between the main terminal (mobile phone) and extended terminals (tablets, laptops): Based on the WebRTC protocol extension, a customized FOVEC protocol is used as the underlying communication standard; a hierarchical transmission architecture is adopted, dividing 8K video into 144 dynamic tiles, and allocating bitrates (2-45Mbps) based on user gaze points; a direct communication link is established between the phone and tablets / laptops via a 5GHz high-frequency channel using the WiFi P2P 802.11ax standard; combining the FOVEC protocol, hierarchical transmission architecture, and high-frequency link forms a distributed rendering channel. The FOVEC protocol and hierarchical transmission architecture enable efficient transmission of high-bitrate content, solving the stuttering and latency issues of traditional protocols in 8K content transmission; the 5GHz high-frequency channel and direct communication link reduce network intermediate node forwarding, lowering transmission latency and meeting the low-latency requirements of games, AR, and other scenarios; dynamic bitrate adaptation and intelligent retransmission mechanisms are supported, improving transmission stability in complex network environments.

[0049] Furthermore, after constructing the distributed rendering pipeline, the method also includes: S106, acquire the user eye-tracking data of the target user, perform scene semantic segmentation and application type feature extraction on the target display content, and obtain the scene semantic segmentation result and application type feature of the target display content; S107, Generate a display weight matrix for the target display content based on the user eye-tracking data, the scene semantic segmentation result, and the application type features; S108, according to the display weight matrix, divide the primary and secondary display areas to generate rendering instructions for the corresponding extended terminals, and send the rendering instructions to the corresponding extended terminals so that the corresponding extended terminals return a rendering confirmation instruction to the primary terminal according to the rendering instructions. S109: When the corresponding extended terminal returns a confirmation instruction to prepare for rendering, and cross-terminal clock calibration is completed through the quantum entanglement timestamp synchronization algorithm, the target display content is started for synchronous output.

[0050] For example, an attention-aware algorithm is employed to automatically divide the primary and secondary display areas through eye tracking with the front-facing camera and usage scenario analysis. A Transformer-based visual attention model is deployed, with input data streams including eye tracking data, scene semantic segmentation results, and application type features (one-hot encoding of games / videos / documents). For instance, the main terminal's front-facing camera collects user eye tracking data at a 100Hz sampling rate (accuracy 0.5°), simultaneously running the DeepLabv3+ model (mIoU=92.3%) to complete game scene semantic segmentation (identifying characters, UI, and background areas), extracting application type features (game-type one-hot encoding), and loading a pre-trained Transformer visual attention model from the cloud onto the main terminal. A content weight matrix is ​​dynamically generated using a content weight calculation formula, and the primary display area is defined as... The continuous blocks are rendered with low bitrate (30% of the original quality) at the edges, and the content weight is adjusted. The calculation formula is: , In the formula, It is the Sigmoid activation function. For eye-tracking thermograms, Score the importance of the scene. Application type coefficient .

[0051] A time-stamp synchronization algorithm based on the principle of quantum entanglement ensures that multi-terminal touch commands are synchronized across terminals within 1μs. The synchronization process includes: the main terminal generating entangled photon pairs (1550nm wavelength), distributing them to each extended terminal via optical fiber, and measuring Bell state correlation to verify clock deviation. : , In the formula, For anti-association coincidence count, To associate the conformity count, For the total number of coincidences, It is the polarization angle; Touch events use the Lamport timestamp protocol, with the following global sorting formula: , In the formula, A global timestamp for the touch event; This is the terminal's local timestamp; This represents the calibrated clock offset.

[0052] In actual testing with 10 devices networked together, the standard deviation of touch latency was <0.3μs, meeting the requirement for tear-free cross-screen dragging. Based on the Transformer visual attention model, combined with eye tracking, scene semantic segmentation, and application type, it achieves intelligent content classification and improves the display quality of the user's core focus areas; dynamic division and synchronous output of primary and secondary display areas balance display quality and resource consumption, adapting to the needs of different application scenarios (games, videos, documents); the quantum entanglement principle synchronization algorithm is unaffected by network latency / jitter, avoiding synchronization errors caused by network transmission, and the clock calibration accuracy breaks through to the sub-microsecond level to eliminate the tearing effect of multi-screen touch operation.

[0053] Furthermore, after receiving the preparation for rendering confirmation instruction returned by the corresponding extended terminal, the method further includes: S110, when the rendering task allocation weight of the corresponding extended terminal is received from the edge node, the rendering task of the target display content is allocated to the corresponding extended terminal according to the rendering task allocation weight, so that the corresponding extended terminal can complete the corresponding rendering task; wherein, the rendering task allocation weight is calculated by the edge node according to the environmental state data of the extended terminal.

[0054] For example, an edge computing-driven power prediction model is deployed, dynamically allocating rendering tasks to terminals with the best energy efficiency ratio by combining parameters such as device remaining power and screen material; a power prediction model based on LSTM (Long Short-Term Memory) is constructed, with input features including: device remaining power (%), screen type (energy consumption coefficient of OLED (Organic Light-Emitting Diode) / LCD (Liquid Crystal Display) / e-ink (Electronic Ink)) and ambient light intensity (lux value); the output is the weight assigned to the rendering tasks; the edge computing-driven power prediction model achieves a balance between energy consumption and quality in task allocation, avoiding excessive load on a single terminal or waste of resources; the weights are dynamically calculated based on real-time environmental status data of the device, adapting to scenarios such as changes in terminal power and fluctuations in ambient light, improving the energy efficiency ratio of multi-terminal collaboration; the deep collaboration between task allocation and distributed rendering channels ensures that high-priority content is presented on high-performance terminals and low-priority content is presented on low-energy terminals, optimizing user experience and energy consumption balance. Among these, the weights assigned to the rendering tasks are... The calculation formula is: ， In the formula, To predict energy consumption (mW). Rate the rendering quality (0-1); It is a normalized exponential function.

[0055] Furthermore, after presenting the corresponding target display content, the method further includes: S14, when an offline extended terminal is detected, a state replica retrieval request is sent to the blockchain shard node, so that the blockchain shard node retrieves the most recent several state snapshot replicas of the offline extended terminal according to the state replica retrieval request and returns them to the main terminal; S15, when receiving several state snapshot copies returned by the blockchain sharding node, perform data integrity verification on the several state snapshot copies to generate a state snapshot corresponding to the offline extended terminal; S16, generate a content reorganization instruction based on the rendering task of the offline extended terminal, and send the content reorganization instruction and the status snapshot to other online extended terminals so that the online extended terminals can complete the content reorganization according to the content reorganization instruction and the status snapshot.

[0056] For example, the disaster recovery mechanism employs blockchain sharding technology (an improved blockchain architecture) to store snapshots of each terminal's state. When a single device goes offline, content reassembly can be achieved within 17ms. The improved blockchain architecture specifically includes: dividing device state information into 64-byte Merkle leaf nodes, aggregating them into 16 shard chains using BLS (Blockchain Signature Algorithm) signatures, and when a device goes offline, reassembly is achieved by retrieving the three nearest neighbors using the KademliaDHT protocol. A lightweight GKR protocol (GKR-Proof) is executed to verify computational integrity, achieving data verification with a single verification time of <5ms. In actual testing, a 10-node network can recover a 4K video stream within 17ms. Blockchain sharding technology enables distributed storage of terminal state snapshots, ensuring no data loss and solving the single point of failure problem of traditional disaster recovery solutions. The fast reassembly algorithm achieves content reassembly within 17ms, meeting the real-time requirements of high frame rate content and avoiding user experience interruptions. The lightweight GKR protocol ensures data integrity, reduces verification time, and improves the efficiency and reliability of disaster recovery. The calculation formula for the recombination algorithm is as follows: , In the formula, For disaster recovery time, Let D be the time complexity of sharded retrieval, S be the number of data shards, S be the shard size, and B be the bandwidth. This represents the number of shard nodes in the blockchain.

[0057] Furthermore, after constructing the distributed rendering pipeline, the method also includes: S17. When the model increment package is received from the cloud, the updated dual-loop deep reinforcement learning model is loaded according to the model increment package; wherein, the model increment package is obtained by the cloud through user behavior data periodically uploaded by each extended terminal, so that the cloud can train the dual-loop deep reinforcement learning model according to the user behavior data. S18, using the updated model, generate personalized extension instructions based on the current usage scenario of the target display content, and send the personalized extension instructions to the corresponding extension terminal so that the corresponding extension terminal adjusts the display layout and rendering parameters of the target display content according to the personalized extension instructions.

[0058] For example, a deep reinforcement learning model is embedded to construct a dual-loop reinforcement learning framework, continuously learning users' multi-screen usage habits and automatically generating personalized expansion strategies monthly, supporting OTA neural network architecture updates. The construction of the dual-loop reinforcement learning framework specifically includes: offline training of a 3D-CNN-LSTM hybrid model using user behavior logs (user behavior data, averaging 100GB per day); real-time generation of policy parameters through online inference, with the system automatically generating personalized configuration schemes monthly. The dual-loop reinforcement learning model and OTA incremental updates enable continuous iteration of personalized policies without requiring manual user settings, improving operational convenience; training the model based on user behavior data ensures policy generation aligns with user habits (e.g., game users prefer low latency and high bitrate), with personalized accuracy far exceeding traditional preset rules; differential neural network update technology reduces bandwidth consumption (incremental packet size is only 5%-10% of the full model), and safety threshold constraints prevent policy mutations, ensuring continuous experience. The objective function of the deep reinforcement learning model is: , In the formula, for User experience rating at any time for Energy consumption at any time Discount factor, Let it be the expected function; OTA (Over-the-Air) updates, employing differential neural network update technology, ensure that the transmitted incremental packet ΔW satisfies the following formula: ( (safety threshold) In the formula, These are the original model parameters before the update; This is the increment of the model parameters.

[0059] This invention discloses an extended display method for a cloud phone. Upon receiving physical environment data uploaded by at least one extended terminal, the method extracts features from the current target display content to obtain a content feature matrix. Based on the content feature matrix and the physical environment data, it generates a driving instruction for the corresponding extended terminal. The driving instruction is then sent to the corresponding extended terminal, causing it to adjust a programmable photonic crystal array to generate a specified target lattice structure and a corresponding light field distribution based on the target lattice structure, thereby presenting the target display content. Photonic topology mapping technology is used to generate a directional light field by controlling the equivalent refractive index through an electric field, avoiding the indiscriminate superposition of display light and ambient light, solving the contrast attenuation problem caused by ambient light interference, and improving the user experience.

[0060] SeeFigure 2 The figure is a schematic flowchart of another cloud phone extended display method provided by an embodiment of the present invention. This cloud phone extended display method is applied to an extended terminal, and the method includes: S21, when the optical field configuration start command is received from the main terminal, the physical environment data of the extended terminal is obtained and the physical environment data is uploaded to the main terminal; S22, when the main terminal receives the driving instruction returned after receiving the physical environment data, the corresponding programmable photonic crystal array is adjusted according to the driving instruction to generate the specified target lattice structure, and the corresponding light field distribution is generated according to the target lattice structure to present the corresponding target display content.

[0061] For example, the extended terminal is a smartwatch, and the main terminal is a mobile phone. After the mobile phone sends a start command for the light field configuration, the smartwatch collects physical environment data (spatial coordinates, surface curvature, ambient light intensity) through its built-in sensors and uploads it to the mobile phone. The smartwatch receives the drive command (including lattice constant and bias voltage) returned by the mobile phone and adjusts its own programmable photonic crystal array to generate the target lattice structure. This structure regulates the light field distribution generated by the incident light to be continuous with the mobile phone screen, displaying the extended content of the cloud phone's notification bar on the small screen of the smartwatch, with a brightness uniformity of 93%. The extended terminal actively collects physical environment data and responds to drive commands, realizing a collaborative mode of scheduling by the main terminal and execution by the extended terminal, adapting to miniaturized and irregularly shaped extended terminals. The dynamic adjustment capability of the programmable photonic crystal array allows the extended terminal to present a high-quality extended image without relying on high-specification hardware, reducing terminal hardware costs. The light field distribution is coordinated with the main terminal, solving the problem of the display content of small extended terminals (such as smartwatches) being disconnected from the main terminal, and improving the consistency of multi-terminal collaboration.

[0062] Specifically, in step S22, generating a corresponding light field distribution based on the target lattice structure to present the corresponding target display content includes: By controlling the optical parameters of the incident light through the target lattice structure, the incident light propagates in the air to generate a light field distribution that coordinates with the main terminal and other extended terminals, so as to present the corresponding target display content.

[0063] For example, a two-dimensional programmable photonic crystal array can be used. By applying a bias voltage of 0-20V, the duty cycle and lattice symmetry of the photonic crystal can be changed in real time. This enables dynamic control of the refractive index spatial gradient distribution, visible light band (380-780nm), and phase delay compensation of emitted light. Optical field synergy is achieved through optical parameter control, breaking through the limitations of traditional physical screen splicing. This supports ubiquitous display for flexible, wearable extended terminals, providing a wide-viewing-angle light field distribution to adapt to different viewing angles and enhance the visual experience of multi-terminal collaboration. Its lattice structure satisfies: , In the formula, The bandgap wavelength of the photonic crystal represents the target wavelength range that enables optical field manipulation. The lattice constant of the photonic crystal is tunable from 0.5 to 2 μm. This represents the equivalent refractive index of the photonic crystal (1.3-3.4, adjustable by electric field). and They represent the incident light at... shaft and Angular parameters in the axial direction.

[0064] Specifically, step S22 includes: S221, when the main terminal receives the driving instruction returned after receiving the physical environment data, the corresponding programmable photonic crystal array is adjusted according to the driving instruction to generate the specified adjustment lattice structure. S222, Obtain the actual light field data of the adjusted lattice structure and upload the actual light field data to the main terminal; S223, when the main terminal receives the calibration instruction returned after receiving the actual light field data, the corresponding adjustment lattice structure is calibrated according to the calibration instruction to generate the specified target lattice structure, and the corresponding light field distribution is generated according to the target lattice structure to present the corresponding target display content.

[0065] For example, after receiving the drive command from the main terminal, the extended terminal adjusts the programmable photonic crystal array to generate an adjusted lattice structure; it collects actual light field data through a built-in spectral sensor and uploads it to the main terminal; the main terminal calculates the residual and generates a calibration command to be issued; the extended terminal calibrates the lattice structure according to the calibration command, generates the target lattice structure, and finally generates a light field distribution on the extended terminal that is continuous with the image on the main terminal. The actual light field data feedback calibration mechanism solves the problem of image distortion caused by projection angle and ambient light interference, improving display accuracy; the calibration process is automated, eliminating the need for users to manually adjust projection parameters, improving operational convenience; and it is compatible with non-standard display media such as desktops and walls, expanding the application scenarios of multi-terminal displays.

[0066] More specifically, step S223 includes: S2231, when receiving a calibration command returned by the main terminal after receiving the actual light field data, the corresponding adjustment lattice structure is calibrated according to the calibration command to generate the specified calibration lattice structure; S2232, Obtain the positioning deviation of the calibration lattice structure and upload the positioning deviation to the main terminal; S2233, when the main terminal receives the fine-tuning instruction returned after receiving the positioning deviation, the corresponding calibration lattice structure is fine-tuned according to the fine-tuning instruction to generate the specified target lattice structure, and the corresponding light field distribution is generated according to the target lattice structure to present the corresponding target display content.

[0067] For example, after the extended terminal generates a calibration lattice structure according to the calibration instructions, it collects the positioning deviation through a quantum dot array and uploads it to the main terminal. The main terminal generates a fine-tuning instruction based on the positioning deviation. After receiving the instruction, the extended terminal fine-tunes the lattice structure to generate the target lattice structure. Ultimately, the positioning accuracy of the extended terminal's image reaches 0.02λ, and the splicing error with the main terminal's image is <0.1 pixels, resulting in a visually seamless appearance. Sub-pixel-level fine-tuning further improves the positioning accuracy of the projected image, solving the splicing gap problem of traditional projection extensions. It ensures that the light field distribution is always accurately coordinated with the main terminal, enhancing the user experience.

[0068] Furthermore, before receiving the optical field configuration start command from the main terminal, the method further includes: S201, when the device type code broadcast by the main terminal is received, an authentication response is generated according to the device type code, and the authentication response is uploaded to the main terminal so that the main terminal generates a capability negotiation request according to the authentication response. S202, when the main terminal receives the capability negotiation request returned after receiving the authentication response, a data certificate to be signed by the terminal to be extended is generated according to the capability negotiation request, the signed data certificate is uploaded to the main terminal so that the main terminal can verify the signature validity of the data certificate, and the certificate chain root verification request of the data certificate is sent to the cloud. S203, when receiving the authentication pass instruction returned by the main terminal after receiving the signed data certificate, it becomes an extended terminal of the main terminal.

[0069] For example, the main terminal broadcasts the device type code (TypeCode=0x8F3A). Upon receiving this, the extended terminal generates an authentication response and uploads it. The main terminal then sends a capability negotiation request. Based on the request, the extended terminal generates an X.509v3 digital certificate (signed using the SM2-SM3 algorithm) containing the device ID, GPU computing power of 3.0 TFLOPS, and screen PPI of 326, and uploads it to the main terminal. After verifying the validity of the certificate signature, the main terminal sends a certificate chain root verification request to the cloud. The cloud returns a verification success result. The main terminal then issues an authentication success command, and the extended terminal becomes an extended terminal of the main terminal, completing the authentication access for cross-brand devices. The standardized device discovery and authentication process enables compatible access for cross-brand and cross-system devices, breaking down the traditional ecosystem barriers of multi-terminal collaboration. The digital certificate signed with the national cryptographic algorithm ensures authentication security, avoiding the risks of device forgery and data leakage. The automated and time-saving authentication process improves user efficiency and lowers the barrier to entry for multi-terminal collaboration.

[0070] Furthermore, before receiving the optical field configuration start command from the main terminal, the method further includes: S204, when the rendering instruction sent by the main terminal is received, a rendering preparation confirmation instruction is generated according to the rendering instruction, and the rendering preparation confirmation instruction is uploaded to the main terminal, so that when the main terminal receives the rendering preparation confirmation instruction, it can start the target display content for synchronous output after completing cross-terminal clock calibration through the quantum entanglement timestamp synchronization algorithm.

[0071] For example, the main terminal generates a rendering command and sends it to the extended terminal. Upon receiving the command, the extended terminal checks its GPU computing power and network bandwidth to ensure they meet the rendering requirements, generates a rendering preparation confirmation command, and uploads it. Upon receiving this command, the main terminal uses a quantum entanglement timestamp synchronization algorithm to distribute 1550nm entangled photon pairs to the extended terminal, calibrating the clock deviation. After completing cross-terminal clock calibration, it starts synchronized video output, with a synchronization delay of <0.3μs between the extended and main terminals. This rendering preparation confirmation mechanism ensures that the extended terminal has rendering capabilities before initiating synchronized output, avoiding screen stuttering or interruptions due to insufficient terminal performance. The quantum entanglement timestamp synchronization algorithm achieves μs-level clock calibration, resolving the time difference issue in cross-terminal synchronization and ensuring consistent screen output.

[0072] Furthermore, after uploading the preparation rendering confirmation instruction to the main terminal, the method further includes: S205, when the rendering task assigned by the main terminal is received, the rendering task is completed.

[0073] For example, the main terminal allocates 55% of the edge tile rendering tasks in the 8K video to the extended terminal based on the rendering task allocation weights issued by the edge nodes. After receiving the tasks, the extended terminal uses its own GPU computing power to complete tile decoding and rendering, generates rendering results, and feeds them back to the main terminal through a distributed rendering channel. The rendering time is less than 5ms, with no stuttering or frame drops. The extended terminal undertakes rendering tasks on demand, making full use of terminal hardware resources and reducing the computing pressure on the main terminal. The rendering tasks, distributed rendering channels, and task allocation weights work in deep coordination to ensure a balance between rendering efficiency and image quality. It supports distributed rendering of high bitrate and high frame rate content, meeting the needs of high-end scenarios such as 8K@120fps and improving the performance ceiling of cloud phones.

[0074] Furthermore, after completing the rendering task, the method further includes: S23, when the content reorganization instruction and status snapshot are received from the main terminal, the content reorganization is completed according to the content reorganization instruction and the status snapshot.

[0075] For example, after extended terminal A completes its rendering task, another extended terminal B in the same scene goes offline due to network interruption. The main terminal generates a content reassembly instruction (requiring extended terminal A to take over the 30% tile rendering task originally undertaken by extended terminal B) and a state snapshot of extended terminal B, and sends it to extended terminal A. After receiving the instruction, extended terminal A, based on its already completed 55% task, quickly allocates GPU resources and completes the reassembly rendering of the remaining 30% of the task within 17ms, resulting in uninterrupted and distortion-free video playback. Extended terminals support content reassembly, ensuring uninterrupted multi-screen collaboration when a single terminal is offline, improving system fault tolerance; the reassembly mechanism based on state snapshots ensures the consistency between the reassembly content and the original terminal's rendered content, avoiding screen fragmentation and content loss; the fast reassembly response speed (<17ms) meets the real-time requirements of high frame rate content, improving the continuity of the user experience.

[0076] Furthermore, after completing content reconstruction according to the content reconstruction instruction and the state snapshot, the method includes: S24, When the main terminal is connected, construct the hardware resource pool of the extended terminal; S25, when an authentication request from a target user is received, the biometric data of the target user is obtained; the biometric data is uploaded to the FPGA so that the FPGA can authenticate the target user based on the biometric data and return the authentication result to the extended terminal; S26. When the verification result returned by the FPGA is received, if the verification is successful, the user account instance is loaded from the cloud by decrypting it with a key. The key is dynamically generated based on the biometric verification result and is automatically updated periodically.

[0077] For example, an isolated privacy sandbox is constructed, implementing multi-user zero-trust isolation at the FPGA (Field-Programmable Gate Array) hardware layer. Each extended terminal can independently run different account instances, supporting iris + vein composite biometric verification. For instance, the hardware layer, implemented using a Xilinx Versal FPGA, specifically includes: allocating an independent programmable logic block (CLB) for each user instance; hard isolation of memory channels to achieve physical isolation; using a composite biometric fusion algorithm to perform zero-trust verification on the target user (iris recognition uses a residual network (ResNet-152, FRR=0.001%), vein recognition uses near-infrared imaging); the data bus uses AES-GCM-SIV (Advanced Encryption Standard-Galois / Counter Mode-Synthetic Initialization Vector) for secure transmission, with a 30-second key rotation cycle; and a side-channel attack resistant design that passes FIPS-140-3 (Federal Information Processing Standard). Standards (Federal Information Processing Standard) certification; FPGA hardware-level isolated resource pools achieve physical isolation of multiple user account instances, preventing cross-access of data and ensuring privacy and security; iris and vein combined biometric verification with a false rejection rate of 0.001%, far exceeding the security of traditional passwords and single biometric authentication; dynamically generated and periodically rotated key mechanism resists eavesdropping and tampering attacks, complying with the FIPS-140-3 security standard. The calculation formula for the combined biometric fusion algorithm is: , In the formula, The total score is matched based on the composite biometrics. For iris recognition matching accuracy; For vein recognition matching accuracy.

[0078] Furthermore, after uploading the preparation rendering confirmation instruction to the main terminal, the method further includes: S27, when a personalized extension instruction is received from the main terminal, the display layout and rendering parameters of the target display content are adjusted according to the personalized extension instruction.

[0079] This invention discloses another method for extending the display of a cloud phone. Upon receiving a light field configuration activation command from the main terminal, the method acquires the physical environment data of the extended terminal and uploads the physical environment data to the main terminal. Upon receiving a drive command returned by the main terminal after receiving the physical environment data, the method adjusts the corresponding programmable photonic crystal array according to the drive command to generate a specified target lattice structure, and generates a corresponding light field distribution based on the target lattice structure to present the corresponding target display content. The dynamic adjustment capability of the programmable photonic crystal array allows the extended terminal to present a high-quality extended image without relying on high-specification hardware, reducing terminal hardware costs. The light field distribution works in conjunction with the main terminal, solving the problem of the display content being disconnected from the main terminal in small extended terminals (such as watches), and improving the consistency of multi-terminal collaboration.

[0080] See Figure 3 , Figure 3 This is a schematic diagram of the structure of an extended display device 10 for a cloud phone according to an embodiment of the present invention. The extended display device 10 for the cloud phone is applied to the main terminal and includes: The content feature extraction module 11 is used to extract features from the current target display content to obtain the content feature matrix of the target display content when receiving physical environment data uploaded by at least one extended terminal; The driver instruction generation module 12 is used to generate driver instructions for the corresponding extended terminal based on the content feature matrix and the physical environment data. The driving instruction issuing module 13 is used to issue the driving instruction to the corresponding extended terminal, so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array according to the driving instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

[0081] Furthermore, the extended display device 10 of the cloud phone also includes: The candidate device creation module is used for: When a cloud phone is detected to have activated an extended display command, the device type code is broadcast within a preset distance. When an authentication response is received from the terminal to be extended after receiving the device type code, a capability negotiation request is generated based on the authentication response, and the capability negotiation request is sent to the terminal to be extended so that the terminal to be extended can generate a data certificate signed by the terminal to be extended based on the capability negotiation request. When the terminal to be extended receives a signed data certificate after receiving the capability negotiation request, the signature validity of the data certificate is verified, and the certificate chain root verification request of the data certificate is sent to the cloud. When the verification result returned after receiving the cloud verification certificate chain root is received, the node weight between the terminal to be expanded and the main terminal is calculated based on the verification result, as well as the device distance and bandwidth between the terminal to be expanded and the main terminal, and the device topology diagram of the main terminal is generated. The authentication pass command is sent to the terminals to be expanded in the device topology diagram, and the device capability fingerprint database of the main terminal is established based on the terminals to be expanded in the device topology diagram.

[0082] Furthermore, the extended display device 10 of the cloud phone also includes: The rendering clock calibration module is used for: Obtain the eye-tracking data of the target user, perform scene semantic segmentation and application type feature extraction on the target display content, and obtain the scene semantic segmentation result and application type feature of the target display content; A display weight matrix for the target display content is generated based on the user's eye-tracking data, the scene semantic segmentation results, and the application type features; The primary and secondary display areas are divided according to the display weight matrix to generate rendering instructions for the corresponding extended terminals. The rendering instructions are then sent to the corresponding extended terminals so that the corresponding extended terminals can return a rendering confirmation instruction to the primary terminal based on the rendering instructions. Upon receiving the preparation rendering confirmation instruction from the corresponding extended terminal, and after completing cross-terminal clock calibration through the quantum entanglement timestamp synchronization algorithm, the target display content is started for synchronous output.

[0083] Furthermore, the extended display device 10 of the cloud phone also includes: The rendering task allocation module is used to allocate the rendering task of the target display content to the corresponding extended terminal according to the rendering task allocation weight issued by the edge node when it receives the rendering task allocation weight of the corresponding extended terminal, so that the corresponding extended terminal can complete the corresponding rendering task; wherein, the rendering task allocation weight is calculated by the edge node according to the environmental state data of the extended terminal.

[0084] Furthermore, the extended display device 10 of the cloud phone also includes: The content reorganization module is used for: When an offline extended terminal is detected, a state replica retrieval request is sent to the blockchain shard node, so that the blockchain shard node retrieves the most recent several state snapshot replicas of the offline extended terminal according to the state replica retrieval request and returns them to the main terminal. When receiving several state snapshot copies returned by the blockchain shard node, perform data integrity verification on the several state snapshot copies to generate the state snapshot corresponding to the offline extended terminal; The offline extended terminal generates a content reorganization instruction based on its rendering task, and sends the content reorganization instruction and the status snapshot to other online extended terminals so that the online extended terminals can complete the content reorganization according to the content reorganization instruction and the status snapshot.

[0085] Furthermore, the extended display device 10 of the cloud phone also includes: The learning model optimization module is used for: When a model increment package is received from the cloud, the updated dual-loop deep reinforcement learning model is loaded according to the model increment package; wherein, the model increment package is obtained by the cloud through user behavior data periodically uploaded by each extended terminal to the cloud, so that the cloud can train the dual-loop deep reinforcement learning model according to the user behavior data; The updated model is used to generate personalized extension instructions based on the current usage scenario of the target display content. The personalized extension instructions are then sent to the corresponding extension terminal so that the corresponding extension terminal can adjust the display layout and rendering parameters of the target display content according to the personalized extension instructions.

[0086] The cloud phone extended display device 10 provided in this embodiment of the invention can realize all the processes of the cloud phone extended display method of the above embodiment. The functions and technical effects of each module in the device are the same as the functions and technical effects of the cloud phone extended display method of the above embodiment, and will not be repeated here.

[0087] See Figure 4 , Figure 4 This is a schematic diagram of the structure of another cloud phone extended display device 20 provided in an embodiment of the present invention. The cloud phone extended display device 20 is applied to an extended terminal and includes: The relevant data acquisition module 21 is used to acquire the physical environment data of the extended terminal when it receives the light field configuration start command issued by the main terminal, and upload the physical environment data to the main terminal. The driving instruction receiving module 22 is used to adjust the corresponding programmable photonic crystal array to generate a specified target lattice structure according to the driving instruction when it receives the driving instruction returned by the main terminal after receiving the physical environment data, and to generate a corresponding light field distribution according to the target lattice structure in order to present the corresponding target display content.

[0088] Furthermore, the extended display device 20 of the cloud phone also includes: Extended terminal authentication module, used for: When the device type code broadcast by the main terminal is received, an authentication response is generated based on the device type code and uploaded to the main terminal so that the main terminal can generate a capability negotiation request based on the authentication response. When the main terminal receives the capability negotiation request returned after receiving the authentication response, a data certificate to be signed by the terminal to be extended is generated according to the capability negotiation request. The signed data certificate is uploaded to the main terminal so that the main terminal can verify the signature validity of the data certificate and send the certificate chain root verification request of the data certificate to the cloud. When a terminal receives an authentication pass instruction returned by the main terminal after receiving a signed data certificate, it becomes an extended terminal of the main terminal.

[0089] Furthermore, the extended display device 20 of the cloud phone also includes: The rendering instruction receiving module is used to generate a rendering preparation confirmation instruction based on the rendering instruction when it receives a rendering instruction from the main terminal, and upload the rendering preparation confirmation instruction to the main terminal so that when the main terminal receives the rendering preparation confirmation instruction, it can start the synchronous output of the target display content after completing cross-terminal clock calibration through the quantum entanglement timestamp synchronization algorithm.

[0090] Furthermore, the extended display device 20 of the cloud phone also includes: The rendering task completion module is used to complete the rendering task when it receives the rendering task assigned by the main terminal.

[0091] Furthermore, the extended display device 20 of the cloud phone also includes: The reassembly instruction receiving module is used to complete content reassembly according to the content reassembly instruction and the status snapshot when it receives the content reassembly instruction and the status snapshot issued by the main terminal.

[0092] Furthermore, the extended display device 20 of the cloud phone also includes: The account instance loading module is used for: When the main terminal is accessed, a hardware resource pool for the extended terminal is constructed. When an authentication request from a target user is received, the biometric data of the target user is obtained; the biometric data is uploaded to the FPGA so that the FPGA can authenticate the target user based on the biometric data and return the authentication result to the extended terminal. When the verification result returned by the FPGA is received, if the verification is successful, the user account instance is decrypted from the cloud using a key. The key is dynamically generated based on the biometric verification result and is automatically updated periodically.

[0093] Furthermore, the extended display device 20 of the cloud phone also includes: The extended instruction receiving module is used to adjust the display layout and rendering parameters of the target display content according to the personalized extended instruction issued by the main terminal when it receives the personalized extended instruction.

[0094] The extended display device 20 for a cloud phone provided in this embodiment of the invention can implement all the processes of the extended display method for a cloud phone in the above embodiment. The functions and technical effects of each module in the device are the same as those of the extended display method for a cloud phone in the above embodiment, and will not be repeated here.

[0095] See Figure 5 , Figure 5 This is a schematic diagram of the structure of an extended display device 30 for a cloud phone according to an embodiment of the present invention. The extended display device 30 for the cloud phone in this embodiment includes: a processor 31, a memory 32, and a computer program stored in the memory 32 and executable on the processor 31. When the processor 31 executes the computer program, it implements the steps in the above-described embodiment of the extended display method for a cloud phone. Alternatively, when the processor 31 executes the computer program, it implements the functions of each module in the above-described embodiment of the extended display device for a cloud phone.

[0096] For example, the computer program may be divided into one or more modules, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the extended display device 30 of the cloud phone.

[0097] The extended display device 30 of the cloud phone can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The extended display device 30 of the cloud phone may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will understand that the schematic diagram is merely an example of the extended display device 30 of the cloud phone and does not constitute a limitation on the extended display device 30 of the cloud phone. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the extended display device 30 of the cloud phone may also include input / output devices, network access devices, buses, etc.

[0098] The processor 31 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 31 is the control center of the cloud phone's extended display device 30, connecting various parts of the cloud phone's extended display device 30 via various interfaces and lines.

[0099] The memory 32 can be used to store the computer programs and / or modules. The processor 31 implements various functions of the extended display device 30 of the cloud phone by running or executing the computer programs and / or modules stored in the memory 32 and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the phone (such as audio data, phonebook, etc.). In addition, the memory 32 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0100] If the module integrated into the extended display device 30 of the cloud phone is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 31, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0101] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0102] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the extended display method of the cloud phone as described in the above embodiments.

[0103] Furthermore, embodiments of the present invention also provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the extended display method for cloud phones described above.

[0104] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for extending the display of a cloud phone, characterized in that, Applied to the main terminal, the method includes: When physical environment data uploaded by at least one extended terminal is received, feature extraction is performed on the current target display content to obtain the content feature matrix of the target display content; The corresponding extended terminal driving instructions are generated based on the content feature matrix and the physical environment data. The driving command is sent to the corresponding extended terminal, so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array according to the driving command to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

2. The extended display method for cloud phones as described in claim 1, characterized in that, The step of generating the corresponding extended terminal's driver instructions based on the content feature matrix and the physical environment data includes: Calculate the virtual space coordinates of the corresponding extended terminal to adapt to the target display content based on the content feature matrix and the physical environment data; The corresponding extended terminal driving instructions are generated based on the virtual space coordinates and the content feature matrix.

3. The extended display method for cloud phones as described in claim 2, characterized in that, The step of generating the corresponding extended terminal's driving instructions based on the virtual space coordinates and the content feature matrix includes: Calculate the target phase of the corresponding extended terminal based on the virtual space coordinates and the content feature matrix; The photonic crystal driving matrix of the corresponding extended terminal is calculated based on the target phase, and the driving command of the corresponding extended terminal is generated based on the photonic crystal driving matrix.

4. The extended display method for cloud phones as described in claim 1, characterized in that, The step of sending the driving command to the corresponding extended terminal, so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array according to the driving command to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content, includes: The driving command is sent to the corresponding extended terminal so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array to generate the specified adjustment lattice structure according to the driving command. When the actual optical field data uploaded by the corresponding extended terminal after adjusting the corresponding programmable photonic crystal array is received, the photonic crystal driving matrix of the corresponding extended terminal is calibrated according to the actual optical field data, and a calibration instruction for the corresponding extended terminal is generated. The calibration command is sent to the corresponding extended terminal, so that the corresponding extended terminal calibrates the corresponding adjustment lattice structure according to the calibration command, generates the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

5. The extended display method for cloud phones as described in claim 4, characterized in that, The step of sending the calibration command to the corresponding extended terminal, so that the corresponding extended terminal calibrates the corresponding adjustment lattice structure according to the calibration command to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content, includes: The calibration command is sent to the corresponding extended terminal so that the corresponding extended terminal calibrates the corresponding adjustment lattice structure according to the calibration command and generates the specified calibration lattice structure. When the positioning deviation uploaded by the corresponding extended terminal after calibrating the corresponding adjusted lattice structure is received, the photonic crystal driving matrix after calibration of the corresponding extended terminal is fine-tuned according to the positioning deviation, and the fine-tuning instruction of the corresponding extended terminal is generated. The fine-tuning command is sent to the corresponding extended terminal, so that the corresponding extended terminal fine-tunes the corresponding calibration lattice structure according to the fine-tuning command to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

6. The extended display method for cloud phones as described in claim 1, characterized in that, Before receiving physical environment data uploaded by at least one extended terminal, the method further includes: When a cloud phone is detected to have activated an extended display command, the device type code is broadcast within a preset distance. When an authentication response is received from the terminal to be extended after receiving the device type code, a capability negotiation request is generated based on the authentication response, and the capability negotiation request is sent to the terminal to be extended so that the terminal to be extended can generate a data certificate signed by the terminal to be extended based on the capability negotiation request. When the terminal to be extended receives a signed data certificate after receiving the capability negotiation request, the signature validity of the data certificate is verified, and the certificate chain root verification request of the data certificate is sent to the cloud. When the verification result returned after receiving the cloud verification certificate chain root is received, the node weight between the terminal to be expanded and the main terminal is calculated based on the verification result, as well as the device distance and bandwidth between the terminal to be expanded and the main terminal, and the device topology diagram of the main terminal is generated. The authentication pass command is sent to the terminals to be expanded in the device topology diagram, and the device capability fingerprint database of the main terminal is established based on the terminals to be expanded in the device topology diagram.

7. The extended display method for cloud phones as described in claim 1, characterized in that, If the data channel between the main terminal and the corresponding extended terminal is a distributed rendering channel, then the method for constructing the distributed rendering channel includes: Based on the WebRTC protocol extension, a FOVEC proprietary protocol is constructed as the underlying communication standard for the data channel; A hierarchical transmission architecture is constructed based on a preset transmission strategy. A direct communication link for the data channel is established based on the high-frequency channel between the main terminal and the corresponding extended terminal. A distributed rendering channel is formed based on the underlying communication standard, the hierarchical transmission architecture, and the direct communication link.

8. The extended display method for cloud phones as described in claim 7, characterized in that, After constructing the distributed rendering pipeline, the method further includes: Obtain the eye-tracking data of the target user, perform scene semantic segmentation and application type feature extraction on the target display content, and obtain the scene semantic segmentation result and application type feature of the target display content; A display weight matrix for the target display content is generated based on the user's eye-tracking data, the scene semantic segmentation results, and the application type features; The primary and secondary display areas are divided according to the display weight matrix to generate rendering instructions for the corresponding extended terminals. The rendering instructions are then sent to the corresponding extended terminals so that the corresponding extended terminals can return a rendering confirmation instruction to the primary terminal based on the rendering instructions. Upon receiving the preparation rendering confirmation instruction from the corresponding extended terminal, and after completing cross-terminal clock calibration through the quantum entanglement timestamp synchronization algorithm, the target display content is started for synchronous output.

9. The extended display method for cloud phones as described in claim 8, characterized in that, After receiving the preparation for rendering confirmation instruction from the corresponding extended terminal, the method further includes: When the edge node receives the rendering task allocation weight for the corresponding extended terminal, the rendering task of the target display content is allocated to the corresponding extended terminal according to the rendering task allocation weight, so that the corresponding extended terminal can complete the corresponding rendering task; wherein, the rendering task allocation weight is calculated by the edge node based on the environmental state data of the extended terminal.

10. The extended display method for cloud phones as described in claim 8, characterized in that, After presenting the corresponding target display content, the method further includes: When an offline extended terminal is detected, a state replica retrieval request is sent to the blockchain shard node, so that the blockchain shard node retrieves the most recent several state snapshot replicas of the offline extended terminal according to the state replica retrieval request and returns them to the main terminal. When receiving several state snapshot copies returned by the blockchain shard node, perform data integrity verification on the several state snapshot copies to generate the state snapshot corresponding to the offline extended terminal; The offline extended terminal generates a content reorganization instruction based on its rendering task, and sends the content reorganization instruction and the status snapshot to other online extended terminals so that the online extended terminals can complete the content reorganization according to the content reorganization instruction and the status snapshot.

11. The extended display method for cloud phones as described in claim 8, characterized in that, After constructing the distributed rendering pipeline, the method further includes: When a model increment package is received from the cloud, the updated dual-loop deep reinforcement learning model is loaded according to the model increment package; wherein, the model increment package is obtained by the cloud through user behavior data periodically uploaded by each extended terminal to the cloud, so that the cloud can train the dual-loop deep reinforcement learning model according to the user behavior data; The updated model is used to generate personalized extension instructions based on the current usage scenario of the target display content. The personalized extension instructions are then sent to the corresponding extension terminal so that the corresponding extension terminal can adjust the display layout and rendering parameters of the target display content according to the personalized extension instructions.

12. A method for extending the display of a cloud phone, characterized in that, Applied to an extended terminal, the method includes: When the optical field configuration start command is received from the main terminal, the physical environment data of the extended terminal is obtained and uploaded to the main terminal. When the main terminal receives the driving instruction returned after receiving the physical environment data, it adjusts the corresponding programmable photonic crystal array according to the driving instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

13. The extended display method for cloud phones as described in claim 12, characterized in that, The step of generating a corresponding light field distribution based on the target lattice structure to present the corresponding target display content includes: By controlling the optical parameters of the incident light through the target lattice structure, the incident light propagates in the air to generate a light field distribution that coordinates with the main terminal and other extended terminals, so as to present the corresponding target display content.

14. The extended display method for cloud phones as described in claim 12, characterized in that, When the main terminal receives a driving instruction returned after receiving the physical environment data, the corresponding programmable photonic crystal array is adjusted according to the driving instruction to generate a specified target lattice structure, and a corresponding light field distribution is generated according to the target lattice structure to present the corresponding target display content, including: When the main terminal receives the driving instruction returned after receiving the physical environment data, the corresponding programmable photonic crystal array is adjusted according to the driving instruction to generate the specified adjustment lattice structure. Obtain the actual light field data of the adjusted lattice structure and upload the actual light field data to the main terminal; When the main terminal receives a calibration instruction after receiving the actual light field data, it calibrates the corresponding adjustment lattice structure according to the calibration instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

15. The extended display method for cloud phones as described in claim 14, characterized in that, When the main terminal receives a calibration command after receiving the actual light field data, the system calibrates the corresponding lattice structure according to the calibration command to generate a specified target lattice structure, and generates a corresponding light field distribution according to the target lattice structure to present the corresponding target display content, including: When a calibration command is received from the main terminal after receiving the actual light field data, the corresponding adjustment lattice structure is calibrated according to the calibration command to generate the specified calibration lattice structure. The positioning deviation of the calibration lattice structure is obtained and uploaded to the main terminal; When the main terminal receives a fine-tuning instruction returned after receiving the positioning deviation, the corresponding calibration lattice structure is fine-tuned according to the fine-tuning instruction to generate the specified target lattice structure, and the corresponding light field distribution is generated according to the target lattice structure to present the corresponding target display content.

16. The extended display method for cloud phones as described in claim 12, characterized in that, Before receiving the optical field configuration start command from the main terminal, the method further includes: When the device type code broadcast by the main terminal is received, an authentication response is generated based on the device type code and uploaded to the main terminal so that the main terminal can generate a capability negotiation request based on the authentication response. When the main terminal receives the capability negotiation request returned after receiving the authentication response, a data certificate to be signed by the terminal to be extended is generated according to the capability negotiation request. The signed data certificate is uploaded to the main terminal so that the main terminal can verify the signature validity of the data certificate and send the certificate chain root verification request of the data certificate to the cloud. When a terminal receives an authentication pass instruction returned by the main terminal after receiving a signed data certificate, it becomes an extended terminal of the main terminal.

17. The extended display method for cloud phones as described in claim 12, characterized in that, Before receiving the optical field configuration start command from the main terminal, the method further includes: When a rendering instruction is received from the main terminal, a rendering preparation confirmation instruction is generated based on the rendering instruction, and the rendering preparation confirmation instruction is uploaded to the main terminal. This allows the main terminal to perform cross-terminal clock calibration using a quantum entanglement timestamp synchronization algorithm after receiving the rendering preparation confirmation instruction, and then start synchronously outputting the target display content.

18. The extended display method for cloud phones as described in claim 17, characterized in that, After uploading the preparation rendering confirmation instruction to the main terminal, the method further includes: When a rendering task is received from the main terminal, the rendering task is completed.

19. The extended display method for cloud phones as described in claim 18, characterized in that, After completing the rendering task, the method further includes: When the main terminal sends a content reorganization instruction and a status snapshot, the content reorganization is completed according to the content reorganization instruction and the status snapshot.

20. The extended display method for cloud phones as described in claim 19, characterized in that, After completing content reconstruction according to the content reconstruction instruction and the state snapshot, the method includes: When the main terminal is accessed, a hardware resource pool for the extended terminal is constructed. When an authentication request from a target user is received, the biometric data of the target user is obtained; the biometric data is uploaded to the FPGA so that the FPGA can authenticate the target user based on the biometric data and return the authentication result to the extended terminal. When the verification result returned by the FPGA is received, if the verification is successful, the user account instance is decrypted from the cloud using a key. The key is dynamically generated based on the biometric verification result and is automatically updated periodically.

21. The extended display method for cloud phones as described in claim 17, characterized in that, After uploading the preparation rendering confirmation instruction to the main terminal, the method further includes: When a personalized extension instruction is received from the main terminal, the display layout and rendering parameters of the target display content are adjusted according to the personalized extension instruction.

22. An extended display device for a cloud phone, characterized in that, Applied to the main terminal, including: The content feature extraction module is used to extract features from the current target display content to obtain the content feature matrix of the target display content when receiving physical environment data uploaded by at least one extended terminal; The driver instruction generation module is used to generate driver instructions for the corresponding extended terminal based on the content feature matrix and the physical environment data. The driving instruction issuing module is used to issue the driving instruction to the corresponding extended terminal, so that the corresponding extended terminal adjusts the corresponding programmable photonic crystal array according to the driving instruction to generate the specified target lattice structure, and generates the corresponding light field distribution according to the target lattice structure to present the corresponding target display content.

23. An extended display device for a cloud phone, characterized in that, Applied to extended terminals, including: The relevant data acquisition module is used to acquire the physical environment data of the extended terminal when it receives the light field configuration start command issued by the main terminal, and upload the physical environment data to the main terminal. The driving instruction receiving module is used to adjust the corresponding programmable photonic crystal array to generate a specified target lattice structure according to the driving instruction when it receives the driving instruction returned by the main terminal after receiving the physical environment data, and to generate a corresponding light field distribution according to the target lattice structure in order to present the corresponding target display content.

24. An extended display device for a cloud phone, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the extended display method of a cloud phone as described in any one of claims 1-11 or the extended display method of a cloud phone as described in any one of claims 12-21.

25. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the extended display method of a cloud phone as described in any one of claims 1-11 or the extended display method of a cloud phone as described in any one of claims 12-21.

26. A computer program product, characterized in that, The computer program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the extended display method of the cloud phone as described in any one of claims 1-11 or the steps of the extended display method of the cloud phone as described in any one of claims 12-21.

Images