Flexible device display system and method based on dynamic screen form adaptation

Abstract

The invention discloses a flexible device display system and method based on dynamic screen form adaptation, and relates to the technical field of flexible display. Comprising a sensing module used for monitoring multi-modal data of a dynamic screen in real time, a processing module used for constructing a resolution conversion model and calculating logic resolution, a rendering selection module used for carrying out pre-rendering analysis on the dynamic screen and a feedback module used for responding to a pre-rendering signal sent by the rendering selection module, the rendering selection module comprises a resolution prediction unit, a content identification unit, a content classification unit and a dynamic reconstruction unit; full-dimension real-time monitoring of multi-form changes such as curling and stretching is achieved through the sensing module; the logic resolution generation can adjust the resolution according to the curling length, and a system is notified to only render a visible area, so that resources are saved; the rendering preparation time is shortened by pre-rendering and allocating rendering resources in advance, the application interface layout is adjusted in real time according to the screen form, and the interaction experience of a user is improved.

Description

Technical Field

[0001] The present invention relates to the field of flexible display technology, and in particular to a flexible device display system and method based on dynamic screen form adaptation. Background Art

[0002] With the rapid development of flexible display technology, display devices with dynamic shape-changing functions are gradually entering the market, but the existing operating systems and application ecosystems cannot adapt well to them.

[0003] The present invention aims to overcome the adaptation difficulties between multi-form screens and traditional operating systems and various applications through deep collaborative optimization of software and hardware, to bring users a smoother and more efficient use experience, and to promote the widespread application and popularization of flexible devices; on rollable devices, when the screen changes shape from unfolding / curling, the application interface layout cannot be adjusted in real time according to the screen shape, and problems such as misalignment, overlap, and incomplete display of interface elements often occur, making it difficult for users to use application functions normally; during the screen shape switching process, the display resolution will change dynamically. However, existing applications lack an effective response mechanism to dynamic changes in resolution. Taking video playback as an example, when the screen shape changes, the video playback ratio may be out of balance, and the picture may be stretched and deformed, seriously damaging the user's interactive experience.

[0004] Therefore, a flexible display system and method based on dynamic screen form adaptation are needed to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to provide a cross-platform flexible display system and method based on dynamic screen form adaptation to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: A flexible device display system based on dynamic screen form adaptation includes a perception module, a processing module, a rendering selection module, and a feedback module: The sensing module is used to monitor multimodal data of the dynamic screen in real time when the user triggers a curling action on the flexible device, and feed the data back to the processing module, wherein the multimodal data includes physical form data and display data; The processing module is used to construct a resolution conversion model based on the multimodal data fed back by the perception module, calculate the logical resolution, and transmit the logical resolution to the rendering selection module; The rendering selection module includes a resolution prediction unit, a content identification unit, a content classification unit and a dynamic reconstruction unit: The resolution prediction unit is used to predict the actual resolution of future frames; The content recognition unit is used to analyze the interface elements of the dynamic screen in real time, identify the interface elements of the dynamic screen, and assign weights to the interface elements; The content classification unit is used to calculate the priority index of the interface elements according to the content recognition result and classify the interface elements into priorities; The dynamic reconstruction unit is used to pre-render and select interface elements according to the actual resolution and content classification results of the future frame, and send the pre-rendering signal to the feedback module;

[0007] The feedback module is used to respond to the pre-rendering signal sent by the rendering selection module and pre-render the dynamic screen.

[0008] As a further preferred solution of the present invention, the perception module includes a collection unit and a synchronization unit: The acquisition unit monitors the physical form data of the dynamic screen in real time through a laser ranging sensor, an acceleration sensor, a flexible curvature sensor, and a distributed optical fiber sensor, wherein the physical form data includes the real-time curling length, curling speed, and curling acceleration of the dynamic screen; and extracts the display data of the dynamic screen through the work log of the flexible device, wherein the display data includes the actual resolution of the dynamic screen and the real-time pixel points distributed on the dynamic screen; The synchronization unit periodically calibrates the clock offset of the sensor through the GPS clock and the PTP protocol, performs reverse offset compensation on the timestamp of the adaptation parameter through the fixed transmission delay of each sensor, and synchronizes the adaptation parameter according to the timestamp.

[0009] It's important to explain that the core goal of the synchronization unit is to resolve spatiotemporal data inconsistencies caused by hardware differences, varying data collection frequencies, or uneven physical distribution across multiple sensors. Different sensors can experience millisecond-level time deviations in data collected at the same moment due to differences in hardware sampling frequency, signal transmission paths (such as wired / wireless transmission delays), or clock drift, directly impacting subsequent fusion accuracy. A high-precision hardware clock (such as a GPS clock) is designated as the master clock, while the built-in clocks of other sensors serve as slave clocks. The IEEE 1588 Precision Time Protocol (PTP) is used to periodically calibrate the slave clock offsets to ensure a consistent global time base.

[0010] As a further preferred solution of the present invention, the processing module includes: Construct a resolution conversion model, take multimodal data as input data, and output logical resolution. The specific formula of the resolution conversion model is as follows: ; ; in, Indicates the deformation ratio of the dynamic screen, Indicates the real-time curling length of the dynamic screen, Indicates the curl length of the dynamic screen when it is fully curled. Indicates the logical resolution of the dynamic screen, It represents the maximum resolution when the dynamic screen is fully expanded, and k represents the characteristic coefficient of the flexible device.

[0011] It's important to explain that logical resolution bridges the gap between the physical screen form factor and software display adaptation. When the screen is warped, the actual display area changes, but operating systems typically rely on fixed resolution parameters. Logical resolution generation dynamically adjusts these parameters, enabling the system to correctly render the UI and avoid display errors. When the screen is partially warped, the effective display area shrinks, and logical resolution generation adjusts the resolution based on the warp length, instructing the system to render only the visible area, saving resources. At the same time, application compatibility is maintained, as applications are still laid out according to the logical resolution, without needing to perceive physical changes.

[0012] As a further preferred solution of the present invention, the resolution prediction unit includes: An LSTM prediction model based on the spatiotemporal attention mechanism is constructed. The input layer receives the physical morphology data and logical resolution sequence of a specified number of consecutive frames, and the output layer predicts the probability distribution of the actual resolution of future frames. The specific formula is as follows: ; in, represents the probability matrix of the actual resolution of future frames, M represents the total number of specified frames, represents the time decay factor of the i-th frame, Indicates the curling speed of the i-th frame dynamic screen, Indicates the curl acceleration of the i-frame dynamic screen. Indicates the logical resolution of the i-frame dynamic screen.

[0013] As a further preferred solution of the present invention, the content identification unit includes: It is used to analyze and identify interface elements on dynamic screens in real time, including video regions, text line candidate frames, and original saliency maps. It uses inter-frame difference and optical flow analysis to capture the motion trajectory of dynamic images and detect video regions on dynamic screens. It also obtains text line candidate frames through text detection. Using the DeepLabv3+ lightweight model, it takes the screen image as input and outputs the original saliency map. The step of detecting the video area includes: Collect two consecutive frames of images and calculate the pixel difference value: ; in, Indicates two consecutive frames and The difference value of the image at the pixel point (x, y), Represents a frame Pixels, Represents a frame Pixels of When the pixel difference When it is greater than the pixel difference threshold, the pixel point (x, y) is marked as a pixel point in the candidate dynamic area; The Lucas-Kanade algorithm is used to extract the motion vectors of the candidate dynamic area pixels. , the modulus length of the screening speed || ||The area with a motion greater than the specified threshold is marked as a video area; To obtain the text line candidate boxes, we load the EAST model through OpenCV, preprocess the input image, perform forward propagation, and use non-maximum suppression to obtain the text line candidate boxes. The original saliency map is obtained by loading the pre-trained DeepLabv3+ model based on MobileNetV3, scaling and normalizing the image, inputting it into the model, performing forward propagation, and outputting the semantic segmentation result as the original saliency map.

[0014] As a further preferred solution of the present invention, the content classification unit includes: The priority index of the interface element is calculated based on the content recognition results. The specific formula is as follows: ; in, Indicates the priority index of the interface element. 、 、 represents the preconfigured weight coefficient, and , Indicates the percentage of pixels in the video area on the dynamic screen. Indicates the percentage of pixels of the text line candidate box on the dynamic screen. represents the percentage of pixels in the original saliency map on the dynamic screen, and ; According to the priority index, the interface elements are prioritized into high priority, medium priority and low priority: When an interface element has a high priority, it will be marked and cropping is prohibited; When an interface element is of medium priority, it is marked and deformed in a limited way; When an interface element is of low priority, the interface element is marked and processed freely.

[0015] if = 0, that is, there is no video area on the current screen, then = 0, the video area does not contribute to the priority index calculation of the interface element, and the priority index of the interface element will be completely determined by the text line candidate box and the original saliency map; if = 0, that is, there is no text line candidate box detected on the current screen, then = 0, the text line candidate box does not contribute to the priority index calculation of the interface element, and the priority index of the interface element will be completely determined by the text line candidate box and the original saliency map.

[0016] As a further preferred solution of the present invention, the dynamic reconstruction unit includes: Identify the stretching direction of the dynamic screen according to the physical morphology data, where the stretching direction includes a curling direction and a stretching direction; Pre-render different interface elements according to priority tags: When the interface element is prohibited from being cropped, it is only scaled proportionally. When the edge of the interface element reaches the edge of the maximum display area, the scaling of the interface element stops. When interface elements are deformed to a limited extent, elastic layout adjustment and non-uniform scaling are applied to the text area of the interface elements; deformation compensation distortion and content reverse cropping are applied to the image area of the interface elements; and fluid layout is enabled for the saliency map of the interface elements. Based on the actual resolution of the future frame and the pre-rendering selection result, a picture pre-rendering signal of the future frame is sent to the feedback module.

[0017] As a further preferred solution of the present invention, the specific implementation process of the feedback module includes: According to the received pre-rendering signal, the pre-rendering command is executed and the pre-rendering result is stored in the buffer.

[0018] A display method for a flexible device display system based on dynamic screen form adaptation, the method comprising the following steps: Step S100: monitoring multimodal data of a dynamic screen in real time, wherein the multimodal data includes physical form data and display data; Step S200: constructing a resolution conversion model based on the multimodal data and outputting a logical resolution; Step S300: predicting the actual resolution of future frames, analyzing the interface elements of the dynamic screen in real time, identifying the interface elements of the dynamic screen, and pre-rendering and selecting the interface elements; Step S400: Pre-rendering is performed on the dynamic screen according to the actual resolution of the future frame and the pre-rendering selection.

[0019] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the flexible device display system based on dynamic screen form adaptation provided by the present invention includes a perception module for real-time monitoring of multimodal data of dynamic screens, a processing module for building a resolution conversion model and calculating logical resolution, a rendering selection module for pre-rendering analysis of dynamic screens, and a feedback module for responding to the pre-rendering signal sent by the rendering selection module, wherein the rendering selection module includes a resolution prediction unit, a content recognition unit, a content classification unit, and a dynamic reconstruction unit; full-dimensional real-time monitoring of multi-form changes such as curling and stretching is achieved through the perception module; logical resolution generation adjusts the resolution according to the curling length, and notifies the system to render only the visible area, thereby saving resources; through pre-rendering, rendering resources are allocated in advance, rendering preparation time is reduced, and the application interface layout is adjusted in real time according to the screen form, thereby improving the user's interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort. Among them: Figure 1 Schematic diagram of the structure of an embodiment of the present invention; Figure 2 Schematic diagram of a method in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0022] Example 1 like Figure 1 FIG. 1 is an embodiment of the present invention, which provides a flexible device display system based on dynamic screen form adaptation. The system includes a perception module, a processing module, a rendering selection module, and a feedback module. The sensing module is used to monitor multimodal data of the dynamic screen in real time when the user triggers a curling action on the flexible device, and feed the data back to the processing module, wherein the multimodal data includes physical form data and display data; The processing module is used to construct a resolution conversion model based on the multimodal data fed back by the perception module, calculate the logical resolution, and transmit the logical resolution to the rendering selection module; The rendering selection module includes a resolution prediction unit, a content identification unit, a content classification unit and a dynamic reconstruction unit: The resolution prediction unit is used to predict the actual resolution of future frames; The content recognition unit is used to analyze the interface elements of the dynamic screen in real time, identify the interface elements of the dynamic screen, and assign weights to the interface elements; The content classification unit is used to calculate the priority index of the interface elements according to the content recognition result and classify the interface elements into priorities; The dynamic reconstruction unit is used to pre-render and select interface elements according to the actual resolution and content classification results of the future frame, and send the pre-rendering signal to the feedback module; The feedback module is used to respond to the pre-rendering signal sent by the rendering selection module and pre-render the dynamic screen.

[0023] Specifically, the perception module includes a collection unit and a synchronization unit: The acquisition unit monitors the physical form data of the dynamic screen in real time through a laser ranging sensor, an acceleration sensor, a flexible curvature sensor, and a distributed optical fiber sensor, wherein the physical form data includes the real-time curling length, curling speed, and curling acceleration of the dynamic screen; and extracts the display data of the dynamic screen through the work log of the flexible device, wherein the display data includes the actual resolution of the dynamic screen and the real-time pixel points distributed on the dynamic screen; The synchronization unit periodically calibrates the clock offset of the sensor through the GPS clock and the PTP protocol, performs reverse offset compensation on the timestamp of the adaptation parameter through the fixed transmission delay of each sensor, and synchronizes the adaptation parameter according to the timestamp.

[0024] Specifically, the specific implementation process of the processing module includes: Construct a resolution conversion model, take multimodal data as input data, and output logical resolution. The specific formula of the resolution conversion model is as follows: ; ; in, Indicates the deformation ratio of the dynamic screen, Indicates the real-time curling length of the dynamic screen, Indicates the curl length of the dynamic screen when it is fully curled. Indicates the logical resolution of the dynamic screen, It represents the maximum resolution when the dynamic screen is fully expanded, and k represents the characteristic coefficient of the flexible device.

[0025] Specifically, the resolution prediction unit includes: An LSTM prediction model based on the spatiotemporal attention mechanism is constructed. The input layer receives the physical morphology data and logical resolution sequence of a specified number of consecutive frames, and the output layer predicts the probability distribution of the actual resolution of future frames. The specific formula is as follows: ; in, represents the probability matrix of the actual resolution of future frames, M represents the total number of specified frames, represents the time decay factor of the i-th frame, Indicates the curling speed of the i-th frame dynamic screen, Indicates the curl acceleration of the i-frame dynamic screen. Indicates the logical resolution of the i-frame dynamic screen.

[0026] Specifically, the content identification unit includes: It is used to analyze and identify interface elements on dynamic screens in real time, including video regions, text line candidate frames, and original saliency maps. It uses inter-frame difference and optical flow analysis to capture the motion trajectory of dynamic images and detect video regions on dynamic screens. It also obtains text line candidate frames through text detection. Using the DeepLabv3+ lightweight model, it takes the screen image as input and outputs the original saliency map. The step of detecting the video area includes: Collect two consecutive frames of images and calculate the pixel difference value: ; in, Indicates two consecutive frames and The difference value of the image at the pixel point (x, y), Represents a frame Pixels, Represents a frame Pixels of When the pixel difference When it is greater than the pixel difference threshold, the pixel point (x, y) is marked as a pixel point in the candidate dynamic area; The Lucas-Kanade algorithm is used to extract the motion vectors of the candidate dynamic area pixels. , the modulus of screening speed The area with motion greater than the specified threshold is marked as the video area; To obtain the text line candidate boxes, we load the EAST model through OpenCV, preprocess the input image, perform forward propagation, and use non-maximum suppression to obtain the text line candidate boxes. The original saliency map is obtained by loading the pre-trained DeepLabv3+ model based on MobileNetV3, scaling and normalizing the image, inputting it into the model, performing forward propagation, and outputting the semantic segmentation result as the original saliency map.

[0027] Specifically, the content classification unit includes: The priority index of the interface element is calculated based on the content recognition results. The specific formula is as follows: ; in, Indicates the priority index of the interface element. 、 、 represents the preconfigured weight coefficient, and , Indicates the percentage of pixels in the video area on the dynamic screen. Indicates the percentage of pixels of the text line candidate box on the dynamic screen. represents the percentage of pixels in the original saliency map on the dynamic screen, and ; According to the priority index, the interface elements are prioritized into high priority, medium priority and low priority: When an interface element has a high priority, it will be marked and cropping is prohibited; When an interface element is of medium priority, it is marked and deformed in a limited way; When an interface element is of low priority, the interface element is marked and processed freely.

[0028] Specifically, the dynamic reconstruction unit includes: Identify the stretching direction of the dynamic screen according to the physical morphology data, where the stretching direction includes a curling direction and a stretching direction; Pre-render different interface elements according to priority tags: When the interface element is prohibited from being cropped, it is only scaled proportionally. When the edge of the interface element reaches the edge of the maximum display area, the scaling of the interface element stops. When interface elements are deformed to a limited extent, elastic layout adjustment and non-uniform scaling are applied to the text area of the interface elements; deformation compensation distortion and content reverse cropping are applied to the image area of the interface elements; and fluid layout is enabled for the saliency map of the interface elements. Based on the actual resolution of the future frame and the pre-rendering selection result, a picture pre-rendering signal of the future frame is sent to the feedback module.

[0029] Specifically, the specific implementation process of the feedback module includes: According to the received pre-rendering signal, the pre-rendering command is executed and the pre-rendering result is stored in the buffer.

[0030] Example 2 like Figure 2 FIG. 1 is an embodiment of the present invention, which provides a display method of a flexible device display system based on dynamic screen form adaptation, and the method includes the following steps: Step S100: monitoring multimodal data of a dynamic screen in real time, wherein the multimodal data includes physical form data and display data; Step S200: constructing a resolution conversion model based on the multimodal data and outputting a logical resolution; Step S300: predicting the actual resolution of future frames, analyzing the interface elements of the dynamic screen in real time, identifying the interface elements of the dynamic screen, and pre-rendering and selecting the interface elements; Step S400: Pre-rendering is performed on the dynamic screen according to the actual resolution of the future frame and the pre-rendering selection.

[0031] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any other combination. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product, which includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0032] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the aforementioned integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0033] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A flexible device display system based on dynamic screen form adaptation, characterized in that: Including perception module, processing module, rendering selection module and feedback module: The sensing module is used to monitor multimodal data of the dynamic screen in real time when the user triggers a curling action on the flexible device, and feed the data back to the processing module, wherein the multimodal data includes physical form data and display data; The processing module is used to construct a resolution conversion model based on the multimodal data fed back by the perception module, calculate the logical resolution, and transmit the logical resolution to the rendering selection module; The rendering selection module includes a resolution prediction unit, a content identification unit, a content classification unit and a dynamic reconstruction unit: The resolution prediction unit is used to predict the actual resolution of future frames; The content recognition unit is used to analyze the interface elements of the dynamic screen in real time, identify the interface elements of the dynamic screen, and assign weights to the interface elements; The content classification unit is used to calculate the priority index of the interface elements according to the content recognition result and classify the interface elements into priorities; The dynamic reconstruction unit is used to pre-render and select interface elements according to the actual resolution and content classification results of the future frame, and send the pre-rendering signal to the feedback module; The feedback module is used to respond to the pre-rendering signal sent by the rendering selection module and pre-render the dynamic screen.

2. The flexible device display system based on dynamic screen form adaptation according to claim 1, characterized in that: The perception module includes a collection unit and a synchronization unit: The acquisition unit monitors the physical form data of the dynamic screen in real time through a laser ranging sensor, an acceleration sensor, a flexible curvature sensor, and a distributed optical fiber sensor, wherein the physical form data includes the real-time curling length, curling speed, and curling acceleration of the dynamic screen; and extracts the display data of the dynamic screen through the work log of the flexible device, wherein the display data includes the actual resolution of the dynamic screen and the real-time pixel points distributed on the dynamic screen; The synchronization unit periodically calibrates the clock offset of the sensor through the GPS clock and the PTP protocol, performs reverse offset compensation on the timestamp of the adaptation parameter through the fixed transmission delay of each sensor, and synchronizes the adaptation parameter according to the timestamp.

3. The flexible device display system based on dynamic screen form adaptation according to claim 2, characterized in that: The specific implementation process of the processing module includes: Construct a resolution conversion model, take multimodal data as input data, and output logical resolution. The specific formula of the resolution conversion model is as follows: ； ； in, Indicates the deformation ratio of the dynamic screen, Indicates the real-time curling length of the dynamic screen, Indicates the curl length of the dynamic screen when it is fully curled. Indicates the logical resolution of the dynamic screen, It represents the maximum resolution when the dynamic screen is fully expanded, and k represents the characteristic coefficient of the flexible device.

4. The flexible device display system based on dynamic screen form adaptation according to claim 3, characterized in that: The resolution prediction unit includes: An LSTM prediction model based on the spatiotemporal attention mechanism is constructed. The input layer receives the physical morphology data and logical resolution sequence of a specified number of consecutive frames, and the output layer predicts the probability distribution of the actual resolution of future frames. The specific formula is as follows: ； in, represents the probability matrix of the actual resolution of future frames, M represents the total number of specified frames, represents the time decay factor of the i-th frame, Indicates the curling speed of the i-th frame dynamic screen, Indicates the curl acceleration of the i-frame dynamic screen. Indicates the logical resolution of the i-frame dynamic screen.

5. The flexible device display system based on dynamic screen form adaptation according to claim 1, characterized in that: The content identification unit includes: It is used to analyze and identify interface elements on dynamic screens in real time, including video regions, text line candidate frames, and original saliency maps. It uses inter-frame difference and optical flow analysis to capture the motion trajectory of dynamic images and detect video regions on dynamic screens. It also obtains text line candidate frames through text detection. Using the DeepLabv3+ lightweight model, it takes the screen image as input and outputs the original saliency map. The step of detecting the video area includes: Collect two consecutive frames of images and calculate the pixel difference value: ; in, Indicates two consecutive frames and The difference value of the image at the pixel point (x, y), Represents a frame Pixels, Represents a frame Pixels of When the pixel difference When it is greater than the pixel difference threshold, the pixel point (x, y) is marked as a pixel point in the candidate dynamic area; The Lucas-Kanade algorithm is used to extract the motion vectors of the candidate dynamic area pixels. , the modulus of screening speed The area with motion greater than the specified threshold is marked as the video area; To obtain the text line candidate boxes, we load the EAST model through OpenCV, preprocess the input image, perform forward propagation, and use non-maximum suppression to obtain the text line candidate boxes. The original saliency map is obtained by loading the pre-trained DeepLabv3+ model based on MobileNetV3, scaling and normalizing the image before inputting it into the model, performing forward propagation, and outputting the semantic segmentation result as the original saliency map.

6. The flexible device display system based on dynamic screen form adaptation according to claim 1, characterized in that: The content classification unit includes: The priority index of the interface element is calculated based on the content recognition results. The specific formula is as follows: ； in, Indicates the priority index of the interface element. 、 、 represents the preconfigured weight coefficient, and , Indicates the percentage of pixels in the video area on the dynamic screen. Indicates the percentage of pixels of the text line candidate box on the dynamic screen. represents the percentage of pixels in the original saliency map on the dynamic screen, and ; According to the priority index, the interface elements are prioritized into high priority, medium priority and low priority: When an interface element has a high priority, it will be marked and cropping is prohibited; When an interface element is of medium priority, it is marked and deformed in a limited way; When an interface element is of low priority, the interface element is marked and processed freely.

7. The flexible device display system based on dynamic screen form adaptation according to claim 1, characterized in that: The dynamic reconstruction unit includes: Identify the stretching direction of the dynamic screen according to the physical morphology data, where the stretching direction includes a curling direction and a stretching direction; Pre-render different interface elements according to priority tags: When the interface element is prohibited from being cropped, it is only scaled proportionally. When the edge of the interface element reaches the edge of the maximum display area, the scaling of the interface element stops. When interface elements are deformed to a limited extent, elastic layout adjustment and non-uniform scaling are applied to the text area of the interface elements; deformation compensation distortion and content reverse cropping are applied to the image area of the interface elements; and fluid layout is enabled for the saliency map of the interface elements. Based on the actual resolution of the future frame and the pre-rendering selection result, a picture pre-rendering signal of the future frame is sent to the feedback module.

8. The flexible device display system based on dynamic screen form adaptation according to claim 1, characterized in that: The specific implementation process of the feedback module includes: According to the received pre-rendering signal, the pre-rendering command is executed and the pre-rendering result is stored in the buffer.

9. The display method of a flexible device display system based on dynamic screen form adaptation according to any one of claims 1 to 8, characterized in that: The method comprises the following steps: Step S100: monitoring multimodal data of a dynamic screen in real time, wherein the multimodal data includes physical form data and display data; Step S200: constructing a resolution conversion model based on the multimodal data and outputting a logical resolution; Step S300: predicting the actual resolution of future frames, analyzing the interface elements of the dynamic screen in real time, identifying the interface elements of the dynamic screen, and pre-rendering and selecting the interface elements; Step S400: Pre-rendering is performed on the dynamic screen according to the actual resolution of the future frame and the pre-rendering selection.

Images