Image-based interactive display system

Abstract

This invention relates to the field of interactive display technology, and more particularly to a display system based on image interaction. The invention uses an interactive motion capture module to acquire the user's horizontal line of sight while viewing aerospace interactive materials and to determine the corresponding line-of-sight pitch angles at several moments. An information aggregation module statistically analyzes the line-of-sight pitch angles of several users experiencing interactive sub-materials and plots an attitude change curve showing the change in line-of-sight pitch angle as the interactive sub-materials play. A feature filtering module determines feature-specific interactive sub-materials based on the duration of feature states in the attitude change curve. An interactive classification module determines the feature categories of the feature-specific interactive sub-materials. A display control module determines the display adjustment method for the feature-specific interactive sub-materials based on their feature categories, either by adjusting the playback frame rate of the aerospace interactive materials or by adjusting the center position of the screen of the feature-specific interactive sub-materials according to the changes in line-of-sight pitch angles in the attitude change curve.

Description

Technical Field

[0001] This invention relates to the field of interactive display technology, and more particularly to an image-based interactive display system. Background Technology

[0002] With the increasingly widespread application of immersive interactive technology in the aerospace exhibition field, image-interactive aerospace exhibition systems have become an important carrier for popularizing aerospace scenarios and simulating experiences. Through precise user interaction adaptation, they ensure the clarity and viewing angle stability of the displayed images, enhancing the user's immersive experience. However, existing related exhibition systems still have significant shortcomings: most systems do not accurately capture and quantify the user's eye level during the viewing of aerospace interactive materials, and cannot identify users' behaviors such as prolonged head tilting, head tilting, or frequent switching of viewing angles. Consequently, they cannot solve problems such as image jitter and decreased clarity caused by these behaviors. They only use fixed playback parameters and image settings, making it difficult to balance user viewing comfort and exhibition effect.

[0003] For example, Chinese invention patent CN109976527A discloses an interactive VR display system, including a motion acquisition module for collecting the user's location information; a processing module for generating a building model and receiving the location information, determining the user's perspective based on the location information, and generating a partial VR image of the building based on the building model and the user's perspective; a display module for displaying the partial VR image of the building generated by the processing module; and an interaction module for collecting the user's interaction commands. The processing module is also used to adjust the partial VR image of the building according to the interaction commands. The technical solution of this invention enhances the interactive experience of users when viewing VR images.

[0004] The following problems still exist in the existing technology:

[0005] Existing technologies cannot adjust display strategies based on user gaze behavior, nor can they be adapted differently according to the causes of unstable gaze posture, thus failing to meet the requirements for highly stable and immersive aerospace interactive displays. Summary of the Invention

[0006] To address this, the present invention provides an image-interactive display system to overcome the problems of existing technologies that cannot adjust display strategies based on user gaze behavior, cannot perform differentiated adaptation according to the causes of unstable gaze posture, and cannot meet the requirements of highly stable and highly immersive aerospace interactive display.

[0007] To achieve the above objectives, the present invention provides an image-interactive display system, comprising:

[0008] An interactive motion capture module is used to acquire the user's horizontal line of sight during the viewing of aerospace interactive materials, and to determine the line of sight pitch angle at several moments based on the horizontal line of sight.

[0009] The information aggregation module, which is connected to the interactive action capture module, is used to divide the aerospace interactive material into several interactive sub-materials, to count the line-of-sight pitch angle of several users during the experience of the interactive sub-materials, and to draw the attitude change curve of the line-of-sight pitch angle as the playback progress of the interactive sub-materials changes.

[0010] The feature filtering module, which is connected to the information aggregation module, is used to determine the feature interactive sub-materials based on the feature state duration of the posture change curve corresponding to the interactive sub-materials.

[0011] An interactive classification module, which is connected to the feature filtering module, is used to mark state switching points on the posture change curve corresponding to the feature interactive sub-material; and to determine the feature category of the feature interactive sub-material based on the number of state switching points.

[0012] The display control module, which is connected to the interaction classification module, is used to determine the display adjustment method of the feature interaction sub-materials according to the feature category of the feature interaction sub-materials. The display adjustment method is to adjust the playback frame rate of the aerospace interaction material; or to adjust the center position of the screen of the feature interaction sub-materials according to the change of the line of sight pitch angle of the attitude change curve.

[0013] Furthermore, the interactive motion capture module is used to determine the gaze pitch angle at several moments, wherein,

[0014] The interactive action capture module uses the user's horizontal line of sight looking straight ahead as a reference, records the angle of upward deviation from the horizontal line of sight as the line of sight elevation angle, and the angle of downward deviation from the horizontal line of sight as the line of sight depression angle. It collects the line of sight elevation angle or line of sight depression angle during the user experience process at preset time intervals, and determines the line of sight elevation angle corresponding to several collection moments.

[0015] Furthermore, the information aggregation module is used to plot the attitude change curve, wherein,

[0016] The information aggregation module divides each interactive sub-material into segments according to the playback timeline of the aerospace interactive material, and calculates the average line-of-sight pitch angle of all users at each collection time during the playback period of each interactive sub-material.

[0017] The information aggregation module plots the posture change curves corresponding to each interactive sub-material by using the playback progress time as the horizontal axis and the average line-of-sight pitch angle as the vertical axis.

[0018] Furthermore, the feature filtering module is used to determine the duration of characteristic states in the attitude change curve, wherein,

[0019] The feature filtering module pre-sets a tilt angle threshold and an elevation angle threshold, and respectively counts the first duration when the elevation angle is greater than or equal to the elevation angle threshold and the second duration when the tilt angle is less than or equal to the tilt angle threshold in the posture change curve corresponding to each interactive sub-material. The sum of the first duration and the second duration is recorded as the feature state duration.

[0020] Furthermore, the feature filtering module is used to determine feature interaction sub-materials, wherein,

[0021] The feature filtering module is used to calculate the ratio of the feature state duration to the total playback duration of the interactive sub-materials;

[0022] If the ratio is greater than a preset ratio reference value, the feature filtering module determines the interactive sub-material as a feature interactive sub-material.

[0023] Furthermore, the interactive classification module is used to mark state transition points, wherein,

[0024] The interaction classification module traverses the coordinate points of the posture change curves corresponding to the feature interaction sub-materials, and marks the coordinate points in the posture change curves that change from positive to negative or from negative to positive and pass through the horizontal axis as the state switching points.

[0025] Furthermore, the interaction classification module is used to determine the feature categories of the feature interaction sub-materials, wherein,

[0026] The interactive classification module is used to count the number of state switching points and compare the number with a preset reference value.

[0027] If the number of state switching points is greater than the reference value, the interaction classification module will divide the feature interaction sub-material into the first feature interaction sub-material.

[0028] If the number of state switching points is less than or equal to the reference value, the interaction classification module will divide the feature interaction sub-material into the second feature interaction sub-material.

[0029] Furthermore, the display control module is used to determine the display adjustment method of the feature interactive sub-materials, wherein,

[0030] If the feature interaction sub-material is classified as the first feature interaction sub-material, then the display control module determines that the display adjustment method for the feature interaction sub-material is to adjust the playback frame rate of the aerospace interaction material;

[0031] If the feature interaction sub-material is classified as the second feature interaction sub-material, the display control module determines that the display adjustment method for the feature interaction sub-material is to adjust the center position of the feature interaction sub-material according to the change in the viewing angle of the posture change curve.

[0032] Furthermore, the playback frame rate is positively correlated with the number of state switching points.

[0033] Furthermore, the display control module is used to adjust the center position of the feature interactive sub-materials on the screen, wherein,

[0034] The display control module is used to determine the average of the line-of-sight pitch angles at all acquisition moments in the posture change curve corresponding to the second feature interactive sub-material as the overall offset reference angle;

[0035] The direction of adjusting the center position of the aerospace interactive material is opposite to the direction of the overall offset reference angle, and the adjustment distance of the center position is positively correlated with the angle of the overall offset reference angle.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention obtains the user's horizontal line of sight during the viewing of aerospace interactive materials through an interactive motion capture module, and determines the corresponding line of sight pitch angle at several moments; it statistically analyzes the line of sight pitch angle of several users during the experience of interactive sub-materials through an information aggregation module, and plots the attitude change curve of the line of sight pitch angle as the playback progress of the interactive sub-materials; it determines the feature interactive sub-materials based on the feature state duration of the attitude change curve through a feature filtering module; it determines the feature category of the feature interactive sub-materials through an interactive classification module; and it determines the display adjustment method of the feature interactive sub-materials according to the feature category of the feature interactive sub-materials, which is to adjust the playback frame rate of the aerospace interactive materials or adjust the center position of the screen of the feature interactive sub-materials according to the line of sight pitch angle changes in the attitude change curve through a display control module.

[0037] Furthermore, by standardizing and visualizing user group gaze data, this invention transforms discrete gaze pitch angle data into a trend curve that can be intuitively analyzed, thus visually presenting the dynamic change trend of the gaze pitch angle as the user group watches the interactive sub-material.

[0038] Furthermore, based on the objective laws of ergonomics and immersive visual experience, this invention defines an abnormal tilt posture—where the user's gaze deviates from the comfortable eye level and is sufficient to cause visual instability and decreased image clarity—as a characteristic state. It separately counts the first duration of the eye level exceeding the tilt angle threshold and the second duration of the eye level exceeding the tilt angle threshold. This can accurately quantify the total duration of the user's excessive head tilt or excessive head tilt while watching the interactive sub-material, and can fully reflect the overall duration of the user's gaze deviating from the comfortable eye level state, enabling the system to accurately identify interactive content with a risk of visual instability.

[0039] Furthermore, in this invention, the positive and negative vertical coordinates of the posture change curve correspond to the elevation and depression angles of the gaze, respectively. When the curve crosses the horizontal axis, it indicates that the user's gaze has undergone a clear posture switch between looking up and looking down. By traversing all the coordinate points of the curve and accurately marking such state switch points, the conversion nodes of the user's gaze posture can be objectively captured, thereby achieving refined identification of the visual disturbance features of the feature interaction sub-materials.

[0040] Furthermore, this invention refines and categorizes the causes of visual instability based on the variation pattern of the user's line of sight pitch angle, achieving differentiated adaptation based on the causes of visual instability, thus meeting the requirements for highly stable and immersive aerospace interactive display. Attached Figure Description

[0041] Figure 1 A system block diagram of an image-interactive display system according to an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram illustrating the determination of the duration of a characteristic state on an attitude change curve according to an embodiment of the present invention;

[0043] Figure 3 The flowchart for determining feature-interactive sub-materials by the feature filtering module in this embodiment of the invention;

[0044] Figure 4 A logical flowchart for determining the feature categories and display adjustment methods of feature-interactive sub-materials in embodiments of the present invention. Detailed Implementation

[0045] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0046] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0047] It should be noted that in the description of this invention, the terms "upper," "lower," "inner," "outer," etc., which indicate the direction or positional relationship, are based on the direction or positional relationship shown in the drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0048] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0049] Please see Figure 1 As shown, it is a system block diagram of an image-interactive display system according to an embodiment of the present invention. The image-interactive display system of the present invention includes:

[0050] An interactive motion capture module is used to acquire the user's horizontal line of sight during the viewing of aerospace interactive materials, and to determine the line of sight pitch angle at several moments based on the horizontal line of sight.

[0051] In this invention, the horizontal line of sight is the visual baseline for the user's natural forward gaze. In the prior art, the user's head posture information can be obtained through head posture detection, and then the horizontal line of sight can be determined.

[0052] This invention does not limit the interactive motion capture module. It can use an infrared camera to capture images of the user's head posture, determine the user's horizontal line of sight when the user is naturally looking straight ahead based on the images of the user's head posture, and calculate the line of sight pitch angles corresponding to several acquisition times based on the direction of the user's line of sight turning upward or downward.

[0053] The information aggregation module, which is connected to the interactive action capture module, is used to divide the aerospace interactive material into several interactive sub-materials, to count the line-of-sight pitch angle of several users during the experience of the interactive sub-materials, and to draw the attitude change curve of the line-of-sight pitch angle as the playback progress of the interactive sub-materials changes.

[0054] The number of users counted in the implementation of this invention needs to be greater than or equal to 50.

[0055] For example, if the aerospace interactive material is a 180-second immersive video of a space station tour, its content includes: 0-30 seconds of the space station hatch opening scene, 31-90 seconds of the space station cabin passageway walking scene, 91-150 seconds of the space cabin external observation scene, and 151-180 seconds of the Earth panoramic observation scene; then the information aggregation module divides the complete aerospace interactive material into four interactive sub-materials according to the playback timeline of the aerospace interactive material, combined with the completeness and independence of the scene content. These sub-materials are: the first interactive sub-material corresponding to 0-30 seconds, the second interactive sub-material corresponding to 31-90 seconds, the third interactive sub-material corresponding to 91-150 seconds, and the fourth interactive sub-material corresponding to 151-180 seconds. Each interactive sub-material corresponds to an independent and complete aerospace display scene.

[0056] In this invention, the posture change curve is plotted with the playback progress time as the horizontal axis and the line-of-sight pitch angle as the vertical axis.

[0057] The present invention does not limit the information aggregation module, which includes a memory for receiving and storing multiple users’ line-of-sight pitch angle data generated during the viewing process.

[0058] The feature filtering module, which is connected to the information aggregation module, is used to determine the feature interactive sub-materials based on the feature state duration of the posture change curve corresponding to the interactive sub-materials.

[0059] An interactive classification module, which is connected to the feature filtering module, is used to mark state switching points on the posture change curve corresponding to the feature interactive sub-material; and to determine the feature category of the feature interactive sub-material based on the number of state switching points.

[0060] This invention does not limit the feature selection module and the interactive classification module, which can each be a data processor.

[0061] The display control module, which is connected to the interaction classification module, is used to determine the display adjustment method of the feature interaction sub-materials according to the feature category of the feature interaction sub-materials. The display adjustment method is to adjust the playback frame rate of the aerospace interaction material; or to adjust the center position of the screen of the feature interaction sub-materials according to the change of the line of sight pitch angle of the attitude change curve.

[0062] This invention does not limit the display control module, which can be constructed using logic components. The logic components can be field-programmable logic components, microprocessors, processors used in computers, etc., which will not be elaborated here.

[0063] Specifically, the interactive motion capture module is used to determine the gaze pitch angle at several moments, wherein,

[0064] The interactive action capture module uses the user's horizontal line of sight looking straight ahead as a reference, records the angle of upward deviation from the horizontal line of sight as the line of sight elevation angle, and the angle of downward deviation from the horizontal line of sight as the line of sight depression angle. It collects the line of sight elevation angle or line of sight depression angle during the user experience process at preset time intervals, and determines the line of sight elevation angle corresponding to several collection moments.

[0065] In the implementation of this invention, if the preset time interval for collecting the elevation or depression angle of the line of sight during the user experience process is set too small, it will result in a large amount of collected data and high system resource consumption. If it is set too large, it will result in the inability to capture rapid changes in the line of sight, leading to distortion of the posture curve. Optionally, the preset time interval can be 1 second.

[0066] In this invention, the angle at which the user's gaze deviates upward from the horizontal reference is defined as the gaze elevation angle, and the angle at which the user's gaze deviates downward is defined as the gaze depression angle. The interactive action capture module synchronously records the timestamp and the corresponding gaze depression angle and gaze elevation angle. The posture change curve is plotted with the playback progress time as the horizontal axis and the elevation angle as the vertical axis. The elevation angle is positive on the vertical axis and the depression angle is negative on the vertical axis.

[0067] Understandably, in actual aerospace interactive demonstration scenarios, the aerospace interactive materials viewed by users include various scenes with different pitch angles, such as the layout of the spacecraft cabin, the external space environment, and celestial observation. By continuously collecting the user's line of sight elevation or line of sight depression data during the experience through preset time intervals, and synchronously recording the time nodes corresponding to each collection moment, the line of sight elevation angle data at several discrete collection moments is finally determined.

[0068] Specifically, the information aggregation module is used to plot the attitude change curve, wherein,

[0069] The information aggregation module divides each interactive sub-material into segments according to the playback timeline of the aerospace interactive material, and calculates the average line-of-sight pitch angle of all users at each collection time during the playback period of each interactive sub-material.

[0070] The information aggregation module plots the posture change curves corresponding to each interactive sub-material by using the playback progress time as the horizontal axis and the average line-of-sight pitch angle as the vertical axis.

[0071] Understandably, this invention transforms discrete line-of-sight pitch angle data into intuitively analyzable trend curves through standardized processing and visualization of user group line-of-sight data. In practical applications, aerospace interactive materials often contain multiple different display scenarios, and users' line-of-sight postures differ in different scenarios. This invention rationally divides the complete material into several independent interactive sub-materials according to the playback timeline of the aerospace interactive materials, ensuring that each segment of interactive sub-material corresponds to a relatively unified display scenario. For each segment of interactive sub-material, the module will summarize the raw line-of-sight pitch angle data of all users at each preset collection time during the playback period of the sub-material, reflecting the overall line-of-sight posture characteristics of the group of users when watching the segment of interactive sub-material. Using the average line-of-sight pitch angle at each collection time as the vertical axis, a two-dimensional coordinate system is constructed and a posture change curve is plotted, intuitively presenting the dynamic change trend of the line-of-sight pitch angle as the playback progress of the group of users during the viewing of the segment of interactive sub-material.

[0072] Specifically, the feature filtering module is used to determine the duration of characteristic states in the attitude change curve, wherein,

[0073] The feature filtering module pre-sets a tilt angle threshold and an elevation angle threshold, and respectively counts the first duration when the elevation angle is greater than or equal to the elevation angle threshold and the second duration when the tilt angle is less than or equal to the tilt angle threshold in the posture change curve corresponding to each interactive sub-material. The sum of the first duration and the second duration is recorded as the feature state duration.

[0074] In this invention, the elevation and depression thresholds are set according to the ergonomic visual comfort range. With the user's horizontal line of sight as 0° as the reference, the elevation threshold is set to +15° and the depression threshold is set to -15°. The elevation threshold is used to determine whether the user's line of sight shifts upward beyond the comfort range, and the depression threshold is used to determine whether the user's line of sight shifts downward beyond the comfort range. These thresholds are preset by the system and can be adjusted within a reasonable range by those skilled in the art according to the comfort requirements of the experience.

[0075] Please see Figure 2 As shown, this is a schematic diagram of determining the duration of a characteristic state on the attitude change curve in an embodiment of the present invention. The elevation angle threshold β1 is +15°, the depression angle threshold β2 is -15°, the sum of the values ​​of t1 and t4 is the first duration in the attitude change curve corresponding to the interactive sub-material where the elevation angle is greater than or equal to the depression angle threshold β1, and the sum of the values ​​of t2, t3 and t5 is the second duration in the attitude change curve corresponding to the interactive sub-material where the depression angle is less than or equal to the depression angle threshold β2. The sum of the first duration and the second duration is recorded as the duration of the characteristic state, which is the sum of the values ​​of t1, t2, t3, t4 and t5.

[0076] Understandably, this invention is based on the objective laws of ergonomics and immersive visual experience. It defines abnormal tilting postures, where the user's gaze deviates from the comfortable eye level and is sufficient to cause visual instability and decreased image clarity, as characteristic states. It separately counts the first duration of the eye level exceeding the tilting angle threshold and the second duration of the eye level exceeding the tilting angle threshold. This can accurately quantify the total time that the user is in an excessively tilted-down or excessively tilted-up state while watching the interactive sub-material. It can fully reflect the overall duration of the user's gaze deviating from the comfortable eye level state, enabling the system to accurately identify interactive content with a risk of visual instability.

[0077] Specifically, please refer to Figure 3 As shown, this is a flowchart illustrating the logic of the feature filtering module determining feature-interactive sub-materials in an embodiment of the present invention. The feature filtering module is used to determine feature-interactive sub-materials, wherein...

[0078] The feature filtering module is used to calculate the ratio of the feature state duration to the total playback duration of the interactive sub-materials;

[0079] If the ratio is less than or equal to a preset ratio reference value, the feature filtering module determines that the interactive sub-material is not a feature interactive sub-material.

[0080] If the ratio is greater than a preset ratio reference value, the feature filtering module determines the interactive sub-material as a feature interactive sub-material.

[0081] In actual immersive aerospace interactive experiences, the larger the ratio of the duration of characteristic states to the total playback time of interactive sub-materials, the greater the proportion of time that the user spends in an unstable state of excessive head tilting or looking down during the viewing process, which is more likely to cause problems such as shaky viewpoints, decreased image clarity, and visual fatigue.

[0082] The preset ratio reference value is used to define the critical percentage of when the user's line of sight is in an uncomfortable tilt state; the preset ratio reference value can be selected from values ​​within [0.3, 0.55], and optionally, in the implementation of the present invention, the preset ratio reference value is 0.4.

[0083] Specifically, the interactive classification module is used to mark state transition points, wherein,

[0084] The interaction classification module traverses the coordinate points of the posture change curves corresponding to the feature interaction sub-materials, and marks the coordinate points in the posture change curves that change from positive to negative or from negative to positive and pass through the horizontal axis as the state switching points.

[0085] In immersive aerospace interactive display scenarios, the positive and negative vertical coordinates of the attitude change curve correspond to the elevation and depression angles of the viewpoint, respectively. When the curve crosses the horizontal axis, it indicates that the user's gaze has undergone a clear attitude switch between looking up and looking down. This switching behavior is an important cause of frequent changes in perspective and decreased image stability. By traversing all coordinate points of the curve and accurately marking such state switching points, the conversion nodes of the user's gaze attitude can be objectively captured, thereby achieving refined identification of the visual disturbance features of the feature interactive sub-materials.

[0086] Specifically, please refer to Figure 4 As shown, it is a logical flowchart of determining the feature category and display adjustment method of the feature interactive sub-materials in an embodiment of the present invention. The interaction classification module is used to determine the feature category of the feature interactive sub-materials, wherein,

[0087] The interactive classification module is used to count the number of state switching points and compare the number with a preset reference value.

[0088] If the number of state switching points is greater than the reference value, the interaction classification module will divide the feature interaction sub-material into the first feature interaction sub-material.

[0089] If the number of state switching points is less than or equal to the reference value, the interaction classification module will divide the feature interaction sub-material into the second feature interaction sub-material.

[0090] In this invention, a preset quantity reference value is used as a critical quantity value to distinguish the types of feature interaction sub-materials. This value can be calibrated based on the playback duration of the feature interaction sub-materials to effectively distinguish between frequent switching of the user's gaze and continuous unidirectional gaze shift. Optionally, when the playback duration of the feature interaction sub-material is 20 seconds, the quantity reference value can be set to 4. For every 5 seconds the playback duration of the aerospace interaction material exceeds the 20-second baseline, the quantity reference value increases by 1; for every 5 seconds the playback duration of the aerospace interaction material is less than the 20-second baseline, the quantity reference value decreases by 1; and for feature interaction sub-materials with a playback duration of 5 seconds or less, the quantity reference value is 1.

[0091] Specifically, the display control module is used to determine the display adjustment method for the feature interactive sub-materials, wherein,

[0092] If the feature interaction sub-material is classified as the first feature interaction sub-material, then the display control module determines that the display adjustment method for the feature interaction sub-material is to adjust the playback frame rate of the aerospace interaction material;

[0093] If the feature interaction sub-material is classified as the second feature interaction sub-material, the display control module determines that the display adjustment method for the feature interaction sub-material is to adjust the center position of the feature interaction sub-material according to the change in the viewing angle of the posture change curve.

[0094] In this invention, the causes of visual instability are finely differentiated and categorized based on the variation pattern of the user's line of sight pitch angle. The number of state switching points reflects the degree to which the user frequently switches between looking up and looking down. The more points there are, the more drastic the change in the user's line of sight posture, and the more likely it is to cause screen shake and unstable viewing angle. The playback frame rate is adjusted to improve the smoothness and continuity of the screen to offset the visual disturbance caused by frequent posture switching. The fewer state switching points indicate that the user's visual instability is mainly manifested in maintaining a single offset posture of looking up or down for a long time. The center position of the screen is adjusted according to the change in the line of sight pitch angle to compensate and correct the visual offset. By accurately distinguishing different causes of instability, differentiated adaptation is achieved according to the causes of instability in line of sight posture, meeting the requirements of highly stable and highly immersive aerospace interactive display.

[0095] Specifically, the playback frame rate is positively correlated with the number of state switching points.

[0096] In this invention, it can be set that for every additional state switching point compared to the reference value, the playback frame rate increases by 5fps on top of the base frame rate; at the same time, in order to avoid excessive hardware requirements on the display device due to an excessively high playback frame rate setting, the upper limit of the playback frame rate is set to 60fps.

[0097] For example, the system presets a reference quantity of 5 and a base playback frame rate of 30fps;

[0098] If the number of state switching points on the posture change curve corresponding to a certain feature interaction sub-material is 3, and this number is less than or equal to the reference value, then the feature interaction sub-material is classified as the second feature interaction sub-material and will not enter the frame rate adjustment process.

[0099] When the number of state switching points on the posture change curve corresponding to another feature interaction sub-material is 7, which is greater than the reference value, the feature interaction sub-material is classified as the first feature interaction sub-material. The display control module increases the playback frame rate according to the number of state switching points. The playback frame rate corresponding to the feature interaction sub-material is adjusted to 30fps + (7-5) × 5fps = 40fps. In this way, the smoothness of the picture is improved, and the problems of picture shaking, ghosting and reduced clarity caused by frequent switching of eyes are reduced.

[0100] Understandably, the number of state switching points directly reflects the frequency with which the user's gaze alternates between looking up and looking down. The more state switching points there are, the more drastic the change in the user's gaze posture when viewing the corresponding interactive sub-materials, and the more easily the stability of the image is affected. In this case, increasing the playback frame rate can improve the refresh rate and smoothness of the image, reduce problems such as visual jitter, image ghosting, and decreased clarity caused by rapid perspective switching, and enhance the continuity and stability of the user's visual experience. Furthermore, it enables differentiated adaptation based on the causes of gaze posture instability, meeting the requirements of highly stable and highly immersive aerospace interactive display.

[0101] Specifically, the display control module is used to adjust the center position of the feature interactive sub-materials on the screen, wherein,

[0102] The display control module is used to determine the average of the line-of-sight pitch angles at all acquisition moments in the posture change curve corresponding to the second feature interactive sub-material as the overall offset reference angle;

[0103] The direction of adjusting the center position of the aerospace interactive material is opposite to the direction of the overall offset reference angle, and the adjustment distance of the center position is positively correlated with the angle of the overall offset reference angle.

[0104] In this invention, the initial reference position of the center of the feature interactive sub-material is the geometric center of the display screen. The geometric center of various screen resolutions is determined according to the pixel coordinate reference. The adjustment direction of the center position is opposite to the direction of the overall offset reference angle. That is, when the overall offset reference angle is positive, the center of the screen shifts vertically downward; when the overall offset reference angle is negative, the center of the screen shifts vertically upward.

[0105] For example, for a screen with a resolution of 1080p, its geometric center pixel coordinates are (960, 540), and the adjustment distance corresponding to a unit angle is 5 pixels for every 1° corresponding to the center position of the screen being adjusted from the initial reference position in the vertical direction.

[0106] If the total playback duration of the second feature interactive sub-material is 30 seconds, in the posture change curve corresponding to the interactive sub-material, the display control module reads the average line-of-sight pitch angle at all acquisition moments, and obtains the overall offset reference angle of the second feature interactive sub-material as +5°. Here, a positive value indicates that the overall line of sight is offset upward, that is, when the user watches this interactive sub-material, the overall line of sight is tilted upward by an average of 5° relative to the horizontal line of sight. Then the display control module will shift the center of the screen vertically downward, and the offset direction is opposite to the upward direction of the overall offset reference angle. The adjustment distance of the center position of the screen is 5×5=25 pixels, that is, the center of the screen is shifted vertically downward by 25 pixels, that is, the center of the screen is adjusted from the initial geometric center (960, 540) to the pixel coordinates (960, 515) vertically downward, thus completing the adjustment of the center position of the screen.

[0107] Understandably, using the average line-of-sight pitch angle at all acquisition moments in the attitude change curve as the overall offset reference angle can objectively reflect the overall line-of-sight offset direction and magnitude of the group of users during the viewing of the material. By adjusting the center position of the screen in the opposite direction to the overall offset reference angle, and with the adjustment distance being positively correlated with the size of the overall offset reference angle, dynamic compensation and correction can be made for the visual offset caused by the user's continuous head tilting or tilting. This makes the offset screen center more closely match the user's line-of-sight center, thereby effectively reducing the posture burden on the user's neck and eyes, reducing the instability of the viewing angle and visual fatigue caused by the long-term deviation of the line of sight from the eye level reference. While ensuring the integrity of the immersive aerospace interactive experience, it significantly improves the stability of the displayed image and the viewing comfort.

[0108] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0109] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A display system based on image interaction, characterized in that, include: An interactive motion capture module is used to acquire the user's horizontal line of sight during the viewing of aerospace interactive materials, and to determine the line of sight pitch angle at several moments based on the horizontal line of sight. The information aggregation module, which is connected to the interactive action capture module, is used to divide the aerospace interactive material into several interactive sub-materials, to count the line-of-sight pitch angle of several users during the experience of the interactive sub-materials, and to draw the attitude change curve of the line-of-sight pitch angle as the playback progress of the interactive sub-materials changes. The information aggregation module divides each interactive sub-material into segments according to the playback timeline of the aerospace interactive material, and calculates the average line-of-sight pitch angle of all users at each collection time during the playback period of each interactive sub-material. The information aggregation module plots the attitude change curve corresponding to each interactive sub-material with the playback progress time as the horizontal axis and the average line-of-sight pitch angle as the vertical axis. The feature filtering module, which is connected to the information aggregation module, is used to determine the feature interactive sub-materials based on the feature state duration of the posture change curve corresponding to the interactive sub-materials. The feature filtering module pre-sets a tilt angle threshold and an elevation angle threshold, and respectively counts the first duration when the elevation angle is greater than or equal to the elevation angle threshold and the second duration when the tilt angle is less than or equal to the tilt angle threshold in the posture change curve corresponding to each interactive sub-material. The sum of the first duration and the second duration is recorded as the feature state duration. The feature filtering module is used to calculate the ratio of the feature state duration to the total playback duration of the interactive sub-material. If the ratio is greater than a preset ratio reference value, the feature filtering module determines the interactive sub-material as a feature interactive sub-material. An interactive classification module, which is connected to the feature filtering module, is used to mark state switching points on the posture change curve corresponding to the feature interactive sub-material; and to determine the feature category of the feature interactive sub-material based on the number of state switching points. The interaction classification module traverses the coordinate points of the posture change curve corresponding to the feature interaction sub-materials, and marks the coordinate points in the posture change curve where the vertical coordinate changes from positive to negative or from negative to positive and passes through the horizontal axis as the state switching points. The display control module, which is connected to the interaction classification module, is used to determine the display adjustment method of the feature interaction sub-materials according to the feature category of the feature interaction sub-materials. The display adjustment method is to adjust the playback frame rate of the aerospace interaction material; or to adjust the center position of the screen of the feature interaction sub-materials according to the change of the line of sight pitch angle of the attitude change curve.

2. The image-interactive display system according to claim 1, characterized in that, The interactive motion capture module is used to determine the line-of-sight pitch angle at several moments, wherein... The interactive action capture module uses the user's horizontal line of sight looking straight ahead as a reference, records the angle of upward deviation from the horizontal line of sight as the line of sight elevation angle, and the angle of downward deviation from the horizontal line of sight as the line of sight depression angle. It collects the line of sight elevation angle or line of sight depression angle during the user experience process at preset time intervals, and determines the line of sight elevation angle corresponding to several collection moments.

3. The image-interactive display system according to claim 1, characterized in that, The interaction classification module is used to determine the feature categories of the feature interaction sub-materials, wherein, The interactive classification module is used to count the number of state switching points and compare the number with a preset reference value. If the number of state switching points is greater than the reference value, the interaction classification module will divide the feature interaction sub-material into the first feature interaction sub-material. If the number of state switching points is less than or equal to the reference value, the interaction classification module will divide the feature interaction sub-material into the second feature interaction sub-material.

4. The image-interactive display system according to claim 3, characterized in that, The display control module is used to determine the display adjustment method for the feature interactive sub-materials, wherein, If the feature interaction sub-material is classified as the first feature interaction sub-material, then the display control module determines that the display adjustment method for the feature interaction sub-material is to adjust the playback frame rate of the aerospace interaction material; If the feature interaction sub-material is classified as the second feature interaction sub-material, the display control module determines that the display adjustment method for the feature interaction sub-material is to adjust the center position of the feature interaction sub-material according to the change in the viewing angle of the posture change curve.

5. The image-interactive display system according to claim 4, characterized in that, The playback frame rate is positively correlated with the number of state switching points.

6. The image-interactive display system according to claim 4, characterized in that, The display control module is used to adjust the center position of the interactive feature sub-materials on the screen, wherein, The display control module is used to determine the average of the line-of-sight pitch angles at all acquisition moments in the posture change curve corresponding to the second feature interactive sub-material as the overall offset reference angle; The direction of adjusting the center position of the aerospace interactive material is opposite to the direction of the overall offset reference angle, and the adjustment distance of the center position is positively correlated with the angle of the overall offset reference angle.

Images