A method, device and system for processing image display on a special-shaped display screen

Abstract

The present invention discloses a method, device and system for processing image display of a special-shaped display screen, which belongs to the field of display image processing, and includes: making the vertices of each screen inscribed on the same sphere; establishing a spherical coordinate system, obtaining the posture parameters of each screen in the spliced ​​screen; constructing an image display system for the special-shaped display screen, the image display system includes a video image generation system and an image processing device; the video image generation system is used to generate a spherical video image in real time, and then intercept the image according to the camera model and the posture parameters and resolution of the special-shaped display screen, and then flatten the spherical figure and output it, and add an image processing device at the back end of the video image generation system to restore the distorted plane figure to the effect of the spherical figure after geometric correction, and output it to the special-shaped display screen for display. The present invention enables the flat display screen used for simulation training to also obtain the effect of spherical display.

Description

Technical Field

[0001] The present invention relates to the field of display image processing, and more specifically, to a method, device and system for display processing of images on a special-shaped display screen. Background Art

[0002] According to the imaging principle of the human eye, when the display screen is designed as a sphere, people can get the visual experience closest to the real scene when observing the video image at the center of the sphere. Therefore, in the field of simulation training, engineers are trying to design the visual display system into a sphere and expand the display field of view as much as possible, so that trainers can get a stronger sense of immersion and achieve better training results. Fortunately, with the continuous development of display technology, this spherical screen visual display system can be realized. Among them, the spherical screen display system based on projection technology is the earliest developed and the most mature technology at present. Its main working principle is: first design and manufacture a spherical structure, arrange and install multiple projectors on the back of the spherical structure, and multiple projectors project images onto the spherical surface and realize display, and then perform nonlinear distortion correction and edge fusion processing on the images projected by the multiple projectors, and finally display a normal visual image on the spherical structure. Due to the existence of the projection optical path, this spherical screen display system cannot realize full field display. In addition, projection technology belongs to passive display technology, so its display brightness and contrast are limited, which can no longer meet the higher requirements of simulation training for display fidelity. Therefore, in recent years, an LED ball screen display system based on the active display principle has been developed. The LED display screen has the advantages of high brightness, high contrast, wide color gamut, high refresh rate and full field of view, which can present better immersion and realism than projection technology in the visual display process of simulated training. In addition, its long life, easy installation and easy maintenance make LED as a visual display system have great research significance and application value. For example, invention CN202022818262.2 (a display system for an LED spherical screen) and invention CN202110634391.2 (a display method for an LED spherical screen) describe the use of LED light panels to splice into a spherical display screen and a display method. The basic feature is that the display sphere is first divided into longitude and latitude. Therefore, the LED light panels are designed to be fan-shaped, and from the equator to the pole, the arc edge of each LED light panel gradually becomes smaller. At the same time, the arrangement of LED lamp beads and the division of driving channels are also carried out according to the longitude and latitude division method, which reduces the difficulty of system display driving and ensures that the spherical screen can display video images normally. Finally, a full-field spherical screen vision display system based on LED display technology is realized. However, the dome screen also has two major disadvantages. First, such a design requires a large number of LED light panels, usually between 20 and 35 types. Different types of LED light panels need to be developed and designed separately, so the standardization is low and the cost is very high. Second, after dividing the channels in this way, the equatorial region usually has the highest pixels, and the pixels gradually decrease towards the poles. Therefore, there is a large pixel sampling for each channel image, and the image pixels have an increased loss, which affects the display clarity.

[0003] Furthermore, due to the cost sensitivity of most simulation training systems, many visual display systems cannot be designed as spherical screens. Therefore, splicing walls, multi-fold screens, column screens based on flat panel displays, or polyhedron screens that have wrapping characteristics but still exist in a flat surface, still have a large market. How to effectively improve the display effects of such screens and enhance their role in simulation training is also a key research direction for engineers and scientists. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and to provide a method, device and system for processing image display on a special-shaped display screen. Through the relevant methods of graphic image processing, a flat display screen used for simulation training can also obtain a spherical display effect, thereby achieving the purpose of improving immersion and enabling trainees to obtain better training effects.

[0005] The object of the present invention is achieved through the following solutions:

[0006] A method for processing image display on a special-shaped display screen comprises the following steps:

[0007] Pre-adjust the bending angle of the spliced ​​screens in the special-shaped display screen so that the vertices of each screen are inscribed on the same sphere; and establish a spherical coordinate system, based on which, a camera model is used to simulate the imaging principle of the human eye to obtain the posture parameters of each screen in the spliced ​​screen;

[0008] An image display system for the special-shaped display screen is constructed, and the image display system includes a video image generation system and an image processing device; the video image generation system is used to generate a spherical video image in real time, and then capture the image according to the camera model and in combination with the posture parameters and resolution of the special-shaped display screen, and then flatten the spherical figure and output it. The image processing device is added to the back end of the video image generation system to geometrically correct the distorted plane figure to restore the effect of the spherical figure, and the corrected flat figure is output to the special-shaped display screen for display.

[0009] Furthermore, the method of pre-adjusting the bending angle of the screen splicing in the special-shaped display screen so that the vertices of each screen are inscribed on the same sphere specifically includes the following sub-steps: if the vertices of each screen are not on the same sphere, establishing multiple coordinate systems, and then translating the coordinate systems to unify them.

[0010] Furthermore, the establishing of the spherical coordinate system specifically includes establishing the spherical coordinate system with the center of the sphere as the starting point.

[0011] Furthermore, the attitude parameters specifically include: a horizontal field of view angle FOV-X, a vertical field of view angle FOV-Y, a horizontal offset △X, a vertical offset △Y, a heading angle α, a pitch angle β and a roll angle θ.

[0012] Furthermore, the special-shaped display screen includes a tri-fold screen, and the initial posture of the virtual camera is specified to be in the equatorial direction. The tri-fold screen is symmetrical about the equator, and the horizontal offset △X, vertical offset △Y, pitch angle β and roll angle θ are all 0°. If the middle screen of the tri-fold screen is taken as the starting point, the heading angle is 0°, the heading angle of the left screen is -α, and the heading angle of the right screen is +α.

[0013] Furthermore, the image processing device is added to the back end of the video image generation system to perform geometric correction on the distorted plane image to restore the effect of a spherical image, which specifically includes the following sub-steps:

[0014] Step 1: Project a point P' on the plane onto the spherical surface to obtain point P, and establish a mapping relationship between point P' and P. Specifically, the following sub-steps are included: Select the middle screen to establish a plane coordinate system. The heading angle of the middle screen is 0°, then the OV axis of the spherical coordinate system is perpendicular to the middle screen. Assume that the distance between the screen and the center of the sphere is d. Calculate the x coordinate of point P' according to the following formula:

[0015]

[0016] In the formula, t represents the x-axis coordinate of the projection line of the spherical equator line projected onto the plane screen in the plane coordinate system, which is a fixed value; d′ represents the projection length of point P′ to the equatorial plane; lon represents the longitude of spherical point P (equal to the angle between the projection point of the UOV plane and the plane WOV); lat represents the latitude of point P (equal to the angle between the projection line of line OP on the UOV plane and line OP); that is, the plane point is represented by parameters x and y, and the spherical point is represented by parameters lon and lat, so the mapping relationship between plane point P′ and spherical point P is actually the relationship between (x, y) and (lon, lat);

[0017] According to the extreme points of the plane screen, t and d in formula (1) are calculated. It is known that the resolution of the plane screen is W×H, where W represents the horizontal resolution, corresponding to the horizontal field of view, and H represents the vertical resolution, corresponding to the vertical field of view. It is easy to know that the four vertices A, B, C and D on the plane screen are also on the spherical surface. Therefore, points A and C correspond to H pixels on the plane screen, which represent the maximum vertical field of view FOV on the spherical surface. Y ; Points E and F correspond to W pixels on the flat screen, and on the spherical surface they represent the maximum horizontal field of view FOV X ; Let N = ∠AOE and S = ∠COE, then:

[0018]

[0019] Considering that the plane screen is symmetric about the equator, N = S = FOV Y / 2, formula (2) is simplified to:

[0020]

[0021] From formula (3), we can know that t and d are fixed values. Substituting formula (3) into formula (1), we can get the mapping relationship of x pixels between the screen and the projection sphere:

[0022]

[0023] Step 2, processing the mapping relationship of y pixels between the screen and the projection sphere, specifically includes sub-steps: It is known that the OV axis of the spherical coordinate system is perpendicular to the plane screen, then the y coordinate of point P' is calculated according to the following formula:

[0024]

[0025] Where s represents the y coordinate of the line perpendicular to the equatorial projection line and intersecting the OV axis in the plane coordinate system, and f′ represents the projection length of point P′ to the WOV surface. The plane screen is symmetrical about the WOV surface, and combined with the d value calculated by formula (3), it is calculated:

[0026]

[0027] From formulas (4) and (6), we can analyze that for the parallels of equal latitude on the sphere, due to the influence of cos(lon), as lon increases, x decreases, so the parallels of the same latitude are mapped on the plane as concave lines with high sides and low in the middle; at the same time, y has nothing to do with the latitude lat, and for different longitudes, its y is a fixed value, so it will not be deformed and appears as a straight line;

[0028] Step 3: For other screens on both sides of the middle screen, different continuous images are obtained by adjusting the heading angle in the video image generation system to ensure that all screens can display continuous video image images, and then step 1 and step 2 are performed. That is, the processing of other screens on both sides of the middle screen is exactly the same as the above. In order to ensure that all screens can display continuous video image images, different continuous images are obtained by adjusting the heading angle in the video image generation system, and then the above processing is performed on the image processing device.

[0029] Furthermore, the image processing device is added to the back end of the video image generation system to perform geometric correction on the distorted plane image to restore the effect of a spherical image, which specifically includes the following sub-steps:

[0030] Step 1: The special-shaped display screen is a football-shaped display screen, which is composed of a regular pentagon and five regular hexagonal screens; each pentagon and hexagon in the special-shaped display screen is driven by an independent video channel, and the video image generation system generates a spherical video image in real time;

[0031] The regular pentagonal and hexagonal screens are treated as circular screens, that is, the circumscribed circles of the regular pentagonal and hexagonal screens are taken as processing objects. First, a point Q′ on the plane is projected onto the sphere to obtain point Q, and a mapping relationship between point Q′ and Q is established, specifically including: selecting a pentagonal screen to establish a plane coordinate system, assuming that the distance between the screen and the center of the sphere is h, and obtaining the x and y coordinates of point Q′ as:

[0032]

[0033] Similarly, the coordinates of u and v of point Q on the sphere are:

[0034]

[0035] Among them, R is the radius of the spherical screen; h and R are both fixed constants. Since the circumscribed circle of the regular pentagonal screen is taken as the processing object, the corresponding horizontal field of view angle is equal to the vertical field of view angle, that is, FOV X =FOV Y , from which it is easy to calculate h = R·cos(FOV X / 2), the point Q on the sphere and the point Q′ on the pentagon are only related to the angles lat and lon, based on which the mapping relationship between the two points is calculated;

[0036] Step 2: All hexagons next to the pentagon are processed according to the method of step 1, specifically including: selecting a hexagonal screen to establish a plane coordinate system, and taking the circumscribed circle of the regular hexagon as the processing object, assuming that the distance between the screen and the center of the sphere is h', and obtaining the x' and y' coordinates of point B' as:

[0037]

[0038] Similarly, the coordinates of u′ and v′ of point B on the sphere are:

[0039]

[0040] Among them, R is the radius of the spherical screen; h′ and R are both fixed constants. Taking the circumscribed circle of the regular hexagonal screen as the processing object, the horizontal field of view angle is equal to the vertical field of view angle, that is, FOV′ X =FOV′ Y , calculate h′=R·cos(FOV′ X / 2), point B on the sphere and point B′ on the hexagon are only related to the angles lat and lon, based on which the mapping relationship between the two points is calculated.

[0041] Step three, in the video image generation system, different continuous pictures are obtained by adjusting the heading angle, pitch angle and roll angle to ensure that all screens can display continuous video image pictures. The pictures are presented as circles of different sizes, and then the above processing is performed on the image processing device, that is, step one and step two are executed. Finally, the corresponding pentagonal and hexagonal images are captured from the circular picture for driving.

[0042] Furthermore, in the special-shaped display screen, the spliced ​​screen includes a liquid crystal display screen, an LED display screen, a front projection display screen or a rear projection display screen; when the special-shaped display screen adopts a projector, it also includes the following sub-steps: performing edge fusion processing at the spliced ​​edges; when the special-shaped display screen is spliced ​​by liquid crystal display screens, OLED display screens or LED display screens, each screen can directly display the video image without the need for edge fusion processing.

[0043] An image display processing device for a special-shaped display screen, comprising:

[0044] The posture parameter acquisition unit is used to pre-adjust the bending angle of the screen splicing in the special-shaped display screen so that the vertices of each screen are inscribed on the same sphere; and establish a spherical coordinate system, based on which the camera model is used to simulate the imaging principle of the human eye to obtain the posture parameters of each screen in the spliced ​​screen;

[0045] A video image generation unit, used to generate a spherical video image in real time, then intercept the image according to the camera model and in combination with the posture parameters and resolution of the special-shaped display screen, and then flatten the spherical image and output it;

[0046] An image processing unit, used for geometrically correcting the distorted plane image output by the video image generating unit to restore the effect of a spherical image;

[0047] The display unit is used to display the image output after geometric correction by the image processing unit.

[0048] A special-shaped display screen image display processing system comprises the special-shaped display screen image display processing device as described above.

[0049] The beneficial effects of the present invention include:

[0050] 1. The present invention uses a method of geometric deformation of graphics to allow a plane graphic to display the content of a spherical image, thereby optimizing the display effect of the existing special-shaped screen without increasing excessive costs, especially improving the immersion and realism of the visual display in the field of simulation training, and can also greatly improve the training effect.

[0051] Second, the present invention is applicable to various types of special-shaped screens and is also compatible with various display modes such as liquid crystal display, OLED display, projection display and LED display. The display effect can be improved by adding a set of image processing equipment externally and adjusting the parameters of the video image generation system. Most of the existing visual display systems can be quickly optimized and upgraded.

[0052] 3. The present invention proposes a new solution for the realization of the LED ball screen system. Under the existing manufacturing process conditions, firstly, only 1-2 kinds of LED light panels need to be designed and generated, and then they are spliced ​​into a spherical surface according to the football surface or the Gothenburg polyhedron, and finally the image displayed on the plane is corrected into a spherical graphic by image processing, and finally the same display effect as the ball screen display system is obtained. Since this method only needs to design 1-2 kinds of LED light panels, the screen is more standardized, the cost of production and manufacturing is lower, and it also has the advantages of easy disassembly and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 is a flow chart of the steps of the method of the present invention;

[0055] Figure 2 This is a system composition diagram of Embodiment 1 of the present invention;

[0056] Figure 3 This is a schematic diagram of the geometric correction principle of Embodiment 1 of the present invention;

[0057] Figure 4 This is a schematic diagram of the geometric correction principle of Embodiment 2 of the present invention;

[0058] Figure 5 This is a schematic diagram of the plane expansion of the special-shaped screen of Example 2 of the present invention. DETAILED DESCRIPTION

[0059] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0060] Example 1

[0061] For a large screen composed of multiple rectangular screens, in order to make the observer have a better sense of immersion, it is usually designed in a wrapping form, and we also call this large screen a multi-fold screen. The spliced ​​rectangular screens can be LCD screens, LED screens, front projection screens or rear projection screens, etc. The advantages of this type of screen are easy to implement and low cost. The disadvantage is that since the visual image content is directly displayed, the image will have geometric distortion, resulting in a difference between the display effect and the real view, which also leads to a reduction in display fidelity.

[0062] This embodiment specifically proposes an image display processing method (mainly for images on special-shaped display screens) based on the imaging principle of the human eye. By performing geometric deformation and edge fusion processing on the displayed image, the observer can obtain a visual effect as close as possible to the real scene, further achieving the purpose of obtaining a realistic display effect in the simulation training process. Figure 1 The specific implementation method is as follows:

[0063] 1) In the multi-fold screen embodiment, taking the tri-fold screen as an example, first adjust the bending angles of the three rectangular screens so that the four vertices of each screen are inscribed on the same sphere (if they are not on the same sphere, it is necessary to establish multiple coordinate systems, and then translate the coordinate systems to unify them. For ease of understanding, this situation is not described here).

[0064] 2) A spherical coordinate system is established with the center of the sphere as the starting point. This embodiment conceives to use a camera model to simulate the imaging principle of the human eye, and it is easy to obtain the posture parameters of each screen in the three-fold screen: horizontal field of view angle (FOV-X), vertical field of view angle (FOV-Y), horizontal offset (△X), vertical offset (△Y), heading angle (α), pitch angle (β) and roll angle (θ). For ease of understanding, this embodiment stipulates that the initial posture of the virtual camera is in the equatorial direction, and the multi-fold screen is symmetrical about the equator, so the horizontal offset (△X), vertical offset (△Y), pitch angle (β) and roll angle (θ) are all 0°. If the middle screen in the three-fold screen is used as the starting point, the heading angle is 0°, the heading angle of the left screen is -α, and the heading angle of the right screen is +α. It should be noted that in actual implementation, it can be set according to actual conditions and is not limited to this.

[0065] 3) If Figure 5As shown, the three-fold screen image display system conceived in this embodiment includes three parts: a video image generation system, an image processing device, and a special-shaped display screen (such as a three-fold screen). Among them, the video image generation system generates spherical video images in real time. Since the existing hardware devices transmit video images in the form of rectangular graphics, the video image generation system also intercepts the image based on the camera model, combined with the above-mentioned posture parameters and resolution of the special-shaped display screen, and then flattens the spherical graphics and outputs them. If the special-shaped display screen directly displays the plane graphics, serious geometric distortion will occur. Therefore, an image processing device is added to the back end of the video image generation system, which mainly performs geometric inverse transformation (geometric correction) on the distorted plane graphics, and finally outputs them to the special-shaped display screen for display.

[0066] 4) In the image processing device, a geometric correction process is involved. This geometric correction process is specifically conceived as a process of projecting a point P' on a plane onto a spherical surface to obtain point P, that is, a process of establishing a mapping relationship between point P' and P. Figure 5 As shown, a screen is selected to establish a plane coordinate system. Assuming that the distance between the screen and the center of the sphere is d, the x coordinate of point P′ can be obtained as:

[0067]

[0068] Wherein, t represents the x-axis coordinate of the projection line of the spherical equator line projected onto the plane screen in the plane coordinate system, which is a fixed value; d′ represents the projection length of point P′ to the equatorial plane; lon represents the longitude of spherical point P (equal to the angle between the projection point on the UOV plane and the plane WOV); lat represents the latitude of point P (equal to the angle between the projection line of line OP on the UOV plane and line OP); that is, the plane point is represented by the parameters x and y, and the spherical point is represented by the parameters lon and lat, so the mapping relationship from plane point P′ to spherical point P is actually the relationship between (x, y) and (lon, lat).

[0069] First, t and d in formula (1) are calculated based on the extreme points of the plane screen. It is known that the resolution of the plane screen is W×H, where W represents the horizontal resolution, corresponding to the horizontal field of view, and H represents the vertical resolution, corresponding to the vertical field of view. It is easy to know that the four vertices A, B, C and D on the plane screen are also on the spherical surface. Therefore, the two points AC correspond to H pixels on the plane screen, which represent the maximum vertical field of view FOV on the spherical surface. Y EF points correspond to W pixels on the plane screen, and represent the maximum horizontal field of view FOV on the sphere. X . Let N = ∠AOE and S = ∠COE, then:

[0070]

[0071] Considering that the flat screen is symmetric about the equator, we have N = S = FOVY / 2, formula (2) can be simplified as:

[0072]

[0073] From formula (3), we can know that t and d are fixed values. Finally, by substituting formula (3) into formula (1), we can get the mapping relationship of x pixels between the screen and the projection sphere:

[0074]

[0075] The processing of the mapping relationship of y pixels between the screen and the projection sphere specifically includes: knowing that the OV axis of the spherical coordinate system is perpendicular to the plane screen, the y coordinate of point P' can be calculated according to the following company:

[0076]

[0077] Where s represents the y coordinate of the line perpendicular to the equatorial projection line and intersecting the OV axis in the plane coordinate system, and f′ represents the projection length of point P′ to the WOV surface. It is easy to know that the plane screen is symmetrical about the WOV surface, and combined with the d value calculated by formula (3), it can be calculated:

[0078]

[0079] From formulas (4) and (6), we can analyze that for the parallels of equal latitudes on the sphere, due to the influence of cos(lon), as lon increases, x decreases, so the parallels of the same latitude are mapped on the plane as concave lines with high sides and low in the middle; at the same time, y is independent of the latitude lat, and for different longitudes, its y is a fixed value, so it will not be deformed and appears as a straight line.

[0080] The processing of other screens on both sides of the middle screen is exactly the same as above. In order to ensure that all screens can display continuous video image frames, different continuous frames are obtained by adjusting the heading angle in the video image generation system, and then the above processing is performed on the image processing device.

[0081] 5) If the tri-fold screen is made of LCD screens, OLED screens or LED screens, each screen can directly display the video image without edge fusion processing. If the tri-fold screen uses a projector, edge fusion processing is required at the spliced ​​edges.

[0082] Example 2

[0083] In cost-sensitive areas, a plane-based polyhedron display screen has also been widely used. This polyhedron includes multiple pentagonal screens spliced ​​into a semi-wrapped large screen, or a combination of pentagons and hexagons spliced ​​into a large screen similar to a football surface or a Gothenburg polyhedron. The display method mostly uses a projector, and is implemented in a rear projection manner. With the continuous development of LED display technology, as the dot pitch continues to shrink and the display density continues to increase, LEDs can also be used to design such polyhedron display screens. When LEDs are used for design, in addition to the advantages of good display effects, another advantage is that only 1 to 2 types of LED light boards need to be designed, which not only improves the standardization of display screens, but also greatly reduces costs. Therefore, without increasing too much cost, how to make this type of display screen further obtain a display effect similar to a ball screen display screen becomes very valuable. This embodiment proposes a processing method, which specifically includes the following contents:

[0084] 1) Take the football-shaped display screen as an example, which includes 1 regular pentagon and 5 regular hexagonal screens. By adjusting the screen angle, it is ensured that the vertices of each screen are on the same sphere (if they are not on the same sphere, it involves the need to establish multiple coordinate systems, and then translate the coordinate systems to unify them. For ease of understanding, this situation is not repeated here).

[0085] 2) A spherical coordinate system is established with the center of the sphere as the starting point. We use the camera model to simulate the imaging principle of the human eye, and it is easy to obtain the posture parameters of each screen in the special-shaped screen: horizontal field of view angle (FOV-X), vertical field of view angle (FOV-Y), horizontal offset (△X), vertical offset (△Y), heading angle (α), pitch angle (β) and roll angle (θ). For ease of understanding, this implementation case stipulates that the initial posture of the virtual camera is in the equatorial direction, and the pentagon is at the pole. For the pentagon, its horizontal offset (△X), vertical offset (△Y) and roll angle (θ) are all 0°, the pitch angle (β) is 90°, and the horizontal field of view (FOV-X) and vertical field of view (FOV-Y) are obtained by the circumscribed circle of the pentagon, and FOV-X=FOV-Y; for any hexagon, its horizontal offset (△X) and vertical offset (△Y) are all 0°, the pitch angle is β, the roll angle is θ, and the heading angle is a+n·72°, where n is an integer from 0 to 4. Similarly, the horizontal field of view (FOV-X) and vertical field of view (FOV-Y) are obtained by the circumscribed circle of the hexagon, and FOV-X=FOV-Y.

[0086] 3) Similarly, the special-shaped screen image display system of this embodiment includes three parts: a video image generation system, an image processing device, and a special-shaped display screen. Each pentagon and hexagon in the special-shaped screen is driven by an independent video channel, wherein the video image generation system generates a spherical video image in real time. Since the existing hardware devices transmit video images in the form of rectangular graphics, the video image generation system also first captures the image based on the camera model and combines the above-mentioned posture parameters and resolution of the special-shaped display screen, and then flattens and outputs the spherical graphics. If the special-shaped display screen directly displays the plane graphics, serious geometric distortion will occur. Therefore, this embodiment adds an image processing device at the back end of the video image generation system, which mainly performs geometric inverse transformation (geometric correction) on the distorted plane graphics, and finally outputs them to the special-shaped display screen for display.

[0087] 4) In the image processing device, an improved geometric correction process is involved. The geometric correction process is specifically conceived as a process of projecting a point Q' on a plane onto a spherical surface to obtain point Q, that is, establishing a mapping relationship between point Q' and Q. Figure 5 As shown, a pentagonal screen is selected to establish a plane coordinate system. Assuming that the distance between the screen and the center of the sphere is h, the x and y coordinates of point Q′ can be obtained as:

[0088]

[0089] Similarly, the coordinates of u and v of point Q on the sphere can be obtained as:

[0090]

[0091] Among them, R is the radius of the spherical screen; h and R are both fixed constants. Since the circumscribed circle of the regular pentagonal screen is taken as the processing object, the corresponding horizontal field of view angle is equal to the vertical field of view angle, that is, FOV X =FOV Y It is also easy to calculate h = R·cos(FOV X / 2), the point Q on the sphere and the point Q′ on the pentagon are only related to the angles lat and lon, and the mapping relationship between the two points is calculated based on this; all hexagons next to the pentagon are processed according to this method, specifically including: selecting a hexagonal screen to establish a plane coordinate system, and taking the circumscribed circle of the regular hexagon as the processing object, assuming that the distance between the screen and the center of the sphere is h′, and the x′ and y′ coordinates of point B′ are obtained as follows:

[0092]

[0093] Similarly, the coordinates of u′ and v′ of point B on the sphere are:

[0094]

[0095] Among them, h′ and R are fixed constants. The difference between the pentagon and the hexagon is that the field of view angle parameter is larger. Similarly, the circumscribed circle of the regular hexagonal screen is taken as the processing object, and the horizontal field of view angle is equal to the vertical field of view angle, that is, FOV′ X =FOV′ Y It is also easy to calculate h′=R·cos(FOV′ X / 2), point B on the sphere and point B′ on the hexagon are only related to the angles lat and lon, based on which the mapping relationship between the two points is calculated.

[0096] In order to ensure that all screens can display continuous video images, different continuous images are obtained by adjusting the heading angle, pitch angle and roll angle in the video image generation system. The images are presented as circles of different sizes. The above processing is then performed on the image processing device. Finally, the corresponding pentagonal and hexagonal images are captured from the circular images for driving. Figure 5 This is a schematic diagram of the plane expansion of the special-shaped screen of Example 2 of the present invention.

[0097] 5) If the special-shaped screen is made of LED display screens, each screen can directly display the video image without edge fusion processing. From the plane expansion diagram, it can be seen that although there are areas where the images overlap at the boundaries, since each pentagon and hexagon is driven by an independent channel, the overlapping areas will not interfere with each other and can be driven directly. If the special-shaped screen uses a projector, edge fusion processing is also required at the splicing edges.

[0098] Example 3

[0099] In other embodiments, based on Embodiment 1 or Embodiment 2, a device for processing image display on a special-shaped display screen is further provided, including:

[0100] The posture parameter acquisition unit is used to pre-adjust the bending angle of the screen splicing in the special-shaped display screen so that the vertices of each screen are inscribed on the same spherical surface; and establish a spherical coordinate system, based on the spherical coordinate system, use a camera model to simulate the imaging principle of the human eye to obtain the posture parameters of each screen in the spliced ​​screen;

[0101] A video image generation unit, used to generate a spherical video image in real time according to the camera model, and to capture an image in combination with the posture parameters and resolution of the special-shaped display screen, and then to flatten and output the spherical image;

[0102] An image processing unit, used for geometrically correcting the distorted plane image output by the video image generating unit to restore the effect of a spherical image;

[0103] The display unit is used to display the image output after geometric correction by the image processing unit.

[0104] Example 4

[0105] In other embodiments, a special-shaped display screen image display processing system is also provided, including the special-shaped display screen image display processing device described in Example 3.

[0106] The above description is only the technical principle and preferred embodiment used in the present invention. It will be understood by those skilled in the art that the present invention is not limited to the specific embodiments described herein. It is possible for those skilled in the art to make various obvious changes, adjustments and substitutions without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the principles and concepts of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for processing image display on a special-shaped display screen, characterized in that: The following steps are involved: Pre-adjust the bending angle of the spliced ​​screens in the special-shaped display screen so that the vertices of each screen are inscribed on the same sphere; and establish a spherical coordinate system, based on which, a camera model is used to simulate the imaging principle of the human eye to obtain the posture parameters of each screen in the spliced ​​screen; Construct an image display system for the special-shaped display screen, the image display system comprising a video image generation system and an image processing device; the video image generation system is used to generate a spherical video image in real time, and then capture an image according to a camera model and in combination with the posture parameters and resolution of the special-shaped display screen, and then flatten and output the spherical image, and add the image processing device at the back end of the video image generation system to geometrically correct the distorted plane image to restore the effect of the spherical image, and output the corrected image to the special-shaped display screen for display; The image processing device is added to the back end of the video image generation system to perform geometric correction on the distorted plane image to restore the spherical image, which specifically includes the following sub-steps: Step 1: Move a point on the plane Projecting onto the sphere, we get Point, build Point and The mapping relationship includes the following sub-steps: Select the middle screen to establish a plane coordinate system. The heading angle of the middle screen is 0°, then the OV axis of the spherical coordinate system is perpendicular to the middle screen. Suppose the distance between the screen and the center of the sphere is , calculated according to the following formula of The coordinates are: （1） Wherein, t represents the x-axis coordinate of the projection line after the spherical equator line is projected onto the plane screen in the plane coordinate system, which is a fixed value; Indicate point The projection length to the equatorial plane; lon represents a spherical point Longitude; lat indicates a point Latitude; According to the extreme points of the plane screen, t and d in formula (1) are calculated. It is known that the resolution of the plane screen is W×H, where W represents the horizontal resolution, corresponding to the horizontal field of view, and H represents the vertical resolution, corresponding to the vertical field of view. The four vertices A, B, C and D on the plane screen are also on the spherical surface. Points A and C correspond to H pixels on the plane screen, which represents the maximum vertical field of view on the spherical surface. ; Points E and F correspond to W pixels on a flat screen, and represent the maximum horizontal field of view on a spherical surface ;remember and , then: （2） Considering the plane screen is symmetric about the equator, we have , formula (2) is simplified to: （3） From equation (3), we know that t and d are fixed values. Substituting equation (3) into equation (1), we can get the distance between the screen and the projection sphere: The mapping relationship of pixels is: （4） Step 2: Between the screen and the projection sphere The processing of the pixel mapping relationship specifically includes the following sub-steps: It is known that the OV axis of the spherical coordinate system is perpendicular to the plane screen, then the point is calculated according to the following formula of The coordinates are: （5） Where s represents the y coordinate of the line perpendicular to the equatorial projection line and intersecting the OV axis in the plane coordinate system. Indicate point The projection length to the WOV surface; the plane screen is symmetrical about the WOV surface, and combined with the d value calculated by formula (3), it is calculated: （6） Step three, for the other screens on both sides of the middle screen, different continuous images are obtained by adjusting the heading angle in the video image generation system to ensure that all screens can display continuous video image images, and then execute steps one and two.

2. The image display processing method for a special-shaped display screen according to claim 1, characterized in that: The method of pre-adjusting the bending angle of the screen splicing in the special-shaped display screen so that the vertices of each screen are inscribed on the same sphere specifically includes the following sub-steps: if the vertices of each screen are not on the same sphere, establishing multiple coordinate systems, and then translating the coordinate systems to unify them.

3. The image display processing method for a special-shaped display screen according to claim 1, characterized in that: The establishing of the spherical coordinate system specifically includes establishing the spherical coordinate system with the center of the sphere as the starting point.

4. The image display processing method for a special-shaped display screen according to claim 1, characterized in that: The attitude parameters specifically include: horizontal field of view angle FOV-X, vertical field of view angle FOV-Y, horizontal offset △X, vertical offset △Y, heading angle α, pitch angle β and roll angle θ.

5. The image display processing method for a special-shaped display screen according to claim 1, characterized in that: The special-shaped display screen includes a three-fold screen, and the initial posture of the virtual camera is specified to be in the equatorial direction. The three-fold screen is symmetrical about the equator, and the horizontal offset △X, vertical offset △Y, pitch angle β and roll angle θ are all 0°. If the middle screen of the three-fold screen is taken as the starting point, the heading angle is 0°, the heading angle of the left screen is -α, and the heading angle of the right screen is +α.

6. The image display processing method for a special-shaped display screen according to claim 1, characterized in that: The image processing device is added to the back end of the video image generation system to perform geometric correction on the distorted plane image to restore the spherical image, which specifically includes the following sub-steps: Step 1: The special-shaped display screen is a football-shaped display screen, which is composed of a regular pentagon and five regular hexagonal screens; each pentagon and hexagon in the special-shaped display screen is driven by an independent video channel, and the video image generation system generates a spherical video image in real time; Treat the regular pentagon and hexagon screens as circular screens, that is, take the circumscribed circle of the regular pentagon and hexagon screens as the processing object. First, Projecting onto the sphere, we get Point, build Point and The mapping relationship includes: selecting a pentagonal screen to establish a plane coordinate system, assuming that the distance between the screen and the center of the sphere is , get the point of and The coordinates are: (3) Similarly, we get Pointed and The coordinates are: (4) in, is the radius of the ball screen; and are fixed constants, , calculate , a point on the sphere Points on the pentagon Only with angle and Correlation, based on which the mapping relationship between the two points is calculated; Step 2: All hexagons next to the pentagon are processed according to the method of step 1, which specifically includes: selecting a hexagonal screen to establish a plane coordinate system, and taking the circumscribed circle of the regular hexagon as the processing object, assuming that the distance between the screen and the center of the sphere is , get the point of and The coordinates are: (3) Similarly, we get Pointed and The coordinates are: (4) in, is the radius of the ball screen; and are all fixed constants. Taking the circumscribed circle of the regular hexagonal screen as the processing object, the horizontal field of view angle is equal to the vertical field of view angle, that is, , calculate , a point on the sphere Points on the hexagon Only with angle and Correlation, based on which the mapping relationship between the two points is calculated; Step three, in the video image generation system, different continuous pictures are obtained by adjusting the heading angle, pitch angle and roll angle to ensure that all screens can display continuous video image pictures. The pictures are presented as circles of different sizes, and then steps one and two are executed. Finally, the corresponding pentagonal and hexagonal images are captured from the circular pictures for driving.

7. The image display processing method for a special-shaped display screen according to claim 1, characterized in that: In the special-shaped display screen, the spliced ​​screen includes a liquid crystal display screen, an LED display screen, a front projection display screen or a rear projection display screen; when the special-shaped display screen adopts a projector, it also includes the following sub-steps: performing edge fusion processing at the spliced ​​edge; when the special-shaped display screen is spliced ​​by liquid crystal display screens, OLED display screens or LED display screens, each screen can directly display the video image without the need for edge fusion processing.

8. An image display processing device for a special-shaped display screen, characterized in that: include: The posture parameter acquisition unit is used to pre-adjust the bending angle of the screen splicing in the special-shaped display screen so that the vertices of each screen are inscribed on the same sphere; and establish a spherical coordinate system, based on which the camera model is used to simulate the imaging principle of the human eye to obtain the posture parameters of each screen in the spliced ​​screen; A video image generation unit, used to generate a spherical video image in real time, then intercept the image according to the camera model and in combination with the posture parameters and resolution of the special-shaped display screen, and then flatten the spherical image and output it; An image processing unit, used for geometrically correcting the distorted plane image output by the video image generating unit to restore the effect of a spherical image; A display unit, used for displaying the image output after geometric correction by the image processing unit; The device is used to execute the image display processing method of the special-shaped display screen described in claim 1.

9. A special-shaped display screen image display processing system, characterized in that: It includes the image display processing device for the special-shaped display screen as described in claim 8.

Images