A method, device, terminal and medium for processing images displayed on a spherical screen

Abstract

The embodiments of the present application relate to the fields of spherical screen display and image processing, and provide a method, device, terminal and medium for processing spherical screen display images. The method comprises: acquiring k images to be displayed of a target display scene through multiple acquisition cameras; fusing the k images to be displayed to obtain a reference display image for spherical screen display; acquiring attribute information of a target spherical screen; performing three-dimensional processing on the reference display image according to the attribute information to obtain a target display image; and displaying the target display image through the target spherical screen, so that the images to be displayed can be fused and three-dimensionalized, and the three-dimensional target display image can be directly displayed, thereby improving the accuracy when displaying spherical screen images.

Description

Technical Field

[0001] The present application relates to the field of spherical screen display and image processing technology, and in particular to a method, device, terminal and medium for processing spherical screen display images. Background Art

[0002] With the development of science and technology, dome screens are increasingly used in a variety of image and video display scenarios. At present, the playback mode of LED dome screens mostly adopts the combination playback mode of image sequence frames. For example, since the display area of ​​LED dome screens is very large, multiple images are usually collected when the original display image is collected and then fused for display. In the existing scheme, the image to be displayed is usually converted into a sequence frame image through experience or multiple tests. The image obtained after conversion usually loses image quality, resulting in low accuracy when displaying the image. Summary of the invention

[0003] The embodiments of the present application provide a method, device, terminal and medium for processing images displayed on a spherical screen, which can fuse and three-dimensionally process the images to be displayed, and directly display the three-dimensional target display images, thereby improving the accuracy of displaying images on the spherical screen.

[0004] A first aspect of an embodiment of the present application provides a method for processing a spherical screen display image, the method comprising:

[0005] Acquire k images to be displayed of the target display scene through multiple acquisition cameras;

[0006] Performing fusion processing on the k images to be displayed to obtain a reference display image for spherical screen display;

[0007] Get the attribute information of the target ball screen;

[0008] Performing three-dimensional processing on the reference display image according to the attribute information to obtain a target display image;

[0009] The target display image is displayed through the target ball screen.

[0010] In this example, k images to be displayed of the target display scene are acquired through multiple acquisition cameras, the k images to be displayed are fused to obtain a reference display image for spherical screen display, attribute information of the target spherical screen is acquired, the reference display image is three-dimensionalized according to the attribute information to obtain a target display image, and the target display image is displayed through the target spherical screen. Therefore, the images to be displayed can be fused and three-dimensionalized, and the three-dimensional target display image can be directly displayed, thereby improving the accuracy of spherical screen image display.

[0011] In a possible implementation, the fusing process of the k images to be displayed to obtain a reference display image for spherical screen display includes:

[0012] Obtain the splicing positions of k images to be displayed and obtain the splicing position information;

[0013] Performing preliminary splicing processing on k images to be displayed according to the splicing position information to obtain an intermediate splicing display image;

[0014] Extracting an image of a stitching area of ​​the intermediate stitching display image to obtain a first stitching area image;

[0015] Performing contour correction and fusion processing on the first stitching area image to obtain a second stitching area image;

[0016] The second stitching area image and the intermediate stitching display image are fused to obtain the reference display image.

[0017] In a possible implementation, performing contour correction and fusion processing on the first stitching area image to obtain the second stitching area image includes:

[0018] Extracting scene overlapping regions from the first stitching region images to obtain a set of overlapping region pairs;

[0019] Obtaining a weight matrix of pixels in a target overlapping region pair, where the target overlapping region is any one of the overlapping region pair sets;

[0020] Determine the Gaussian blur value of the corresponding pixel point according to the weight matrix of the pixel points in the overlapping area pair to obtain a Gaussian blurred image of the overlapping area pair;

[0021] Perform contour comparison on the Gaussian blurred images of the overlapping area pairs to obtain a contour overlapping pixel point group;

[0022] Performing fusion processing on the contour pixel points in the contour pixel point group according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain a target splicing area;

[0023] Repeat the above method of obtaining the weight matrix of the pixel points in the target overlapping area pair, until the contour pixel points in the contour pixel point group are merged according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain the target stitching area, until the target stitching area corresponding to each overlapping area pair is obtained;

[0024] Performing first-order stitching processing on the target stitching area to obtain a reference stitching area image set;

[0025] A second-order stitching process is performed on the reference stitching area images in the reference stitching area image set to obtain a second stitching area image.

[0026] In a possible implementation, the three-dimensionalizing the reference display image according to the attribute information to obtain the target display image includes:

[0027] According to the attribute information, a virtual ball screen is constructed, wherein the attribute information of the virtual ball screen is the same as that of the target ball screen;

[0028] Calculate and obtain the projection area of ​​the virtual spherical screen in a plane with the origin of the spherical bottom directly facing the spherical screen as the tangent point;

[0029] Processing the pixels of the reference display image and the pixels of the projection area accordingly to obtain an intermediate two-dimensional display image of the reference display image in the projection area;

[0030] Processing the pixel points of the intermediate two-dimensional display image and the pixel points of the corresponding spherical screen segmentation area accordingly to obtain an intermediate three-dimensional display image;

[0031] Obtaining the angle between the projection of the vertical line between the center of the virtual spherical screen and the pixel module of the virtual spherical screen and the horizontal plane, obtaining the spherical center angle corresponding to each pixel block, and the pixel block corresponds to the pixel point in the intermediate three-dimensional display image one by one;

[0032] Determine pixel adjustment parameters of corresponding pixel points of the intermediate three-dimensional display image according to the spherical center angle of each pixel block, and obtain a pixel adjustment parameter set;

[0033] Adjusting corresponding pixel values ​​according to pixel adjustment parameters in the pixel adjustment parameter set to obtain a target pixel value set;

[0034] The target display image is determined according to the intermediate three-dimensional display image and the target pixel value set.

[0035] In a possible implementation, determining the pixel adjustment parameters of the pixel points corresponding to the intermediate 3D display image according to the spherical center angle of each pixel block to obtain a pixel adjustment parameter set includes:

[0036] Determine the spherical curvature information corresponding to each pixel block according to each pixel block and the corresponding spherical center angle, and obtain a spherical curvature set;

[0037] Determining the display influence of each pixel block relative to other pixel blocks according to the spherical radians in the spherical radian set and the horizontal coordinate values ​​of the corresponding pixel blocks, and obtaining a display influence set;

[0038] The pixel adjustment parameters of the corresponding pixel blocks are determined according to the display influence in the display influence set to obtain a pixel adjustment parameter set.

[0039] A second aspect of an embodiment of the present application provides a device for processing a spherical screen display image, the device comprising:

[0040] An acquisition unit, used for acquiring k images to be displayed of a target display scene through multiple acquisition cameras;

[0041] A fusion unit, used for fusing the k images to be displayed to obtain a reference display image for spherical screen display;

[0042] A second acquisition unit, used to acquire attribute information of the target spherical screen;

[0043] A processing unit, configured to perform three-dimensional processing on the reference display image according to the attribute information to obtain a target display image;

[0044] A display unit is used to display the target display image through the target ball screen.

[0045] In a possible implementation, the fusion unit is specifically used to:

[0046] Obtain the splicing positions of k images to be displayed and obtain the splicing position information;

[0047] Performing preliminary splicing processing on k images to be displayed according to the splicing position information to obtain an intermediate splicing display image;

[0048] Extracting an image of a stitching area of ​​the intermediate stitching display image to obtain a first stitching area image;

[0049] Performing contour correction and fusion processing on the first stitching area image to obtain a second stitching area image;

[0050] The second stitching area image and the intermediate stitching display image are fused to obtain the reference display image.

[0051] In a possible implementation manner, in the aspect of performing contour correction and fusion processing on the first stitching area image to obtain the second stitching area image, the fusion unit is specifically used to:

[0052] Extracting scene overlapping regions from the first stitching region images to obtain a set of overlapping region pairs;

[0053] Obtaining a weight matrix of pixels in a target overlapping region pair, where the target overlapping region is any one of the overlapping region pair sets;

[0054] Determine the Gaussian blur value of the corresponding pixel point according to the weight matrix of the pixel points in the overlapping area pair to obtain a Gaussian blurred image of the overlapping area pair;

[0055] Perform contour comparison on the Gaussian blurred images of the overlapping area pairs to obtain a contour overlapping pixel point group;

[0056] Performing fusion processing on the contour pixel points in the contour pixel point group according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain a target splicing area;

[0057] Repeat the above method of obtaining the weight matrix of the pixel points in the target overlapping area pair, until the contour pixel points in the contour pixel point group are merged according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain the target stitching area, until the target stitching area corresponding to each overlapping area pair is obtained;

[0058] Performing first-order stitching processing on the target stitching area to obtain a reference stitching area image set;

[0059] A second-order stitching process is performed on the reference stitching area images in the reference stitching area image set to obtain a second stitching area image.

[0060] In a possible implementation, the processing unit is specifically configured to:

[0061] According to the attribute information, a virtual ball screen is constructed, wherein the attribute information of the virtual ball screen is the same as that of the target ball screen;

[0062] Calculate and obtain the projection area of ​​the virtual spherical screen in a plane with the origin of the spherical bottom directly facing the spherical screen as the tangent point;

[0063] Processing the pixels of the reference display image and the pixels of the projection area accordingly to obtain an intermediate two-dimensional display image of the reference display image in the projection area;

[0064] Processing the pixel points of the intermediate two-dimensional display image and the pixel points of the corresponding spherical screen segmentation area accordingly to obtain an intermediate three-dimensional display image;

[0065] Obtaining the angle between the projection of the vertical line between the center of the virtual spherical screen and the pixel module of the virtual spherical screen and the horizontal plane, obtaining the spherical center angle corresponding to each pixel block, and the pixel block corresponds to the pixel point in the intermediate three-dimensional display image one by one;

[0066] Determine pixel adjustment parameters of corresponding pixel points of the intermediate three-dimensional display image according to the spherical center angle of each pixel block, and obtain a pixel adjustment parameter set;

[0067] Adjusting corresponding pixel values ​​according to pixel adjustment parameters in the pixel adjustment parameter set to obtain a target pixel value set;

[0068] The target display image is determined according to the intermediate three-dimensional display image and the target pixel value set.

[0069] In a possible implementation, in determining the pixel adjustment parameters of the pixel points corresponding to the intermediate 3D display image according to the spherical center angle of each pixel block to obtain the pixel adjustment parameter set, the processing unit is specifically configured to:

[0070] Determine the spherical curvature information corresponding to each pixel block according to each pixel block and the corresponding spherical center angle, and obtain a spherical curvature set;

[0071] Determining the display influence of each pixel block relative to other pixel blocks according to the spherical radians in the spherical radian set and the horizontal coordinate values ​​of the corresponding pixel blocks, and obtaining a display influence set;

[0072] The pixel adjustment parameters of the corresponding pixel blocks are determined according to the display influence in the display influence set to obtain a pixel adjustment parameter set.

[0073] A third aspect of an embodiment of the present application provides a terminal, comprising a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the step instructions in the first aspect of the embodiment of the present application.

[0074] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiments of the present application.

[0075] A fifth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0077] Figure 1 A flowchart of a method for processing a spherical screen display image is provided for an embodiment of the present application;

[0078] Figure 2 A splicing schematic diagram is provided for an embodiment of the present application;

[0079] Figure 3 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0080] Figure 4 A structural schematic diagram of a device for processing images displayed on a spherical screen is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0082] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0083] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0084] In order to better understand the processing method of a spherical screen display image provided in an embodiment of the present application, the processing method of a spherical screen display image in an existing scheme is first briefly introduced below. In the existing scheme, when performing a spherical screen display, the position coordinates of the marked pixels of the display image and the actual displayed image coordinates are usually obtained and mapped to obtain the actual display coordinate points of the display image, which are filled into the actual display coordinate points for display. However, the image is not processed, and only the coordinate system transformation and other processing are performed, which reduces the effect when the image is displayed. Alternatively, the spherical screen is virtualized, and virtual cutting and image corresponding conversion are performed to obtain the final display image, but it does not take into account that the spherical screen display screen is usually a large display screen. Displaying only through a single image will cause the image area to be over-stretched during display, and a high-resolution display effect cannot be obtained.

[0085] In order to solve the above-mentioned problems, an embodiment of the present application provides a method for processing a spherical screen display image, which can fuse and three-dimensionally process the image to be displayed, and directly display the three-dimensional target display image, thereby improving the accuracy of the spherical screen image display and improving the display effect.

[0086] See also Figure 1 , Figure 1 A flowchart of a method for processing a spherical screen display image is provided for an embodiment of the present application. Figure 1 As shown, the method includes:

[0087] 101. Acquire k images to be displayed of a target display scene through multiple acquisition cameras.

[0088] The specific method is to use a camera to collect multiple sets of static images of real scenes to obtain images to be displayed. Wherein k is a positive integer. This application is illustrated by taking k as 4 as an example, which does not constitute a limitation on this application. Optionally, four cameras can be used to collect static images of the target scene, and the four cameras can respectively collect images of four different areas in the target scene to obtain four images to be displayed. Since the four cameras will inevitably overlap the collection areas when performing image collection, therefore, when performing image fusion processing later, it is necessary to process the overlapping areas in the image to avoid image overlap during subsequent spherical screen display, which affects the display effect.

[0089] 102. Perform fusion processing on the k images to be displayed to obtain a reference display image for spherical screen display.

[0090] The stitching position information of k images to be displayed can be obtained, and then a preliminary stitching process is performed according to the stitching position information to obtain an intermediate stitching display image, an image of the stitching area of ​​the intermediate stitching display image is extracted, and contour correction and fusion processing is performed on it to obtain a stitching area image, and finally the corresponding part of the intermediate stitching display image is replaced to obtain a reference display image.

[0091] 103. Obtain attribute information of the target ball screen.

[0092] The property information of the ball screen includes the size of the ball screen, the number of pixels of the ball screen, and the number of LED pixel blocks. Specifically, one LED pixel block corresponds to one pixel point, and the pixel block is approximately equal to one point.

[0093] 104. Perform three-dimensional processing on the reference display image according to the attribute information to obtain a target display image.

[0094] Among them, a virtual ball screen can be constructed according to the attribute information, and the attribute information of the virtual ball screen is the same as that of the target ball screen. That is, a virtual ball screen identical to the target ball screen is constructed. The projection area of ​​the virtual ball screen in the plane with the origin of the ball bottom facing the ball screen as the tangent point is calculated, and the reference display image is processed accordingly to obtain an intermediate two-dimensional display image, and then the intermediate two-dimensional display image is converted into an intermediate three-dimensional display image, and the intermediate three-dimensional display image is adjusted to obtain the target display image. This improves the accuracy of determining the target display image.

[0095] 105. Display the target display image through the target ball screen.

[0096] In this example, k images to be displayed of the target display scene are acquired through multiple acquisition cameras, the k images to be displayed are fused to obtain a reference display image for spherical screen display, attribute information of the target spherical screen is acquired, the reference display image is three-dimensionalized according to the attribute information to obtain a target display image, and the target display image is displayed through the target spherical screen. Therefore, the images to be displayed can be fused and three-dimensionalized, and the three-dimensional target display image can be directly displayed, thereby improving the accuracy of spherical screen image display.

[0097] In a possible implementation, this example uses four images to be displayed as an example for explanation. Specifically, a method for fusing k images to be displayed to obtain a reference display image for spherical screen display includes:

[0098] A1. Obtain the splicing positions of k images to be displayed and obtain the splicing position information;

[0099] A2, performing preliminary splicing processing on k images to be displayed according to the splicing position information to obtain an intermediate splicing display image;

[0100] A3, extracting an image of the stitching area of ​​the intermediate stitching display image to obtain a first stitching area image;

[0101] A4, performing contour correction and fusion processing on the first stitching area image to obtain a second stitching area image;

[0102] A5. Fusing the second stitching area image and the intermediate stitching display image to obtain the reference display image.

[0103] The position of the image to be displayed in the reference display image can be determined by the position of the camera corresponding to the k images to be displayed, so that the splicing position can be determined according to the position of the image to be displayed in the reference display image to obtain the splicing position information. The splicing position can be the position where the display image is adjacent to other images to be displayed. The k images to be displayed are spliced ​​adjacent to each other to obtain an intermediate spliced ​​display image.

[0104] Here, four images to be displayed are used as an example for explanation. Specifically, after the four images to be displayed are spliced, a complete rectangular image can be formed, which is the middle spliced ​​display image. The splicing area can be an image within a preset distance range symmetrical to the splicing line as the center line. Figure 2 As shown, the first stitching area image may be in a cross shape.

[0105] The scene overlapping area can be extracted from the first stitching area image to obtain a set of overlapping area pairs, as shown in the following example: Figure 2 As shown, the overlapping area pairs are horizontal overlapping area pairs and vertical overlapping area pairs. After determining the overlapping area pairs, the Gaussian blur value of the corresponding pixel point is determined for the weight matrix of the pixel points in the overlapping area pairs to perform blurring processing on them. After the blurring processing, the contour comparison is performed to obtain the contour overlapping pixel point group. The pixel points in the contour overlapping pixel point group are averaged to obtain the pixel values ​​of the pixel points in the final target splicing area. When performing the first splicing, horizontal splicing can be used. After the horizontal splicing is completed, the same method as the above method is used for vertical splicing, thereby obtaining the second splicing area image.

[0106] After the second stitching area image is obtained, the image at the stitching position in the intermediate display image is replaced with the second stitching area image, thereby obtaining a reference display image, thereby improving the accuracy of obtaining the reference display image.

[0107] In a possible implementation, a method for performing contour correction and fusion processing on the first stitching area image to obtain a second stitching area image includes:

[0108] B1, extracting scene overlapping areas from the first stitching area image to obtain a set of overlapping area pairs;

[0109] B2. Obtaining a weight matrix of pixels in a target overlapping region pair, where the target overlapping region is any one of the overlapping region pair sets;

[0110] B3, determining the Gaussian blur value of the corresponding pixel point according to the weight matrix of the pixel point in the overlapping area pair, so as to obtain the Gaussian blurred image of the overlapping area pair;

[0111] B4, performing contour comparison on the Gaussian blurred images of the overlapping area pairs to obtain a contour overlapping pixel point group;

[0112] B5, performing fusion processing on the contour pixel points in the contour pixel point group according to the shooting parameter information of the camera corresponding to the contour pixel points, to obtain a target splicing area;

[0113] B6, repeatedly executing the above method of obtaining the weight matrix of the pixel points in the target overlapping area pair, until the contour pixel points in the contour pixel point group are merged according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain the target stitching area, until the target stitching area corresponding to each overlapping area pair is obtained;

[0114] B7, performing first-order stitching processing on the target stitching area to obtain a reference stitching area image set;

[0115] B8. Perform a second-order stitching process on the reference stitching area images in the reference stitching area image set to obtain a second stitching area image.

[0116] There are 2k-4 overlapping area pairs in the overlapping area set, and k is an integer greater than or equal to 4. Therefore, the number of collected images cannot be less than 4, and each time a camera is added, it should be increased by an integer multiple of 2.

[0117] like Figure 2 As shown, Figure 2 The T1 image in the image and the adjacent area in the T2 image are an overlapping area pair (for example, area 12 is an overlapping area pair). The weight matrix of the pixels in each overlapping area pair can be obtained by using the Gaussian weight matrix extraction method, and then the Gaussian blur value is calculated based on the weight matrix combined with the pixel values ​​of the neighboring pixels to obtain the Gaussian blurred image of the overlapping area pair. Then, contour extraction and contour comparison are performed to obtain a contour overlapping pixel group.

[0118] The pixel points in the outline overlapping pixel point group are averaged to obtain the pixel values ​​of the pixel points in the final target stitching area. When performing the first-order stitching, horizontal stitching can be used, that is, the horizontal overlapping area pairs are stitched to obtain a reference stitching area image set, which includes the unstitched images of the longitudinal features, and then a second-order stitching process is performed to obtain a second stitching area image. Specifically, the same stitching method can be used to perform the second-order stitching process to obtain the second stitching area image.

[0119] In this example, the image quality can be improved by performing Gaussian blur processing on the overlapping area, and then contour extraction is performed to obtain a group of contour overlapping pixel points, which are then stitched together, thereby improving the accuracy of acquiring the image of the second stitching area.

[0120] In a possible implementation, a method for performing three-dimensional processing on the reference display image according to the attribute information to obtain a target display image includes:

[0121] C1. constructing a virtual ball screen according to the attribute information, wherein the attribute information of the virtual ball screen is the same as that of the target ball screen;

[0122] C2. Calculate and obtain the projection area of ​​the virtual spherical screen in a plane with the origin of the spherical bottom directly facing the spherical screen as the tangent point;

[0123] C3, performing corresponding processing on the pixel points of the reference display image and the pixel points of the projection area to obtain an intermediate two-dimensional display image of the reference display image in the projection area;

[0124] C4, performing corresponding processing on the pixel points of the intermediate two-dimensional display image and the pixel points of the corresponding spherical screen segmentation area to obtain an intermediate three-dimensional display image;

[0125] C5, obtaining the angle between the projection of the vertical line between the center of the virtual spherical screen and the pixel module of the virtual spherical screen and the horizontal plane, and obtaining the spherical center angle corresponding to each pixel block, and the pixel block corresponds to the pixel point in the intermediate 3D display image one by one;

[0126] C6. Determine pixel adjustment parameters of pixel points corresponding to the intermediate three-dimensional display image according to the spherical center angle of each pixel block, and obtain a pixel adjustment parameter set;

[0127] C7. Adjusting corresponding pixel values ​​according to the pixel adjustment parameters in the pixel adjustment parameter set to obtain a target pixel value set;

[0128] C8. Determine the target display image according to the intermediate 3D display image and the target pixel value set.

[0129] The property information of the ball screen includes the size of the ball screen, the number of pixels of the ball screen, the number of LED pixel blocks, and one LED pixel block corresponds to one pixel point, and the pixel block is approximately equal to one point. Therefore, a virtual ball screen can be constructed according to the property information.

[0130] The origin of the spherical bottom facing the spherical screen can be understood as taking the notch surface of the spherical screen as the reference surface, and the intersection of the straight line passing through the center of the notch surface and perpendicular to the notch surface and the spherical screen is the origin of the spherical bottom.

[0131] The pixel points of the reference display image can be processed accordingly based on the resolution corresponding to the projection area to obtain an intermediate two-dimensional display image. Since the projection points correspond to the points on the virtual spherical screen one by one, the intermediate two-dimensional display image can be mapped to an intermediate three-dimensional display image that can be initially displayed according to the mapping relationship. Since the spherical screen is composed of pixel blocks, the pixel block is usually a plane. Although it appears spherical in the display, it is actually a spherical shape spliced ​​by planes. Therefore, when displaying, it is necessary to consider the display influence between pixel blocks and distortion.

[0132] Specifically, the spherical center angle of the pixel block can be extracted, and the pixel adjustment parameter of the corresponding pixel point of the intermediate 3D display image can be determined according to the spherical center angle, and finally the pixel value can be adjusted by using the pixel adjustment parameter to obtain the target display image. The target pixel value can be determined by multiplying the pixel adjustment parameter and the corresponding pixel value, and then the pixel value of the corresponding pixel point in the intermediate 3D display image is replaced by the target pixel value, so as to obtain the target display image.

[0133] In this example, when the spherical screen is displaying images, different pixel blocks will cause certain interference when performing light-emitting display. At this time, correction processing can be performed based on the spherical center angle of the pixel block, thereby improving the effect of the target display image when displaying.

[0134] In a possible implementation, a method for determining pixel adjustment parameters of pixel points corresponding to an intermediate 3D display image according to a spherical center angle of each pixel block to obtain a pixel adjustment parameter set includes:

[0135] D1. Determine the spherical curvature information corresponding to each pixel block according to each pixel block and the corresponding spherical center angle, and obtain a spherical curvature set;

[0136] D2. Determine the display influence of each pixel block relative to other pixel blocks according to the spherical radians in the spherical radian set and the horizontal coordinate values ​​of the corresponding pixel blocks, and obtain a display influence set;

[0137] D3. Determine pixel adjustment parameters of corresponding pixel blocks according to the display influence in the display influence set to obtain a pixel adjustment parameter set.

[0138] The spherical curvature information can be determined based on the position information of each pixel block and the spherical center angle. Since the display is performed on the inner area of ​​the spherical screen, the pixel blocks will affect each other. If no correction is performed, the light between the displayed images will be contaminated, which will greatly reduce the display effect.

[0139] Therefore, the influence between pixel blocks can be determined based on the spherical curvature and the horizontal coordinate value of the pixel block, and the influence between a pixel block and other pixel blocks can be calculated respectively, and then the influence is weighted to obtain the final display influence of the pixel point. Specifically, the area facing the pixel blocks can be determined based on the spherical curvature, the horizontal coordinate value and the spherical center angle. The larger the facing area, the greater the influence, and the smaller the facing area, the smaller the influence. The greater the distance between the pixel blocks, the smaller the influence, and the smaller the distance, the greater the influence, so that the display influence can be determined. Finally, the pixel value is adjusted as a whole according to the display influence, and the pixel adjustment parameters of each pixel block are generated to obtain a pixel adjustment parameter set.

[0140] In this example, the spherical curvature information corresponding to each pixel block is determined based on each pixel block and the corresponding spherical center angle to obtain a spherical curvature set, the display influence of each pixel block relative to other pixel blocks is determined based on the spherical curvature in the spherical curvature set and the horizontal coordinate value of the corresponding pixel block to obtain a display influence degree set, the pixel adjustment parameters of the corresponding pixel block are determined based on the display influence in the display influence degree set to obtain a pixel adjustment parameter set, which can improve the accuracy of determining the pixel adjustment parameters.

[0141] For the above embodiments, please refer to Figure 3 , Figure 3 A schematic diagram of the structure of a terminal provided in an embodiment of the present application, such as Figure 3 As shown, it includes a processor, an input device, an output device and a memory, which are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and the program includes instructions for executing the following steps;

[0142] Acquire k images to be displayed of the target display scene through multiple acquisition cameras;

[0143] Performing fusion processing on the k images to be displayed to obtain a reference display image for spherical screen display;

[0144] Get the attribute information of the target ball screen;

[0145] Performing three-dimensional processing on the reference display image according to the attribute information to obtain a target display image;

[0146] The target display image is displayed through the target ball screen.

[0147] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that in order to realize the above functions, the terminal includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0148] The embodiment of the present application can divide the terminal into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0149] In line with the above, see Figure 4 , Figure 4 The present invention provides a schematic diagram of a device for processing images displayed on a spherical screen. Figure 4 As shown, the device comprises:

[0150] An acquisition unit 401 is used to acquire k images to be displayed of a target display scene through multiple acquisition cameras;

[0151] A fusion unit 402 is used to perform fusion processing on the k images to be displayed to obtain a reference display image for spherical screen display;

[0152] The second acquisition unit 403 is used to acquire the attribute information of the target spherical screen;

[0153] A processing unit 404 is used to perform three-dimensional processing on the reference display image according to the attribute information to obtain a target display image;

[0154] The display unit 405 is used to display the target display image through the target ball screen.

[0155] In a possible implementation, the fusion unit 402 is specifically configured to:

[0156] Obtain the splicing positions of k images to be displayed and obtain the splicing position information;

[0157] Performing preliminary splicing processing on k images to be displayed according to the splicing position information to obtain an intermediate splicing display image;

[0158] Extracting an image of a stitching area of ​​the intermediate stitching display image to obtain a first stitching area image;

[0159] Performing contour correction and fusion processing on the first stitching area image to obtain a second stitching area image;

[0160] The second stitching area image and the intermediate stitching display image are fused to obtain the reference display image.

[0161] In a possible implementation, in performing contour correction and fusion processing on the first stitching area image to obtain the second stitching area image, the fusion unit 402 is specifically used to:

[0162] Extracting scene overlapping regions from the first stitched region image to obtain a set of overlapping region pairs (2k-4 overlapping region pairs, where k is an integer greater than or equal to 4);

[0163] Obtaining a weight matrix of pixels in a target overlapping region pair, where the target overlapping region is any one of the overlapping region pair sets;

[0164] Determine the Gaussian blur value of the corresponding pixel point according to the weight matrix of the pixel points in the overlapping area pair to obtain a Gaussian blurred image of the overlapping area pair;

[0165] Perform contour comparison on the Gaussian blurred images of the overlapping area pairs to obtain a contour overlapping pixel point group;

[0166] Performing fusion processing on the contour pixel points in the contour pixel point group according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain a target splicing area;

[0167] Repeat the above method of obtaining the weight matrix of the pixel points in the target overlapping area pair, until the contour pixel points in the contour pixel point group are merged according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain the target stitching area, until the target stitching area corresponding to each overlapping area pair is obtained;

[0168] Performing first-order stitching processing on the target stitching area to obtain a reference stitching area image set;

[0169] A second-order stitching process is performed on the reference stitching area images in the reference stitching area image set to obtain a second stitching area image.

[0170] In a possible implementation, the processing unit 404 is specifically configured to:

[0171] According to the attribute information, a virtual ball screen is constructed, wherein the attribute information of the virtual ball screen is the same as that of the target ball screen;

[0172] Calculate and obtain the projection area of ​​the virtual spherical screen in a plane with the origin of the spherical bottom directly facing the spherical screen as the tangent point;

[0173] Processing the pixels of the reference display image and the pixels of the projection area accordingly to obtain an intermediate two-dimensional display image of the reference display image in the projection area;

[0174] Processing the pixel points of the intermediate two-dimensional display image and the pixel points of the corresponding spherical screen segmentation area accordingly to obtain an intermediate three-dimensional display image;

[0175] Obtaining the angle between the projection of the vertical line between the center of the virtual spherical screen and the pixel module of the virtual spherical screen and the horizontal plane, obtaining the spherical center angle corresponding to each pixel block, and the pixel block corresponds to the pixel point in the intermediate three-dimensional display image one by one;

[0176] Determine pixel adjustment parameters of corresponding pixel points of the intermediate three-dimensional display image according to the spherical center angle of each pixel block, and obtain a pixel adjustment parameter set;

[0177] Adjusting corresponding pixel values ​​according to pixel adjustment parameters in the pixel adjustment parameter set to obtain a target pixel value set;

[0178] The target display image is determined according to the intermediate three-dimensional display image and the target pixel value set.

[0179] In a possible implementation, in determining the pixel adjustment parameters of the pixel points corresponding to the intermediate 3D display image according to the spherical center angle of each pixel block to obtain the pixel adjustment parameter set, the processing unit 404 is specifically configured to:

[0180] Determine the spherical curvature information corresponding to each pixel block according to each pixel block and the corresponding spherical center angle, and obtain a spherical curvature set;

[0181] Determining the display influence of each pixel block relative to other pixel blocks according to the spherical radians in the spherical radian set and the horizontal coordinate values ​​of the corresponding pixel blocks, and obtaining a display influence set;

[0182] The pixel adjustment parameters of the corresponding pixel blocks are determined according to the display influence in the display influence set to obtain a pixel adjustment parameter set.

[0183] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the methods for processing a spherical screen display image as recorded in the above method embodiments.

[0184] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any method for processing a spherical screen display image as recorded in the above method embodiments.

[0185] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0186] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0187] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0188] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0189] In addition, the functional units in the various embodiments of the application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0190] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including several instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0191] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0192] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for processing a spherical screen display image, characterized in that: The method comprises: Acquire k images to be displayed of the target display scene through multiple acquisition cameras; Performing fusion processing on the k images to be displayed to obtain a reference display image for spherical screen display; Get the attribute information of the target ball screen; According to the attribute information, a virtual ball screen is constructed, wherein the attribute information of the virtual ball screen is the same as that of the target ball screen; Calculate and obtain the projection area of ​​the virtual spherical screen in a plane with the origin of the spherical bottom directly facing the spherical screen as the tangent point; Processing the pixels of the reference display image and the pixels of the projection area accordingly to obtain an intermediate two-dimensional display image of the reference display image in the projection area; Processing the pixel points of the intermediate two-dimensional display image and the pixel points of the corresponding spherical screen segmentation area accordingly to obtain an intermediate three-dimensional display image; Obtaining the angle between the projection of the vertical line between the center of the virtual spherical screen and the pixel module of the virtual spherical screen and the horizontal plane, obtaining the spherical center angle corresponding to each pixel block, and the pixel block corresponds to the pixel point in the intermediate three-dimensional display image one by one; Determine the spherical curvature information corresponding to each pixel block according to each pixel block and the corresponding spherical center angle, and obtain a spherical curvature set; Determine the facing area between the pixel blocks according to the spherical curvature set, the horizontal coordinate value corresponding to each pixel block, and the spherical center angle corresponding to each pixel block, to obtain a facing area set; Determine the display influence of each pixel block relative to other pixel blocks according to the set of directly facing areas, and obtain a set of display influences; Determining pixel adjustment parameters of corresponding pixel blocks according to the display influence in the display influence set to obtain a pixel adjustment parameter set; Adjusting corresponding pixel values ​​according to pixel adjustment parameters in the pixel adjustment parameter set to obtain a target pixel value set; Determine a target display image according to the intermediate three-dimensional display image and the target pixel value set; The target display image is displayed through the target ball screen.

2. The method for processing a spherical screen display image according to claim 1, characterized in that: The step of fusing the k images to be displayed to obtain a reference display image for spherical screen display includes: Obtain the splicing positions of k images to be displayed and obtain the splicing position information; Performing preliminary splicing processing on k images to be displayed according to the splicing position information to obtain an intermediate splicing display image; Extracting an image of a stitching area of ​​the intermediate stitching display image to obtain a first stitching area image; Performing contour correction and fusion processing on the first stitching area image to obtain a second stitching area image; The second stitching area image and the intermediate stitching display image are fused to obtain the reference display image.

3. The method for processing a spherical screen display image according to claim 2, characterized in that: The step of performing contour correction and fusion processing on the first stitching area image to obtain a second stitching area image includes: Extracting scene overlapping regions from the first stitching region images to obtain a set of overlapping region pairs; Obtaining a weight matrix of pixels in a target overlapping region pair, where the target overlapping region is any one of the overlapping region pair sets; Determine the Gaussian blur value of the corresponding pixel point according to the weight matrix of the pixel points in the overlapping area pair to obtain a Gaussian blurred image of the overlapping area pair; Perform contour comparison on the Gaussian blurred images of the overlapping area pairs to obtain a contour overlapping pixel point group; Performing fusion processing on the contour pixel points in the contour pixel point group according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain a target splicing area; Repeat the above method of obtaining the weight matrix of the pixel points in the target overlapping area pair, until the contour pixel points in the contour pixel point group are merged according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain the target stitching area, until the target stitching area corresponding to each overlapping area pair is obtained; Performing first-order stitching processing on the target stitching area to obtain a reference stitching area image set; A second-order stitching process is performed on the reference stitching area images in the reference stitching area image set to obtain a second stitching area image.

4. A device for processing images displayed on a spherical screen, characterized in that: The device comprises: An acquisition unit, used for acquiring k images to be displayed of a target display scene through multiple acquisition cameras; A fusion unit, used for fusing the k images to be displayed to obtain a reference display image for spherical screen display; A second acquisition unit, used to acquire attribute information of the target spherical screen; A processing unit, configured to construct a virtual ball screen according to the attribute information, wherein the attribute information of the virtual ball screen is the same as that of the target ball screen; Calculate and obtain the projection area of ​​the virtual spherical screen in a plane with the origin of the spherical bottom directly facing the spherical screen as the tangent point; Processing the pixels of the reference display image and the pixels of the projection area accordingly to obtain an intermediate two-dimensional display image of the reference display image in the projection area; Processing the pixel points of the intermediate two-dimensional display image and the pixel points of the corresponding spherical screen segmentation area accordingly to obtain an intermediate three-dimensional display image; Obtaining the angle between the projection of the vertical line between the center of the virtual spherical screen and the pixel module of the virtual spherical screen and the horizontal plane, obtaining the spherical center angle corresponding to each pixel block, and the pixel block corresponds to the pixel point in the intermediate three-dimensional display image one by one; Determine the spherical curvature information corresponding to each pixel block according to each pixel block and the corresponding spherical center angle, and obtain a spherical curvature set; Determine the facing area between the pixel blocks according to the spherical curvature set, the horizontal coordinate value corresponding to each pixel block, and the spherical center angle corresponding to each pixel block; and obtain the facing area set; Determine the display influence of each pixel block relative to other pixel blocks according to the set of directly facing areas, and obtain a set of display influences; Determining pixel adjustment parameters of corresponding pixel blocks according to the display influence in the display influence set to obtain a pixel adjustment parameter set; Adjusting corresponding pixel values ​​according to pixel adjustment parameters in the pixel adjustment parameter set to obtain a target pixel value set; Determine a target display image according to the intermediate three-dimensional display image and the target pixel value set; A display unit is used to display the target display image through the target ball screen.

5. The device for processing images displayed on a spherical screen according to claim 4, characterized in that: The fusion unit is specifically used for: Obtain the splicing positions of k images to be displayed and obtain the splicing position information; Performing preliminary splicing processing on k images to be displayed according to the splicing position information to obtain an intermediate splicing display image; Extracting an image of a stitching area of ​​the intermediate stitching display image to obtain a first stitching area image; Performing contour correction and fusion processing on the first stitching area image to obtain a second stitching area image; The second stitching area image and the intermediate stitching display image are fused to obtain the reference display image.

6. The device for processing images displayed on a spherical screen according to claim 5, characterized in that: In the aspect of performing contour correction and fusion processing on the first stitching area image to obtain the second stitching area image, the fusion unit is specifically used for: Extracting scene overlapping regions from the first stitching region images to obtain a set of overlapping region pairs; Obtaining a weight matrix of pixels in a target overlapping region pair, where the target overlapping region is any one of the overlapping region pair sets; Determine the Gaussian blur value of the corresponding pixel point according to the weight matrix of the pixel points in the overlapping area pair to obtain a Gaussian blurred image of the overlapping area pair; Perform contour comparison on the Gaussian blurred images of the overlapping area pairs to obtain a contour overlapping pixel point group; Performing fusion processing on the contour pixel points in the contour pixel point group according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain a target splicing area; Repeat the above method of obtaining the weight matrix of the pixel points in the target overlapping area pair, until the contour pixel points in the contour pixel point group are merged according to the shooting parameter information of the camera corresponding to the contour pixel points to obtain the target stitching area, until the target stitching area corresponding to each overlapping area pair is obtained; Performing first-order stitching processing on the target stitching area to obtain a reference stitching area image set; A second-order stitching process is performed on the reference stitching area images in the reference stitching area image set to obtain a second stitching area image.

7. A terminal, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method for processing a spherical screen display image as described in any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method for processing a spherical screen display image according to any one of claims 1 to 3.

Images