Dynamic range mapping method and apparatus for panoramic video

Abstract

The application discloses a dynamic range mapping method and device for panoramic video, and relates to the field of video image processing.The method comprises the following steps: dividing a two-dimensional panoramic projection image corresponding to any one three-dimensional panoramic image in a panoramic video to obtain a plurality of regions of the two-dimensional panoramic projection image; acquiring metadata of the two-dimensional panoramic projection image, wherein the metadata comprises metadata information units corresponding to the plurality of regions respectively, and the metadata information unit corresponding to one region comprises dynamic mapping information of the region; determining dynamic mapping information of each pixel in a current view angle range according to the dynamic mapping information of the plurality of regions respectively; and performing dynamic range mapping on each pixel according to the dynamic mapping information of each pixel in the current view angle range to obtain a two-dimensional panoramic projection image in the current view angle range, wherein the two-dimensional panoramic projection image is used for display or subsequent processing, and the method can effectively realize dynamic range mapping on the panoramic video, thereby improving the display effect of the panoramic video.

Description

Technical Field

[0001] This application relates to the field of video image processing, and more particularly to a method and apparatus for dynamic range mapping of panoramic video. Background Technology

[0002] In the field of video / image processing, dynamic range refers to the ratio of the maximum grayscale value (or brightness value) of a pixel to the minimum grayscale value.

[0003] After an electronic device captures video / images, it can display the video / images on that device or transmit them to other display devices for display. Typically, because different display devices have different dynamic ranges of pixels, the dynamic range of the captured video / images may differ from the dynamic range supported by the display device. Therefore, the dynamic range of the captured video / images needs to be adjusted to match the dynamic range supported by the display device. Only then can the video / images be successfully displayed on the display device. This process of adjusting the dynamic range of the video / images is called dynamic range mapping.

[0004] Currently, there are many mature dynamic range mapping methods for 2D video / images (i.e., acquired video / images). However, with the development of technology, 3D video / images (such as panoramic video) are becoming increasingly popular, and there is still no effective dynamic range mapping technology for 3D video / images. Summary of the Invention

[0005] This application provides a method and apparatus for dynamic range mapping of panoramic video, which can effectively realize dynamic range mapping of panoramic video, thereby improving the display effect of panoramic video.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a dynamic range mapping method for panoramic video, applied to an electronic device. The method includes: the electronic device dividing a two-dimensional panoramic projection image to obtain multiple regions of the two-dimensional panoramic projection image, wherein the two-dimensional panoramic projection image is the two-dimensional panoramic projection image corresponding to any frame of a three-dimensional panoramic image in the panoramic video; each of the multiple regions satisfies a preset condition, which includes at least one of the following conditions: adjacent pixels in the two-dimensional panoramic projection image but not adjacent pixels in the three-dimensional panoramic image are not located in the same region; and non-adjacent pixels in the three-dimensional panoramic image are not located in the same region; wherein adjacent pixels in the two-dimensional panoramic projection image include pixels in at least two adjacent projection planes of the three-dimensional panoramic image. The non-adjacent pixels in the 3D panoramic image include pixels in the 3D panoramic image corresponding to at least two non-adjacent projection surfaces; then the electronic device acquires the metadata of the 2D panoramic projection image, which includes metadata information units corresponding to multiple regions of the 2D panoramic projection image, and the metadata information unit corresponding to a region includes the dynamic mapping information of that region; and determines the dynamic mapping information of each pixel within the current viewing angle range based on the dynamic mapping information of each of the multiple regions; and performs dynamic range mapping on each pixel within the current viewing angle range based on the dynamic mapping information of each pixel within the current viewing angle range, thereby obtaining the 2D panoramic projection image within the current viewing angle range, which is used for display or subsequent processing.

[0008] In this embodiment, during the segmentation of the two-dimensional panoramic projection image, each segmented region, when meeting the aforementioned preset conditions, ensures that pixels with similar brightness (similar brightness means similar dynamic range) are grouped into the same region, while pixels with significant brightness differences (significant brightness differences mean significant dynamic range differences) are not grouped into the same region. Pixel 1 in the three-dimensional panoramic projection image might be a sky pixel, and pixel 2 might be a ground pixel. The brightness difference between sky pixels and ground pixels is significant, thus ensuring that pixel 1 and pixel 2 are not located in the same region during the segmentation of the two-dimensional panoramic projection image. Since pixels within the same region have similar dynamic ranges, the same dynamic range mapping information can be used for dynamic range mapping within the same region. Pixels in different regions may have significantly different dynamic ranges, so different dynamic range mapping information can be used for dynamic range mapping within different regions. Therefore, after dividing the two-dimensional panoramic projection image into multiple regions according to this segmentation method, subsequent processing (i.e., dynamic range mapping) for each region can adaptively process the characteristics of pixels in each region, improving the processing effect.

[0009] In this embodiment, taking advantage of the large dynamic range of panoramic video, the electronic device divides the two-dimensional panoramic projection image corresponding to the three-dimensional panoramic image into regions, and then performs dynamic range mapping for different regions. This can effectively realize the dynamic range mapping of panoramic video, thereby improving the display effect of panoramic video.

[0010] In one possible implementation, the dynamic range mapping method for panoramic video provided in this application embodiment further includes: projecting a three-dimensional panoramic image onto the surface of a polyhedron, and unfolding the polyhedron into a two-dimensional plane to obtain a two-dimensional panoramic projection image. The polyhedron includes at least one of the following: a regular tetrahedron, a regular hexahedron, a regular octahedron, a regular dodecahedron, or a regular icosahedron.

[0011] In this embodiment, the process of projecting a three-dimensional panoramic image into a two-dimensional panoramic projection image using a polyhedron includes: placing a sphere capable of representing a three-dimensional panoramic image (which is represented by a three-dimensional sphere) into the polyhedron, such that the sphere becomes the inscribed sphere of the polyhedron (it should be understood that the body center of the polyhedron coincides with the center of the sphere); then, connecting the center of the sphere to any point (pixel) on the surface of the sphere, and extending the line to intersect a surface of the polyhedron. At this time, the point on the sphere after being projected onto a surface of the polyhedron is the intersection point of the extension of the line connecting the center of the sphere and the point on the sphere with the surface of the polyhedron; or, connecting a point on the surface of the polyhedron to the center of the sphere, with the line intersecting the sphere at a point, the intersection point on the sphere and the point on the polyhedron are mutually projected. It should be understood that the pixel value of the point on the sphere is the pixel value of the intersection point on the surface of the polyhedron. Thus, following the projection process described above, all pixels on the three-dimensional sphere are projected to obtain the projected pixels of each surface of the polyhedron (in the case of interpolation of the projected pixels of the two-dimensional plane); finally, the polyhedron is unfolded into a two-dimensional plane according to certain rules, thereby obtaining a two-dimensional panoramic projection image.

[0012] The above-described unfolding of a polyhedron into a two-dimensional plane results in a two-dimensional panoramic projection image that may be irregular. In subsequent processing, optionally, the irregular projection image can be processed directly, or it can be converted into a regular image for processing. In one implementation, the smallest rectangular region enclosing the irregular image can be processed, and the areas within the rectangular region other than the projection area can be filled, for example, with preset pixel values. In another implementation, the individual facets of the irregular projection image can be stitched together to form a regular-shaped projection image (e.g., a rectangle), thus eliminating the need for pixel filling.

[0013] The embodiments of this application do not limit the above-mentioned polyhedrons to regular polyhedra, that is, the two-dimensional panoramic projection of the three-dimensional panoramic image can be composed of several polygons of different sizes.

[0014] In one possible implementation, the above-mentioned division of the two-dimensional panoramic projection image into multiple regions specifically includes: dividing the two-dimensional panoramic projection image according to a first division method to obtain multiple regions of the two-dimensional panoramic projection image, wherein the first division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of a polyhedron into one region.

[0015] For example, taking a regular hexahedron as the polyhedron used for projection, the resulting two-dimensional panoramic projection image after projecting the three-dimensional panoramic image onto the regular hexahedron includes six projection planes. The first division method is to group pixels belonging to the same surface of the polyhedron into one region; that is, to treat each projection plane of the two-dimensional panoramic projection image as a region. For example, according to the first division method, the two-dimensional panoramic projection image can be divided into six regions, namely... Figure 8 Regions A, B, C, D, E, and F are defined in the image. Each of the multiple regions obtained by dividing the two-dimensional panoramic projection image according to the first division method satisfies the above-mentioned preset conditions (including condition 1 or condition 2), that is, there are no adjacent pixels in the two-dimensional panoramic projection image that are not adjacent in the three-dimensional panoramic image and are located in the same region.

[0016] In one possible implementation, the above-mentioned division of the two-dimensional panoramic projection image to obtain multiple regions of the two-dimensional panoramic projection image specifically includes: dividing the two-dimensional panoramic projection image according to a second division method to obtain multiple regions of the two-dimensional panoramic projection image. The second division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of a polyhedron into one region to obtain multiple intermediate regions, and then divide at least one of the intermediate regions.

[0017] Taking a regular hexahedron as an example, the 2D panoramic image obtained after projecting a 3D panoramic image onto the hexahedron includes six projection planes. The second division method involves grouping pixels belonging to the same surface of the polyhedron into a single region, resulting in multiple intermediate regions. Then, at least one of these intermediate regions is further divided using the first division method described above. Each of these multiple regions, divided according to the second method, satisfies the aforementioned preset conditions (including condition 1 or condition 2), meaning that no adjacent pixels in the 2D panoramic image that are not adjacent in the 3D panoramic image reside in the same region.

[0018] In one possible implementation, the above-mentioned division of the two-dimensional panoramic projection image into multiple regions specifically includes: dividing the two-dimensional panoramic projection image according to a third division method to obtain multiple intermediate regions of the two-dimensional panoramic projection image, wherein the third division method is to perform clustering processing on all pixels of the two-dimensional panoramic projection image and assign pixels belonging to the same class to the same intermediate region; and dividing the intermediate regions that do not meet the preset conditions according to a fourth division method to obtain multiple regions; for any intermediate region that does not meet the preset conditions, the fourth division method is to divide the intermediate region along the boundary lines of at least two surfaces of the polyhedron contained in the intermediate region.

[0019] In this embodiment of the application, the process of clustering all pixels of the two-dimensional panoramic projection image according to the third partitioning method includes:

[0020] First, determine the cluster centers, which are one pixel. Optionally, the 2D panoramic projection image can be divided into multiple regions according to a preset division method. Then, determine the pixel value at the center of each region and use the pixel value at the center of each region as the cluster center. For example, assuming the 2D panoramic projection image is obtained by projecting a regular hexahedron, if the 2D panoramic projection image is divided into 6 regions, it can be divided into 6 regions according to the first division method described above, and then the pixel value at the center of each region can be determined to obtain 6 cluster centers. If the 2D panoramic projection image is divided into 12 regions, it can be divided into 12 regions according to the second division method described above, and then the pixel value at the center of each region can be determined to obtain 12 cluster centers.

[0021] Secondly, based on the determined cluster centers, a preset clustering algorithm is executed to calculate the distance between all pixels and each cluster center (this distance is calculated based on the coordinates of the pixels and the coordinates of the pixels corresponding to the cluster centers). This distance can be denoted as D. i Then, calculate the difference between the brightness value of all pixels and the brightness value of the pixel corresponding to the cluster center (or calculate the difference between the brightness value of all pixels and the average brightness value of the pixels in the category corresponding to each cluster center (i.e., the category after division according to the preset division method); or calculate the difference between the color value of all pixels and the color value of the pixel corresponding to the cluster center; or calculate the difference between the color value of all pixels and the average color value of the pixels in the category corresponding to each cluster center (i.e., the category after division according to the preset division method)). Record this difference as E. i Based on distance D i With E iThe weighted values ​​are used to assign each pixel to the category corresponding to the corresponding cluster center, resulting in a clustering result. The category with the smallest weighted value is the category corresponding to the pixel. In this clustering result, the pixels of each category form a region.

[0022] Optionally, dividing the two-dimensional panoramic projection image using the above clustering method cannot guarantee whether the multiple intermediate regions obtained meet the above preset conditions (including condition 1 or condition 2). Therefore, it is necessary to judge whether the multiple intermediate regions obtained by clustering meet the preset conditions. If all the multiple intermediate regions meet the above preset conditions, then the multiple intermediate regions are taken as the final division result, that is, the multiple intermediate regions are taken as the multiple regions after the two-dimensional panoramic projection image is divided; if there are intermediate regions among the multiple intermediate regions that do not meet the above preset conditions, the electronic device continues to divide the intermediate regions among the multiple intermediate regions that do not meet the preset conditions using the fourth division method, thereby obtaining multiple regions.

[0023] In one possible implementation, the first, second, or third partitioning method is pre-defined in the electronic device (e.g., by agreement through a protocol).

[0024] In one possible implementation, the metadata of the two-dimensional panoramic projection image also includes partitioning indication information. This partitioning indication information is used to indicate whether the partitioning method of the two-dimensional panoramic projection image is a first partitioning method, a second partitioning method, or a third partitioning method. That is, the partitioning indication information can be transmitted in the bitstream. After the generating end determines the partitioning method of the two-dimensional panoramic projection image, it carries the indication information (i.e., partitioning indication information) in the bitstream. Thus, after the display end receives the bitstream, it can know the partitioning method of the two-dimensional panoramic projection image based on the partitioning indication information in the bitstream.

[0025] It is understandable that the first and second division methods mentioned above are related, and the division result of the second division method is the result of further division based on the first division method.

[0026] In one implementation, if the candidate partitioning methods for the two-dimensional panoramic projection image include a first partitioning method and a second partitioning method, it can be determined whether further partitioning is needed after using the first partitioning method based on the content of the two-dimensional panoramic projection image, i.e., whether to use the first partitioning method or the second partitioning method.

[0027] Specifically, the following process can be used to determine whether further subdivision of the first subdivision method is needed: First, the 2D panoramic projection image is subdivided according to the first subdivision method to obtain multiple intermediate regions; second, for each of the multiple intermediate regions, the histogram of the intermediate region is calculated, and the intermediate region is further subdivided into multiple regions, and the histogram of each region is calculated. Then, the sum of the differences between the histogram of the intermediate region and the histograms of the subdivided regions is calculated; then, it is determined whether the sum of the differences is greater than a preset threshold. If the sum of the differences is greater than the preset threshold, the intermediate region needs to be further subdivided, that is, the 2D panoramic projection image is subdivided using the second subdivision method; if the sum of the differences is less than or equal to the preset threshold, the intermediate region does not need to be further subdivided, that is, the 2D panoramic projection image is subdivided using the first subdivision method. It should be understood that when the sum of the differences is greater than the preset threshold, it indicates that the brightness difference between different parts of the intermediate region is large, and further subdivision is required.

[0028] Optionally, in some cases, if the default partitioning method is the first partitioning method, relevant flags can be used in the bitstream to indicate whether to further partition. When the flag indicates that no further partitioning is to be performed, the result of partitioning using the first partitioning method is the final result. When the flag indicates that further partitioning is to be performed, the result of partitioning using the first partitioning method is further partitioned.

[0029] In another implementation, if the candidate partitioning methods for the 2D panoramic projection image include a first partitioning method, a second partitioning method, and a third partitioning method, the partitioning method to be used can be determined by performing the following process: First, partition the 2D panoramic projection image according to the first, second, and third partitioning methods; second, calculate the histogram of each region under the first, second, and third partitioning methods, as well as the histogram of the 2D panoramic projection image; then calculate the sum of the differences between the histograms of multiple regions obtained under each partitioning method and the histogram of the 2D panoramic projection image; finally, determine the partitioning method with the largest sum of histogram differences as the final partitioning method. It should be understood that a larger sum of histogram differences indicates a greater difference in brightness between the partitioned regions, thus indicating a more reasonable partitioning.

[0030] Optionally, the metadata size for each of the three partitioning methods can be calculated. The metadata includes dynamic mapping information corresponding to each region of the two-dimensional panoramic projection image. Then, based on the weighted value between the sum of the difference values ​​of the histograms corresponding to the partitioning methods and the metadata size corresponding to the partitioning methods, the partitioning method to be used is determined.

[0031] In one possible implementation, before determining the dynamic mapping information of each pixel in the current region based on the dynamic mapping information of multiple regions, the dynamic range mapping method for panoramic video provided in this application further includes: obtaining the correspondence between regions and metadata information units; and determining the metadata information units corresponding to multiple regions from the metadata based on the correspondence between regions and metadata information units, thereby determining the dynamic mapping information of multiple regions.

[0032] It is understood that, in this embodiment of the application, the approach to dynamic range mapping of panoramic video is to project all the three-dimensional panoramic images contained in the panoramic video onto a two-dimensional plane, perform dynamic range mapping on the resulting two-dimensional panoramic projected image, and then convert the dynamically range-mapped two-dimensional panoramic projected image into a three-dimensional panoramic image, thereby obtaining the dynamically range-mapped panoramic video. In other words, the metadata of the obtained two-dimensional panoramic projected image is used for dynamic range mapping of the three-dimensional panoramic image (i.e., for dynamic range mapping of the panoramic video), so the metadata of the two-dimensional panoramic projected image can be regarded as the metadata of the three-dimensional panoramic image (or the metadata of the panoramic video).

[0033] Optionally, the correspondence between regions and metadata information units can be one-to-one or many-to-one. That is, one region of the 2D panoramic projection image corresponds to one metadata information unit, or multiple regions of the 2D panoramic projection image correspond to one metadata information unit. When the brightness of multiple regions of the 2D panoramic projection image differs significantly, a different dynamic mapping information is used for each region, resulting in a one-to-one correspondence between regions and metadata information units. When the brightness of several regions is similar, these regions can use the same dynamic mapping information, resulting in a many-to-one correspondence between regions and metadata information units.

[0034] In this embodiment, a preset algorithm can also be used to determine whether the correspondence between multiple regions of the aforementioned two-dimensional panoramic image and metadata information units is a one-to-one or many-to-one relationship. For example, the following method can be used to determine which correspondence is one-to-one or many-to-one: First, cluster the multiple regions. The number of clusters can be set according to requirements, such as half or a quarter of the number of regions. For example, if the total number of regions is 8, the 8 regions can be divided into 2 categories or 4 categories, which is equivalent to obtaining 4 larger regions. Second, calculate the sum of differences between the histograms of the regions corresponding to each category in the clustering results and the histogram of the two-dimensional panoramic projection image, and calculate the sum of differences between the histograms of the original multiple regions and the histogram of the two-dimensional panoramic image. Finally, based on the sum of the differences between the two histograms, determine the correspondence between the multiple regions and the metadata information units. Is the relationship one-to-one or many-to-one? For example, when clustering 8 regions into 2 classes, the sum of the differences between the histograms of the regions corresponding to each class in the clustering results and the histograms of the 2D panoramic projection image is DiffHIS_X2. The sum of the differences between the histograms of the original multiple regions and the histograms of the 2D panoramic image is denoted as DiffHIS_org. One method to determine this is: if DiffHIS_org×(1-T2)>DiffHIS_X2, then the correspondence between the region and the metadata information unit is many-to-one; if DiffHIS_org×(1-T2)≤DiffHIS_X2, then the correspondence between the region and the metadata information unit is one-to-one. T2 is a preset threshold.

[0035] In one possible implementation, the correspondence between the aforementioned region and metadata information unit is a correspondence between the index of the metadata information unit and the index of the region; or, the correspondence between the region and metadata information unit is a correspondence between the index of the metadata unit and one or more pixel coordinates in the region.

[0036] In this embodiment, the correspondence between the index of the metadata information unit and the index of the region is essentially a traversal order relationship. For example, for a two-dimensional panoramic projection image comprising six regions, these six regions form a 2×3 rectangle (i.e., two rows, each row containing three regions). The regions are, from left to right and top to bottom, region 4, region 0, region 5, region 3, region 1, and region 2. Therefore, in the correspondence table between regions and metadata information units, traversing the six regions of the two-dimensional panoramic projection image from left to right and top to bottom allows the metadata information units corresponding to each of the six regions to be determined sequentially in the metadata. In one implementation, the traversal order of multiple regions can be indicated in the bitstream. For example, indication information indicating the traversal order can be carried in the metadata. This traversal order is not limited to the aforementioned left-to-right and top-to-bottom order; the specific traversal order depends on the location of each region in the two-dimensional panoramic projection image.

[0037] Optionally, the traversal order of the multiple regions mentioned above can be a pre-set order, or the generator can determine a traversal order from the candidate traversal orders according to a relevant algorithm, and then pass an identifier in the metadata of the bitstream to the generator to indicate the traversal order.

[0038] In this embodiment of the application, the traversal order can be determined by the following method: First, calculate the histogram of each region in the multiple regions; second, for each of the candidate traversal orders, calculate the difference sum of the histograms of adjacent regions; finally, determine the traversal method with the smallest difference sum of the histograms among the candidate traversal orders as the final traversal method.

[0039] In this embodiment, the coordinates of pixels in a region can also be carried in the metadata. Specifically, the coordinates of pixels in each region are carried in the metadata information unit corresponding to each region. After the display end divides the two-dimensional panoramic projection image into multiple regions, for one region, it determines which coordinate in the above correspondence belongs to that region, and then uses the metadata information unit corresponding to that coordinate as the metadata information unit of that region.

[0040] In one possible implementation, the dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a first preset range, where the first preset range is the range centered on the pixel to be processed (Method 1); or, the dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of the region to which the pixel corresponding to the viewpoint center of the current viewing angle range belongs in the two-dimensional panoramic projection image (Method 2); or, the dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a second preset range, where the second preset range is the range centered on the pixel corresponding to the viewpoint center of the current viewing angle range in the two-dimensional panoramic projection image (Method 3); or, the dynamic mapping information of a pixel in the current region is the dynamic mapping information of the current region (Method 4).

[0041] It should be understood that when a user watches a panoramic video, the user (head) cannot see the entire panoramic video content in their current posture, but only a certain range of video content. This range can be understood as the user's field of view. Therefore, the user's field of view is related to their current viewing posture. In this embodiment, dynamic range mapping of the panoramic video involves dynamically mapping the content within the user's current field of view. After acquiring dynamic mapping information for multiple regions, the electronic device can determine the dynamic mapping information of each pixel within the current field of view using any of the four methods described above, based on the dynamic range mapping information for multiple regions.

[0042] Optionally, in practical applications, the generating end can select a suitable method from the above four methods to determine the dynamic mapping information of the pixel to be processed according to actual needs, and then transmit the selected method to the display end in the bitstream (metadata); or the generating end and the display end can pre-determine which method to use, which is not limited in this application embodiment.

[0043] In one possible implementation, the dynamic mapping information in this application embodiment is used to map the dynamic range of pixels in a two-dimensional panoramic projection image from a first dynamic range to a second dynamic range; or, the dynamic mapping information is used to map the dynamic range of pixels in a two-dimensional panoramic projection image from a second dynamic range to a first dynamic range; wherein, the first dynamic range is greater than the second dynamic range, that is, the dynamic range mapping of pixels can be a mapping from high to low dynamic range, or a mapping from low to high dynamic range.

[0044] Correspondingly, after determining the dynamic mapping information of each pixel within the current viewing range through the method, some pixels may correspond to multiple dynamic mapping information (e.g., multiple dynamic mapping curves). After performing dynamic range mapping on the pixel, each dynamic mapping information can obtain a dynamic mapping result, thus obtaining multiple dynamic mapping results for that pixel. In this case, multiple dynamic mapping results can be processed to obtain the final dynamic mapping result. The processing method can include any of the following: taking the median of multiple dynamic mapping results as the final dynamic mapping result or performing a weighted average of multiple dynamic mapping results to obtain the final dynamic mapping result.

[0045] In one possible implementation, obtaining the metadata of the two-dimensional panoramic projection image specifically includes receiving the metadata of the two-dimensional panoramic projection image from other electronic devices. Specifically, after the generating end generates the metadata, it encodes the metadata into a bitstream and sends it to the display end, whereby the display end receives and parses the bitstream to obtain the metadata.

[0046] In one possible implementation, the dynamic range mapping method for panoramic video provided in this application embodiment further includes: performing subsequent processing on the two-dimensional panoramic projection image within the current viewing angle range, the subsequent processing including: performing three-dimensional transformation on the two-dimensional panoramic projection image within the current viewing angle range to obtain a three-dimensional panoramic image within the current viewing angle range, the three-dimensional panoramic image being used for display.

[0047] The two-dimensional panoramic projection image obtained above can be used for display. Furthermore, the two-dimensional panoramic projection image can also be used for subsequent processing, such as converting it into a three-dimensional panoramic image for display. Furthermore, after obtaining all the three-dimensional panoramic images of the panoramic video, the panoramic video can be played.

[0048] Secondly, embodiments of this application provide a dynamic range mapping device for panoramic video, including a segmentation module, an acquisition module, a determination module, and a processing module. The segmentation module is used to divide a two-dimensional panoramic projection image into multiple regions of the two-dimensional panoramic projection image. The two-dimensional panoramic projection image is the two-dimensional panoramic projection image corresponding to any frame of a three-dimensional panoramic image in the panoramic video. Each of the multiple regions satisfies a preset condition, which includes at least one of the following: pixels that are adjacent in the two-dimensional panoramic projection image but not adjacent in the three-dimensional panoramic image are not located in the same region; and pixels that are not adjacent in the three-dimensional panoramic image are not located in the same region. Adjacent pixels in the two-dimensional panoramic projection image include pixels in at least two adjacent projection surfaces of the three-dimensional panoramic image, and non-adjacent pixels in the three-dimensional panoramic image include pixels in the three-dimensional panoramic image corresponding to at least two non-adjacent projection surfaces. The acquisition module is used to acquire metadata of the two-dimensional panoramic projection image. This metadata includes metadata information units corresponding to multiple regions of the two-dimensional panoramic projection image. The metadata information unit corresponding to a region includes the dynamic mapping information of the region. The determination module is used to determine the dynamic mapping information of each pixel within the current viewing angle range based on the dynamic mapping information of each of the multiple regions. The processing module is used to perform dynamic range mapping on each pixel within the current viewing angle range based on the dynamic mapping information of each pixel within the current viewing angle range, thereby obtaining the two-dimensional panoramic projection image within the current viewing angle range. The two-dimensional panoramic projection image within the current viewing angle range is used for display or subsequent processing.

[0049] In one possible implementation, the dynamic range mapping device for panoramic video provided in this application embodiment further includes a conversion module; the conversion module is used to project a three-dimensional panoramic image onto the surface of a polyhedron and unfold the polyhedron into a two-dimensional plane to obtain a two-dimensional panoramic projection image, wherein the polyhedron includes at least one of the following: a regular tetrahedron, a regular hexahedron, a regular octahedron, a regular dodecahedron, or a regular icosahedron.

[0050] In one possible implementation, the aforementioned partitioning module is specifically used to partition the two-dimensional panoramic projection image according to a first partitioning method to obtain multiple regions of the two-dimensional panoramic projection image. The first partitioning method is to partition the pixels of the two-dimensional panoramic projection image that belong to the same surface of a polyhedron into one region.

[0051] In one possible implementation, the above-mentioned segmentation module is specifically used to segment the two-dimensional panoramic projection image according to the second segmentation method to obtain multiple regions of the two-dimensional panoramic projection image. The second segmentation method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron into one region to obtain multiple intermediate regions, and then segment at least one of the intermediate regions.

[0052] In one possible implementation, the aforementioned partitioning module is specifically used to partition the two-dimensional panoramic projection image according to a third partitioning method to obtain multiple intermediate regions of the two-dimensional panoramic projection image. The third partitioning method involves clustering all pixels of the two-dimensional panoramic projection image and partitioning pixels belonging to the same class into the same intermediate region. Furthermore, the module partitions the intermediate regions that do not meet the preset conditions according to a fourth partitioning method, thereby obtaining multiple regions. For any intermediate region that does not meet the preset conditions, the fourth partitioning method involves partitioning the intermediate region along the boundary lines of at least two surfaces of the polyhedron contained in the intermediate region.

[0053] In one possible implementation, the first, second, or third division method is preset in the electronic device.

[0054] In one possible implementation, the metadata of the two-dimensional panoramic projection image also includes partitioning indication information, which indicates whether the partitioning method of the two-dimensional panoramic projection image is a first partitioning method, a second partitioning method, or a third partitioning method.

[0055] In one possible implementation, the acquisition module is further used to acquire the correspondence between regions and metadata information units; the determination module is further used to determine the metadata information units corresponding to each of the multiple regions from the metadata based on the correspondence between regions and metadata information units, thereby determining the dynamic mapping information of each of the multiple regions.

[0056] In one possible implementation, the correspondence between the aforementioned region and metadata information unit is a correspondence between the index of the metadata information unit and the index of the region; or, the correspondence between the region and metadata information unit is a correspondence between the index of the metadata unit and one or more pixel coordinates in the region.

[0057] In one possible implementation, the dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a first preset range, the first preset range being the range centered on the pixel to be processed; or, the dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of the region to which the pixel corresponding to the viewpoint center of the current viewing angle range belongs in the two-dimensional panoramic projection image; or, the dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a second preset range, the second preset range being the range centered on the pixel corresponding to the viewpoint center of the current viewing angle range in the two-dimensional panoramic projection image; or, the dynamic mapping information of a pixel in the current region is the dynamic mapping information of the current region.

[0058] In one possible implementation, the dynamic mapping information in this application embodiment is used to map the dynamic range of pixels in a two-dimensional panoramic projection image from a first dynamic range to a second dynamic range; or, the dynamic mapping information is used to map the dynamic range of pixels in a two-dimensional panoramic projection image from a second dynamic range to a first dynamic range; wherein, the first dynamic range is greater than the second dynamic range.

[0059] In one possible implementation, the acquisition module is specifically used to receive metadata of a two-dimensional panoramic projection image from other electronic devices.

[0060] In one possible implementation, the conversion module is further used to perform subsequent processing on the two-dimensional panoramic projection image within the current viewing angle range. The subsequent processing includes: performing three-dimensional conversion on the two-dimensional panoramic projection image within the current viewing angle range to obtain a three-dimensional panoramic image within the current viewing angle range, which is used for display.

[0061] Thirdly, embodiments of this application provide an electronic device, including a memory and at least one processor connected to the memory, the memory being used to store instructions, which, after being read by the at least one processor, execute the method described in the first aspect and any of its possible implementations.

[0062] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method described in the first aspect and any of its possible implementations.

[0063] Fifthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a computer, execute the method described in the first aspect and any of its possible implementations.

[0064] Sixthly, embodiments of this application provide a chip including a memory and a processor. The memory is used to store computer instructions. The processor is used to retrieve and execute the computer instructions from the memory to perform the method described in the first aspect and any of its possible implementations.

[0065] It should be understood that the beneficial effects achieved by the second to sixth aspects of the technical solutions and the corresponding possible implementations of the embodiments of this application can be referred to the above-described technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0066] Figure 1 A schematic diagram of the video processing process provided in the embodiments of this application;

[0067] Figure 2A hardware schematic diagram of a mobile phone provided for an embodiment of this application;

[0068] Figure 3 One of the schematic diagrams of a dynamic range mapping method for panoramic video provided in this application embodiment;

[0069] Figure 4 This is one of the schematic diagrams illustrating the projection process of a three-dimensional panoramic image provided in an embodiment of this application;

[0070] Figure 5 A second schematic diagram illustrating the projection process of a three-dimensional panoramic image, provided as an embodiment of this application;

[0071] Figure 6 This is the third schematic diagram of a projection process for a three-dimensional panoramic image provided in an embodiment of this application;

[0072] Figure 7 A second schematic diagram illustrating a dynamic range mapping method for panoramic video provided in an embodiment of this application;

[0073] Figure 8 One of the schematic diagrams showing the segmentation result of a two-dimensional panoramic projection image provided in an embodiment of this application;

[0074] Figure 9 A third schematic diagram illustrating a dynamic range mapping method for panoramic video provided in this application embodiment;

[0075] Figure 10 A second schematic diagram illustrating the segmentation result of a two-dimensional panoramic projection image provided in an embodiment of this application;

[0076] Figure 11 A fourth schematic diagram illustrating a dynamic range mapping method for panoramic video provided in this application embodiment;

[0077] Figure 12 A schematic diagram (3) illustrating the segmentation result of a two-dimensional panoramic projection image provided in an embodiment of this application;

[0078] Figure 13 Fifth schematic diagram of a dynamic range mapping method for panoramic video provided in this application embodiment;

[0079] Figure 14 A schematic diagram (sixth) illustrating a dynamic range mapping method for panoramic video provided in this application embodiment;

[0080] Figure 15 One of the structural schematic diagrams of a dynamic range mapping device for panoramic video provided in an embodiment of this application;

[0081] Figure 16 This is a second schematic diagram of a dynamic range mapping device for panoramic video provided in an embodiment of this application. Detailed Implementation

[0082] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0083] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first division method" and "second division method," etc., are used to distinguish different division methods, not to describe a specific order of division methods.

[0084] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0085] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple regions means two or more regions.

[0086] First, some concepts involved in the dynamic range mapping method and apparatus for panoramic video provided in the embodiments of this application will be explained.

[0087] Dynamic range: In most fields, dynamic range is used to represent the ratio of the maximum to the minimum value of a variable. In the field of digital imaging, dynamic range represents the ratio of the maximum to the minimum grayscale value of an image pixel. Grayscale value can also be understood as brightness value, and the unit is nits (nits) or candela per square meter (cd / m²). 2 ), 1 nit = 1 cd / m 2 .

[0088] Typically, the dynamic range of the natural world is quite large, and the brightness of a night scene under the starry sky is approximately 0.001 cd / m². 2 The sun itself can reach a brightness of 10. 9 cd / m 2 Thus, the dynamic range of nature is 10. 9 / 0.001=10 12 In real-world natural environments, the brightness of the sun and the starry sky cannot be simultaneously obtained; therefore, the dynamic range of a real-world natural scene may not reach 10. 12The dynamic range is on the order of magnitude; for real-world natural environments, it is typically around 10. -3 Up to 10 6 Within the range.

[0089] Dynamic range mapping: After capturing video / images in a real natural environment, during the process of displaying the captured video / images on a display device (such as a TV or iPad), the dynamic range supported by the display device may differ from the original dynamic range of the video / image captured from the real natural environment. If the video / image is displayed directly according to its original dynamic range, the display device cannot display the video / image. Therefore, it is necessary to adapt the dynamic range of the captured video / image to the display device, that is, to adjust the dynamic range of the captured video / image to the dynamic range supported by the display device. Only in this way can the captured video / image be displayed on the display device. The process of adjusting the dynamic range of the video / image described above is called dynamic range mapping (also known as tone mapping).

[0090] Understandably, dynamic range is divided into high dynamic range (HDR) and low dynamic range (SDR), with low dynamic range also known as standard dynamic range. Generally, the brightness range is around 10... -3 ~10 6 Images with a brightness range of 0 to 255 are called high dynamic range images, while images with a brightness range of 0 to 255 are called low dynamic range images. Currently, in most color digital images, the pixels of each of the R, G, and B channels are represented by one byte (i.e., 8 bits). In other words, the brightness range of each channel's pixels is 0 to 255, and 0 to 255 is the standard dynamic range of the image.

[0091] In this embodiment, dynamic range mapping includes mapping from high to low dynamic range and mapping from low to high dynamic range. It should be noted that "high" and "low" here are relative concepts, different from the concepts of high and low dynamic range mentioned above. For example, mapping dynamic range from dynamic range 1 to dynamic range 2 means that the relationship between dynamic range 1 and dynamic range 2 may be that dynamic range 1 is greater than dynamic range 2, or dynamic range 1 is less than dynamic range 2. However, both dynamic range 1 and dynamic range 2 can be high dynamic ranges or both are low dynamic ranges.

[0092] For example, if the original dynamic range of the captured video / image is 1000 (HDR), while the display device supports a dynamic range of 100 (SDR), then the dynamic range mapping is a mapping from high to low. The dynamic range before mapping is HDR, and the dynamic range after mapping is SDR. As another example, if the original dynamic range of the captured video / image is 4000 (HDR), while the display device supports a dynamic range of 500 (HDR), then the dynamic range mapping is also a mapping from high to low, but both the dynamic range before and after mapping will be HDR.

[0093] For example, if the original dynamic range of the captured video / image is 100 SDR, while the display device supports a dynamic range of 2000 HDR, then the dynamic range mapping is a mapping from low to high. The dynamic range before mapping is SDR, and the dynamic range after mapping is HDR. As another example, if the original dynamic range of the captured video / image is 100 SDR, while the display device supports a dynamic range of 200 SDR, then the dynamic range mapping is also a mapping from low to high, but both the dynamic range before and after mapping are SDR.

[0094] Based on the examples above, Table 1 below shows several cases of dynamic range mapping.

[0095] Table 1

[0096]

[0097] When performing dynamic range mapping on video / images, several dynamic range mapping models can be used to dynamically map the pixels of the acquired video / image. These models include, but are not limited to, sigmoidal curves and Bézier curves. It should be understood that the aforementioned dynamic range mapping models are all designed for two-dimensional video / images.

[0098] Optionally, the methods for dynamic range mapping of videos / images include static dynamic range mapping and dynamic dynamic range mapping. Static dynamic range mapping refers to using the same dynamic range mapping curve to perform dynamic range mapping on the same video / image content (which may involve multiple scenes) or video / image content on the same hard drive. Dynamic dynamic range mapping refers to using different dynamic range mapping curves to perform dynamic range mapping on the video / image content based on different scenes or different frames.

[0099] The advantage of static dynamic range mapping (VMR) is its simplicity and the limited information it carries (e.g., information about the VMR curve). However, its disadvantage is that using a single VMR curve for video / image mapping can lead to flawed results and poor display quality. For example, while a VMR curve might work well for bright areas, it can cause pixel values ​​to be too small in dark areas, resulting in a loss of detail in the mapped image.

[0100] The advantage of dynamic range mapping is that it uses different dynamic range mapping curves suitable for different scenes or frames to perform dynamic range mapping, resulting in better mapping results, that is, better display effect of video / image after dynamic range mapping. The disadvantage is that the dynamic range mapping process is more complex and the information carried by the transmission of dynamic mapping information is more.

[0101] Panoramic video: A video containing spherical scene content with a horizontal 360-degree and vertical 180-degree range. Panoramic video includes multi-frame three-dimensional (3D) panoramic images. The three-dimensional panoramic images can be expressed using a three-dimensional sphere. Therefore, the three-dimensional panoramic images of panoramic video can also be called the three-dimensional spherical panoramic images of panoramic video.

[0102] Compared to ordinary 2D videos (covering 33 degrees horizontally and 18 degrees vertically), panoramic videos have a larger coverage area, thus providing users with an immersive visual experience. Users watch panoramic videos by wearing assistive devices (such as VR glasses). Specifically, with the user's head as the center point, the head can rotate 360 ​​degrees horizontally and tilt 180 degrees vertically. By adjusting the horizontal and vertical viewing angles, the user can freely choose to view a specific direction's window, which is the user's field of view. It can be understood that if a panoramic video is simulated as the surface of a sphere, and the user's head is at the center of the sphere, then by rotating and / or tilting the head, the user can continuously adjust the window to view the complete panoramic video.

[0103] In this embodiment, panoramic video has a larger brightness coverage area, resulting in a larger dynamic range compared to ordinary two-dimensional video. Specifically, two-dimensional video is captured within a relatively small area, where lighting conditions are similar, meaning the range of lighting variation is small, and therefore, the dynamic range is also small. Panoramic video, on the other hand, is captured within a larger area, where the range of lighting variation is typically larger. For example, panoramic video captured during the day may include areas of the sun in the sky, which are bright (i.e., high brightness), as well as areas inside buildings or in shadow, which are darker (i.e., low brightness). Thus, the dynamic range of panoramic video is much larger, posing significant challenges for its description and display.

[0104] Currently, the technology for dynamic range mapping of 2D video / images is mature. However, with the emergence of panoramic video, there is an urgent need for related technologies to achieve dynamic range mapping of panoramic video. Based on this, embodiments of this application provide a method and apparatus for dynamic range mapping of panoramic video. For any frame of 3D panoramic image in a panoramic video, the electronic device performs dynamic range mapping on the corresponding 2D panoramic projection image. Then, the dynamically range-mapped 2D panoramic projection image is converted into a 3D panoramic image, thereby achieving dynamic range mapping of the panoramic video.

[0105] The specific solution of this application embodiment is as follows: The electronic device divides the two-dimensional panoramic projection image corresponding to the three-dimensional panoramic image into multiple regions of the two-dimensional panoramic projection image. Each of these multiple regions satisfies a preset condition (which will be described in detail in the following embodiments). The electronic device also acquires metadata of the two-dimensional panoramic projection image, which includes metadata information units corresponding to each of the multiple regions of the two-dimensional panoramic projection image. The metadata information unit corresponding to one region includes the dynamic mapping information of that region. Then, the electronic device determines the dynamic mapping information of each pixel within the current viewing angle range based on the dynamic mapping information of each of the multiple regions. It then performs dynamic range mapping on each pixel within the current viewing angle range (which is the viewing angle range of the user currently viewing the panoramic video) to obtain the two-dimensional panoramic projection image within the current viewing angle range. This two-dimensional panoramic projection image within the current viewing angle range is used for display or subsequent processing. By using the dynamic range mapping method for panoramic video provided in this application embodiment, combined with the large dynamic range of panoramic video, the electronic device divides the two-dimensional panoramic projection image corresponding to the three-dimensional panoramic image into regions and then performs dynamic range mapping on different regions. This effectively achieves dynamic range mapping for panoramic video, thereby improving the display effect of panoramic video.

[0106] It should be noted that, in this embodiment of the application, the format of the images (which may be referred to as source data) contained in the panoramic video is not limited; the images may be in YUV format or RGB format. Furthermore, this embodiment of the application limits the bit width of the source data; the bit width of the source data may be 8 bits, 10 bits, or 12 bits.

[0107] Understandably, the processing of panoramic video typically includes production, encoding, decoding, and display. Production and encoding can be completed at the front end (the generation end), while decoding and display can be completed at the back end (the display end). (Reference) Figure 1 At the generation end, the panoramic video production includes performing simulated dynamic range mapping on each frame of the 3D panoramic image to determine dynamic range mapping information (such as information about the curves used for dynamic range mapping). Then, metadata containing the dynamic range mapping information is generated. The panoramic video encoding includes encoding each frame of the 3D panoramic image (e.g., prediction, transform, quantization, entropy coding). The image encoding process can be referenced from existing technologies; this application does not limit the encoding method for panoramic video. The generation end further writes the generated metadata containing dynamic range mapping information into a bitstream and transmits it to the display end. At the display end, the bitstream is parsed to obtain metadata and image decoding data (including syntax elements for image decoding and image data). Then, the image data is decoded according to the image decoding syntax elements (e.g., prediction, inverse quantization, inverse transform) to obtain a 3D panoramic image. The image decoding process can be referenced from existing technologies (the decoding method corresponds to the encoding method); this application does not limit the decoding method for panoramic video. Then, the 3D panoramic image is converted into a 2D panoramic projection image, and the metadata obtained from the bitstream is used to perform dynamic range mapping on the 2D panoramic projection image, thereby displaying the panoramic video after dynamic range mapping.

[0108] This application describes the dynamic range mapping process of panoramic video from the perspective of the display end. Exemplary examples of electronic devices serving as display ends include, but are not limited to, virtual reality (VR) devices (e.g., VR glasses), display devices (e.g., mobile phones), video application conversion devices (e.g., transcoders), and live streaming devices. It should be understood that in VR glasses and display devices, dynamic range mapping is implemented based on hardware chips, while in live streaming devices and video conversion devices, dynamic range mapping is implemented based on software program code.

[0109] For example, this application embodiment uses a mobile phone as the display terminal. Figure 2This is a schematic diagram of the hardware structure of a mobile phone 200 provided in an embodiment of this application. The mobile phone 200 includes a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0110] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the mobile phone 200. In other embodiments of this application, the mobile phone 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0111] Processor 210 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0112] The controller can serve as the central nervous system and command center of the mobile phone 200. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0113] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0114] The charging management module 240 receives charging input from the charger. While charging the battery 242, the charging management module 240 can also supply power to the electronic device through the power management module 241.

[0115] The power management module 241 connects the battery 242, the charging management module 240, and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240, providing power to the processor 210, internal memory 221, external memory, display screen 294, camera 293, and wireless communication module 260. The power management module 241 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 241 may also be located within the processor 210. In other embodiments, the power management module 241 and the charging management module 240 may be housed in the same device.

[0116] The wireless communication function of mobile phone 200 can be realized through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor and baseband processor.

[0117] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0118] The mobile communication module 250 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the mobile phone 200. The mobile communication module 250 can receive electromagnetic waves via the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to the modem processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 250 can be housed in the processor 210. In some embodiments, at least some functional modules of the mobile communication module 250 and at least some modules of the processor 210 can be housed in the same device.

[0119] The wireless communication module 260 can provide solutions for wireless communication applications on the mobile phone 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 260 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 210. The wireless communication module 260 can also receive signals to be transmitted from processor 210, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0120] In some embodiments, the antenna 1 of the mobile phone 200 is coupled to the mobile communication module 250, and the antenna 2 is coupled to the wireless communication module 260, so that the mobile phone 200 can communicate with the network and other devices through wireless communication technology.

[0121] The mobile phone 400 implements its display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0122] Display screen 294 is used to display images, videos, etc. Display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, mobile phone 200 may include one or N displays 294, where N is a positive integer greater than 1.

[0123] The mobile phone 200 can achieve shooting functions through ISP, camera 293, video codec, GPU, display 294 and application processor.

[0124] The ISP (Image Signal Processor) processes data fed back from the camera 293. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 293.

[0125] Camera 293 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, mobile phone 200 may include one or N cameras 293, where N is a positive integer greater than 1.

[0126] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals (such as audio signals).

[0127] Video codecs are used to compress or decompress digital video. Mobile phone 200 can support one or more video codecs. Thus, mobile phone 200 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0128] The external storage interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 200. The external memory card communicates with the processor 210 through the external storage interface 220 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0129] Internal memory 221 can be used to store computer executable program code, which includes instructions. Processor 210 executes various functional applications and data processing of mobile phone 200 by running the instructions stored in internal memory 221. Internal memory 221 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of mobile phone 200 (such as audio data, phonebook, etc.). Furthermore, internal memory 221 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0130] The mobile phone 200 can perform audio functions, such as music playback and recording, through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, and an application processor.

[0131] The audio module 270 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 270 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 270 may be located in the processor 210, or some functional modules of the audio module 270 may be located in the processor 210.

[0132] The speaker 270A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The mobile phone 200 can listen to music or make hands-free calls through the speaker 270A.

[0133] The receiver 270B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the mobile phone 200 answers a call or voice message, the receiver 270B can be brought close to the user's ear to listen to the voice.

[0134] Microphone 270C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 270C, inputting the sound signal into microphone 270C. Mobile phone 200 may have at least one microphone 270C. In some embodiments, mobile phone 200 may have two microphones 270C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, mobile phone 200 may also have three, four, or more microphones 270C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0135] The 270D headphone jack is used to connect wired headphones.

[0136] Keypad 290 includes a power button, volume buttons, etc. Mobile phone 200 can receive keypad input and generate key signal inputs related to user settings and function control.

[0137] Motor 291 can generate vibration alerts. Motor 291 can be used for incoming call vibration alerts or for touch vibration feedback.

[0138] Indicator 292 can be an indicator light, which can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0139] The SIM card interface 295 is used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 295 to make contact with or separate from the mobile phone 200.

[0140] It is understood that in the embodiments of this application, the mobile phone 200 described above can execute some or all of the steps in the embodiments of this application. These steps or operations are merely examples, and the mobile phone 200 can also execute other operations or variations thereof. Furthermore, the various steps can be executed in different orders as presented in the embodiments of this application, and it is not necessary to execute all the operations in the embodiments of this application. The embodiments of this application can be implemented individually or in any combination, and this application does not limit this.

[0141] Based on the above, embodiments of this application provide a dynamic range mapping method for panoramic video, such as... Figure 3 As shown, the method includes steps 301-305.

[0142] Step 301: The electronic device (i.e. the display end) projects any frame of the three-dimensional panoramic image from the panoramic video onto the surface of the polyhedron, and unfolds the polyhedron into a two-dimensional plane to obtain a two-dimensional panoramic projection image.

[0143] It should be understood that after the display end obtains the bitstream of the panoramic video (for example, by receiving the bitstream of the panoramic video from the generation end), it performs video decoding to obtain the panoramic video. Each frame of the panoramic video is a three-dimensional panoramic image, which can be called a three-dimensional spherical panoramic image of the panoramic video.

[0144] Optionally, the polyhedron used to project the three-dimensional panoramic image into a two-dimensional panoramic projection image includes at least one of the following: a regular tetrahedron, a regular hexahedron, a regular octahedron, a regular dodecahedron, or a regular icosahedron.

[0145] In this embodiment, the process of projecting a three-dimensional panoramic image into a two-dimensional panoramic projection image using a polyhedron includes: placing a sphere capable of representing a three-dimensional panoramic image (which is represented by a three-dimensional sphere) into the polyhedron, such that the sphere becomes the inscribed sphere of the polyhedron (it should be understood that the body center of the polyhedron coincides with the center of the sphere); then, connecting the center of the sphere to any point (pixel) on the surface of the sphere, and extending the line to intersect a surface of the polyhedron. At this time, the point on the sphere after being projected onto a surface of the polyhedron is the intersection point of the extension of the line connecting the center of the sphere and the point on the sphere with the surface of the polyhedron; or, connecting a point on the surface of the polyhedron to the center of the sphere, with the line intersecting the sphere at a point, the intersection point on the sphere and the point on the polyhedron are mutually projected. It should be understood that the pixel value of the point on the sphere is the pixel value of the intersection point on the surface of the polyhedron. Thus, following the projection process described above, all pixels on the three-dimensional sphere are projected to obtain the projected pixels of each surface of the polyhedron (interpolation is performed on the projected pixels of the two-dimensional plane as needed); finally, the polyhedron is unfolded into a two-dimensional plane according to certain rules to obtain a two-dimensional panoramic projection image.

[0146] For example, refer to Figure 4 Taking a regular hexahedron as an example, this regular hexahedron is denoted as ABCDEFGH, as follows: Figure 4As shown in (a), the sphere representing the 3D panoramic image is inscribed within the regular hexahedron ABCDEFGH. For point M' on plane ABCD (which is the base of the regular hexahedron ABCDEFGH, and plane A'B'C'D' is the projection plane of plane ABCD) on plane A'B'C'D', connect point M' to the center O of the sphere. The line connecting point M' intersects the sphere at point M. Thus, the pixel value of point M' is the pixel value of point M. Similarly, the pixel values ​​of all points within plane ABCD can be obtained. The pixels within plane ABCD constitute the surface image of plane ABCD. Following the projection method on plane ABCD, the pixels of the 3D panoramic image are projected onto other surfaces of the regular hexahedron to obtain other surface images. Figure 4 As shown in (b), after projecting the 3D panoramic image, the surface images of the upper surface EFGH, lower surface ABCD, front surface HGCD, rear surface EFBA, left surface EHDA, and right surface GFBC can be obtained. Furthermore, the surface images of the three-dimensional panoramic image can be... Figure 4 Expanding (b) in the middle to a two-dimensional plane, we get Figure 4 As shown in (c), the two-dimensional panoramic projection image is irregularly shaped. A face image of one face of a polyhedron can be called an image of a region within the two-dimensional panoramic projection image, or a sub-image of the two-dimensional panoramic projection image.

[0147] refer to Figure 5 ,exist Figure 5 The diagram shows the projection effects using a regular tetrahedron, regular hexahedron, regular octahedron, regular dodecahedron, and regular icosahedron. The 3D panoramic image is... Figure 5 The first row shows the projection of the regular polyhedron, which yields the corresponding two-dimensional panoramic projection image. Figure 5 The second row shows the projection result.

[0148] In this embodiment of the application, during subsequent processing, it is possible to directly... Figure 5 The second line illustrates processing irregular projected images, or the irregular image can be converted into a regular image for processing. In one implementation, the smallest rectangular region enclosing the irregular image can be processed (see reference). Figure 5 The third row, indicated by the dashed box, represents the rectangle with the smallest area enclosing the irregular image. Optionally, the areas within the rectangle other than the projection area can be filled, for example, with preset pixel values. In another implementation, the individual face images of the irregular projection image can be stitched together into a regular-shaped (e.g., rectangular) projection image, thus eliminating the need for pixel filling.

[0149] It should be noted that the embodiments of this application do not limit the above-mentioned polyhedrons to regular polyhedra, that is, the two-dimensional panoramic projection of the three-dimensional panoramic image can be composed of several polygons of different sizes.

[0150] Optionally, in the embodiments of this application, the electronic device may also use other methods to convert the three-dimensional panoramic image into a two-dimensional panoramic projection image. For example, the three-dimensional panoramic image (sphere) can be directly unfolded. Taking the three-dimensional panoramic image of the Earth as an example, the three-dimensional panoramic image of the Earth can be unfolded into a two-dimensional panoramic image. Specifically, in the region near the North and South Poles, the three-dimensional panoramic image is stretched to obtain the content of the North and South Poles region of the two-dimensional panoramic image.

[0151] Step 302: The electronic device divides the two-dimensional panoramic projection image corresponding to any frame of the three-dimensional panoramic image in the panoramic video into multiple regions of the two-dimensional panoramic projection image.

[0152] Each of the above regions satisfies a preset condition, which includes at least one of the following conditions 1 and 2.

[0153] Condition 1: Pixels that are adjacent in the 2D panoramic projection image but not adjacent in the 3D panoramic image are not located in the same region. Adjacent pixels in the 2D panoramic projection image include pixels in at least two adjacent projection planes of the 3D panoramic image, and non-adjacent pixels in the 3D panoramic image include pixels in the 3D panoramic image corresponding to at least two non-adjacent projection planes.

[0154] For example, taking a regular hexahedron as an example, the polyhedron used for projecting a 3D panoramic image, such as... Figure 6 As shown, the six faces of the regular hexahedron are numbered sequentially as 0, 1, 2, 3, 4, and 5, following the order of front face, back face, top face, bottom face, left face, and right face. Figure 6 In the 2D panoramic projection image following the projection of the 3D panoramic image, the arrows indicating the sides of each face (square) represent the corresponding sides in the polyhedron. Figure 6 In the diagram, the edges corresponding to arrows labeled with the same letter are the same edges in the polyhedron. The edge corresponding to the arrow labeled with the letter 'a' in surface image 0 (also called projection surface 0) and the edge corresponding to the arrow labeled with the letter 'a' in surface image 3 are the same edges in the regular hexahedron, that is, the intersection of the front surface 0 and the bottom surface 3.

[0155] exist Figure 6In a 3D panoramic image, for two adjacent projection surfaces, such as projection surface 0 and projection surface 1, in the 2D panoramic projection image, projection surface 0 and projection surface 1 are adjacent. Therefore, some pixels in projection surface 0 are adjacent to some pixels in projection surface 1. For example, the pixels on the side corresponding to the arrow marked with the letter 'a' in projection surface 0 are adjacent to the pixels on the side corresponding to the arrow marked with the letter 'g' in projection surface 1. However, the pixels in the 3D panoramic image corresponding to projection surface 0 and the pixels in the 3D panoramic image corresponding to projection surface 1 are non-adjacent pixels in the 3D panoramic image.

[0156] right Figure 6 When dividing the obtained 2D panoramic projection image, it is necessary to ensure that adjacent pixels in the 2D panoramic projection image but non-adjacent pixels in the 3D panoramic image do not reside in the same region. For example... Figure 6 In the 2D panoramic projection image, the pixels on the side corresponding to the arrow marked with the letter 'a' in projection plane 0 and the pixels on the side corresponding to the arrow marked with the letter 'g' in projection plane 1 are adjacent pixels. However, in the 3D panoramic image, these pixels are not adjacent. Therefore, to ensure that the pixels on the side corresponding to the arrow marked with the letter 'a' in projection plane 0 and the pixels on the side corresponding to the arrow marked with the letter 'g' in projection plane 1 are not assigned to the same region, the 2D panoramic projection image cannot be divided horizontally across the intersection of projection planes 0 and 1. Pixels on either side of this intersection line cannot be assigned to the same region. Similarly, pixels on either side of the intersection line between projection planes 4 and 3, and pixels on either side of the intersection line between projection planes 5 and 2, cannot be assigned to the same region.

[0157] Condition 2: In a 3D panoramic image, non-adjacent pixels are not located in the same region. Non-adjacent pixels in a 3D panoramic image include pixels in the 3D panoramic image corresponding to at least two non-adjacent projection surfaces.

[0158] Referring to the example in condition 1, Figure 6In the 2D panoramic projection image, pixels in the 3D panoramic image corresponding to projection plane 0 and pixels in the 3D panoramic image corresponding to projection plane 1 are non-adjacent pixels. When dividing the 2D panoramic projection image, it is necessary to ensure that pixels in projection plane 0 and pixels in projection plane 1 cannot be assigned to the same region. For example, pixels on the side corresponding to the arrow marked with the letter 'a' in projection plane 0 and pixels on the side corresponding to the arrow marked with the letter 'g' in projection plane 1 cannot be assigned to the same region. In other words, when dividing the 2D panoramic projection image horizontally, the division cannot cross the intersection line of projection plane 0 and projection plane 1; that is, pixels on both sides of the intersection line cannot be assigned to the same region. Similarly, pixels on both sides of the intersection line of projection plane 4 and projection plane 3 cannot be assigned to the same region, and pixels on both sides of the intersection line of projection plane 5 and projection plane 2 cannot be assigned to the same region.

[0159] In this embodiment of the application, during the process of dividing the two-dimensional panoramic projection image, when each divided region meets the above-mentioned preset conditions, it can be ensured that pixels with similar brightness (similar brightness means similar dynamic range) are divided into the same region, while pixels with large brightness differences (large brightness differences mean large dynamic range differences) are not divided into the same region. For example Figure 6 In the 2D panoramic projection image, the pixel corresponding to projection surface 2 might be a sky pixel, while the pixel corresponding to projection surface 3 might be a ground pixel. The brightness difference between sky and ground pixels is significant. Therefore, when dividing the 2D panoramic projection image, the pixels of projection surface 2 and projection surface 3 are not located in the same region. Since pixels within the same region have similar dynamic ranges, the same dynamic range mapping information can be used for pixels within the same region. Conversely, the dynamic ranges of pixels in different regions may differ significantly, requiring different dynamic range mapping information. Therefore, dividing the 2D panoramic projection image into multiple regions using this method, and then performing subsequent processing (i.e., dynamic range mapping) on ​​each region, allows for adaptive processing based on the characteristics of the pixels in each region, improving the processing efficiency.

[0160] Optionally, the electronic device can divide the two-dimensional panoramic projection image using different division methods. In the embodiments of this application, the electronic device can divide the two-dimensional panoramic projection image using any of the following steps 3021, 3022, or 3023-3024.

[0161] Combination Figure 3 ,like Figure 7 As shown, the embodiment of this application provides a method for dividing a two-dimensional panoramic projection image, which specifically includes step 3021 (that is, the above step 302 is implemented through step 3021).

[0162] Step 3021: The electronic device divides the two-dimensional panoramic projection image according to the first division method to obtain multiple regions of the two-dimensional panoramic projection image. The first division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron into one region.

[0163] Still with Figure 6 Taking the regular hexahedron shown as an example, refer to Figure 8 After a 3D panoramic projection image is projected onto a regular hexahedron, the resulting 2D panoramic projection image includes 6 projection planes. The first method of division is to group pixels belonging to the same surface of the polyhedron into a single region; that is, to treat each projection plane of the 2D panoramic projection image as a region. For example... Figure 8 As shown, the two-dimensional panoramic projection image can be divided into 6 regions according to the first division method, namely: Figure 8 Regions A, B, C, D, E, and F are included. (See reference.) Figure 8 It can be seen that each of the multiple regions obtained by dividing the two-dimensional panoramic projection image according to the first division method satisfies the above-mentioned preset conditions (including condition 1 or condition 2), that is, there are no adjacent pixels in the two-dimensional panoramic projection image that are not adjacent in the three-dimensional panoramic image located in the same region.

[0164] Combination Figure 3 ,like Figure 9 As shown, another method for dividing a two-dimensional panoramic projection image provided in this application embodiment specifically includes step 3022 (that is, the above step 302 is implemented through step 3022).

[0165] Step 3022: The electronic device divides the two-dimensional panoramic projection image according to the second division method to obtain multiple regions of the two-dimensional panoramic projection image. The second division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron (used for projection) into one region to obtain multiple intermediate regions, and then divide at least one of the intermediate regions.

[0166] Still with Figure 6 Taking the regular hexahedron shown as an example, refer to Figure 10After the 3D panoramic projection image is projected onto a regular hexahedron, the resulting 2D panoramic projection image includes 6 projection planes. The second division method involves dividing pixels belonging to the same surface of the polyhedron (used for projection) into a single region, resulting in multiple intermediate regions. Then, at least one of these intermediate regions is further divided, following the first division method described above. The resulting regions are then further divided, resulting in multiple intermediate regions. Finally, one or more of these intermediate regions are further divided to obtain the final division result. Figure 10 As shown, after dividing the image according to the first method, six intermediate regions are obtained: intermediate region a, intermediate region b, intermediate region c, intermediate region d, intermediate region e, and intermediate region f. Then, each of these six intermediate regions is horizontally bisected, resulting in 12 regions. Therefore, the second method can divide the 2D panoramic projection image into 12 regions, namely... Figure 10 Regions A, B, C, D, E, F, G, H, I, J, K, and L are included. (Reference) Figure 10 It can be seen that each of the multiple regions obtained by dividing the two-dimensional panoramic projection image according to the second division method satisfies the above-mentioned preset conditions (including condition 1 or condition 2), that is, there are no adjacent pixels in the two-dimensional panoramic projection image that are not adjacent in the three-dimensional panoramic image located in the same region.

[0167] Combination Figure 3 ,like Figure 11 As shown, another method for dividing a two-dimensional panoramic projection image provided in this application embodiment specifically includes steps 3023-3024 (that is, the above step 302 is implemented through steps 3023-3024).

[0168] Step 3023: The electronic device divides the two-dimensional panoramic projection image according to the third division method to obtain multiple intermediate regions of the two-dimensional panoramic projection image. The third division method is to perform clustering processing on all pixels of the two-dimensional panoramic projection image and divide pixels belonging to the same class into the same intermediate region.

[0169] In this embodiment of the application, the process of clustering all pixels of the two-dimensional panoramic projection image according to the third partitioning method includes:

[0170] First, determine the cluster centers, which are one pixel. Optionally, the 2D panoramic projection image can be divided into multiple regions according to a preset partitioning method. Then, determine the pixel value at the center of each region and use the pixel value at the center of each region as the cluster center. For example, assume the 2D panoramic projection image is as described above. Figure 6The image obtained by projection onto a regular hexahedron can be divided into 6 regions according to the first division method described above. Then, the pixel value of the center position of each region is determined to obtain 6 cluster centers. If the image is divided into 12 regions, it can be divided into 12 regions according to the second division method described above. Then, the pixel value of the center position of each region is determined to obtain 12 cluster centers.

[0171] Secondly, based on the determined cluster centers, a preset clustering algorithm is executed to calculate the distance between all pixels and each cluster center (this distance is calculated based on the coordinates of the pixels and the coordinates of the pixels corresponding to the cluster centers). This distance can be denoted as D. i Then, calculate the difference between the brightness value of all pixels and the brightness value of the pixel corresponding to the cluster center (or calculate the difference between the brightness value of all pixels and the average brightness value of the pixels in the category corresponding to each cluster center (i.e., the category after division according to the preset division method); or calculate the difference between the color value of all pixels and the color value of the pixel corresponding to the cluster center; or calculate the difference between the color value of all pixels and the average color value of the pixels in the category corresponding to each cluster center (i.e., the category after division according to the preset division method)). Record this difference as E. i Based on distance D i With E i The weighted values ​​are used to assign each pixel to the category corresponding to the corresponding cluster center, resulting in a clustering result. The category with the smallest weighted value is the category corresponding to the pixel. In this clustering result, the pixels of each category form a region.

[0172] It should be noted that the clustering method in step 3023 for dividing the two-dimensional panoramic projection image cannot guarantee whether the multiple intermediate regions obtained meet the above-mentioned preset conditions (including condition 1 or condition 2). Therefore, it is necessary to judge whether the multiple intermediate regions obtained by clustering meet the preset conditions. If the multiple intermediate regions all meet the above-mentioned preset conditions, then the multiple intermediate regions are taken as the final division result, that is, the multiple intermediate regions are taken as the multiple regions after the two-dimensional panoramic projection image is divided; if there are intermediate regions among the multiple intermediate regions that do not meet the above-mentioned preset conditions, the electronic device continues to execute the following step 3024.

[0173] Step 3024: The electronic device divides the intermediate regions that do not meet the preset conditions according to the fourth division method, thereby obtaining multiple regions. Specifically, for any intermediate region that does not meet the preset conditions, the fourth division method involves dividing the intermediate region along the boundary lines of at least two surfaces of the polyhedron contained within it.

[0174] Still using two-dimensional panoramic projection images as described above Figure 6 Taking the image obtained by projection onto a regular hexahedron as an example, refer to... Figure 12 Assuming the clustering process sets the number of categories to 7, after clustering all pixels of the 2D panoramic projection image, 7 intermediate regions are obtained: intermediate region A, intermediate region B, intermediate region C, intermediate region D, intermediate region E, intermediate region F, and intermediate region G. Since intermediate region E does not meet the above preset conditions (while the other intermediate regions do), it needs to be further subdivided, such as... Figure 12 As shown, the middle region E contains some pixels of two projection surfaces (projection surface 0 and projection surface 1). Therefore, the middle region E can be divided into region I and region H along the boundary line between projection surface 0 and projection surface 1 in the middle region E. In this way, the two-dimensional panoramic projection image is finally divided into 8 regions, namely region A, region B, region C, region I, region D, region H, region F and region G.

[0175] In this embodiment, a suitable division method can be selected from the above three division methods to divide the two-dimensional panoramic projection image according to actual needs. Alternatively, other division methods besides the above three division methods can be used to divide the two-dimensional panoramic projection image. The division method should ensure that each region after division meets the above preset conditions.

[0176] Optionally, in this embodiment of the application, the division method of the two-dimensional panoramic projection image can be preset in the electronic device (e.g., agreed upon by a protocol), such as set as a first division method, a second division method, or a third division method; or, the division method of the two-dimensional panoramic projection image can be transmitted through the bitstream (specifically in the metadata).

[0177] Based on the above description of the division method of the two-dimensional panoramic projection image in the embodiments, it can be seen that the first division method and the second division method are related, and the division result of the second division method is the result of further division based on the first division method.

[0178] In one implementation, if the candidate partitioning methods for a two-dimensional panoramic projection image include a first partitioning method and a second partitioning method, it can be determined whether further partitioning is needed after using the first partitioning method, i.e., whether to use the first partitioning method or the second partitioning method, based on the content of the two-dimensional panoramic projection image.

[0179] Specifically, whether it is necessary to perform a further partitioning of the partitioning result of the first partitioning method by performing the following process.

[0180] S1. Divide the two-dimensional panoramic projection image according to the first division method described above to obtain multiple intermediate regions.

[0181] For each of the multiple intermediate regions, perform the following steps S2-S5 to determine whether each intermediate region needs further subdivision.

[0182] S2. Calculate the histogram of the middle region.

[0183] For example, the histogram of the intermediate region is denoted as HISA. In this embodiment of the application, the histogram can be a grayscale histogram or a color histogram, and this embodiment of the application does not limit it.

[0184] S3. Divide the middle area into multiple regions and calculate the histogram for each region.

[0185] Optionally, an intermediate region can be divided into four regions, each of which occupies one-quarter of the area of ​​the intermediate region. The four regions are denoted as Region 0, Region 1, Region 2 and Region 3, and their histograms are denoted as HISB[0], HISB[1], HISB[2] and HISB[3], respectively.

[0186] S4. Calculate the sum of the differences between the histogram of the middle region and the histogram of the divided regions of the middle region.

[0187] First, calculate the histogram difference between the central region and each of the divided regions. This difference is the difference between the histogram of the central region multiplied by n and the corresponding element of the histogram of each region. Then, sum all the differences, where n is the number of regions into which the central region is divided. For example, if the central region is divided into four regions, then n=4. Finally, sum all the difference values.

[0188] Let the sum of the differences be denoted as DiffAB, then:

[0189] DiffAB=DiffAB[0]+DiffAB[1]+DiffAB[2]+DiffAB[3]

[0190] Wherein, DiffAB[0] represents the histogram difference between the middle region and region 0, DiffAB[1] represents the histogram difference between the middle region and region 1, DiffAB[2] represents the histogram difference between the middle region and region 2, and DiffAB[3] represents the histogram difference between the middle region and region 3.

[0191] S5. If DiffAB>T1, where T1 is a preset threshold, then the intermediate region needs to be further divided, that is, the second division method described above is used to divide the two-dimensional panoramic projection image; if DiffAB≤T1, then the intermediate region does not need to be further divided, that is, the first division method described above is used to divide the two-dimensional panoramic projection image.

[0192] DiffAB>T indicates that the brightness difference between different parts of the middle region is relatively large, and further subdivision is needed.

[0193] Since the first and second partitioning methods are related, in some cases, if the default partitioning method is the first partitioning method, a relevant flag can be used in the bitstream to indicate whether to further partition. When the flag indicates that no further partitioning is to be performed, the result of partitioning using the first partitioning method is the final result. When the flag indicates that further partitioning is to be performed, the result of partitioning using the first partitioning method is further partitioned.

[0194] In another implementation, if the candidate partitioning methods for the two-dimensional panoramic projection image include a first partitioning method, a second partitioning method, and a third partitioning method, the partitioning method to be used can be determined by performing the following process.

[0195] S11. Divide the two-dimensional panoramic projection image according to the first division method, the second division method, and the third division method.

[0196] S12. Calculate the histograms of each region under the first, second, and third division methods, as well as the histogram of the two-dimensional panoramic projection image.

[0197] S13. Calculate the sum of the differences between the histograms of the multiple regions obtained by each division method and the histograms of the two-dimensional panoramic projection image.

[0198] For example, suppose the 2D panoramic projection image is divided into M regions using the first partitioning method. The sum of the differences between the histograms of the multiple regions obtained using the first partitioning method and the histogram of the 2D panoramic projection image is denoted as DiffHISM. Similar to S4 above, firstly, the histogram difference between each region and the 2D panoramic projection image is calculated. This difference is the difference between the corresponding elements of the histogram obtained by multiplying the histogram of each region by M and the histogram of the 2D panoramic projection image, and then all differences are summed; finally, all difference values ​​are summed. That is:

[0199] DiffHISM=DiffHISM[0]+DiffHISM[1]+……+DiffHISM[M]

[0200] Wherein, DiffHISM[0] represents the histogram difference value between region 0 obtained by the first division method and the two-dimensional panoramic projection image, and so on, DiffHISM[M] represents the histogram difference value between region M obtained by the first division method and the two-dimensional panoramic projection image.

[0201] Assume that the 2D panoramic projection image is divided into N regions using the second partitioning method. The sum of the differences between the histograms of the multiple regions obtained using the second partitioning method and the histogram of the 2D panoramic projection image is denoted as DiffHISN. Similar to S4 above, firstly, the histogram difference between each region and the 2D panoramic projection image is calculated. This difference is the difference between the corresponding elements of the histogram obtained by multiplying the histogram of each region by N and the histogram of the 2D panoramic projection image. Then, all differences are summed. Finally, all difference values ​​are summed. That is:

[0202] DiffHISN=DiffHISN[0]+DiffHISN[1]+……+DiffHISN[N]

[0203] Wherein, DiffHISN[0] represents the histogram difference value between region 0 obtained by the second division method and the two-dimensional panoramic projection image, and so on, DiffHISN[N] represents the histogram difference value between region N obtained by the second division method and the two-dimensional panoramic projection image.

[0204] Assume that the 2D panoramic projection image is divided into P regions using the third partitioning method. The sum of the differences between the histograms of these regions and the histogram of the 2D panoramic projection image is denoted as DiffHISP. Similar to S4 above, firstly, the histogram difference between each region and the 2D panoramic projection image is calculated. This difference is the difference between the corresponding elements of the histogram obtained by multiplying the histogram of each region by P and the histogram of the 2D panoramic projection image. Then, all differences are summed. Finally, all differences are summed. That is:

[0205] DiffHISP=DiffHISP[0]+DiffHISP[1]+……+DiffHISP[P]

[0206] Wherein, DiffHISN[0] represents the histogram difference between region 0 obtained by the third division method and the two-dimensional panoramic projection image, and so on, DiffHISP[P] represents the histogram difference between region P obtained by the third division method and the two-dimensional panoramic projection image.

[0207] S14. The partitioning method with the largest sum of histogram differences is determined as the final partitioning method.

[0208] Compare the sums of the histogram differences DiffHISM, DiffHISN, and DiffHISP. The larger the sum of the histogram differences, the greater the difference in brightness between the different regions after division, indicating that the division is more reasonable.

[0209] Optionally, the metadata size for each of the three partitioning methods can be calculated. The metadata includes dynamic mapping information corresponding to each region of the two-dimensional panoramic projection image (detailed in step 303 below). Then, based on the weighted value between the sum of the difference values ​​of the histograms corresponding to the partitioning methods described in S13 and the metadata size corresponding to the partitioning methods, the partitioning method to be used is determined. For example, the metadata size corresponding to the first partitioning method is denoted as Size1, the metadata size corresponding to the second partitioning method is denoted as Size2, and the metadata size corresponding to the third partitioning method is denoted as Size3. The differences between DiffHISM and Size1 (denoted as err1), DiffHISN and Size2 (denoted as err2), and DiffHISP[P] and Size3 (denoted as err3) are calculated. Then, err1, err2, and err3 are compared, and the partitioning method with the largest difference is determined as the final partitioning method.

[0210] Step 303: The electronic device acquires the metadata of the two-dimensional panoramic projection image. The metadata includes metadata information units corresponding to multiple regions of the two-dimensional panoramic projection image. The metadata information unit corresponding to a region includes the dynamic mapping information of that region.

[0211] It is understood that, in this embodiment, the approach to dynamic range mapping of panoramic video involves projecting all three-dimensional panoramic images contained in the panoramic video onto a two-dimensional plane, performing dynamic range mapping on the resulting two-dimensional panoramic projected image, and then converting the dynamically range-mapped two-dimensional panoramic projected image into a three-dimensional panoramic image, thereby obtaining the dynamically range-mapped panoramic video. In other words, the metadata of the obtained two-dimensional panoramic projected image is used for dynamic range mapping of the three-dimensional panoramic image (i.e., for dynamic range mapping of the panoramic video). Therefore, the metadata of the two-dimensional panoramic projected image can be considered as the metadata of the three-dimensional panoramic image (or the metadata of the panoramic video). The metadata mentioned in the following embodiments refers to data used for dynamic range mapping.

[0212] Optionally, the method for an electronic device to obtain metadata of a two-dimensional panoramic projection image can be: the electronic device receives metadata of the two-dimensional panoramic projection image from another electronic device (the generating end). As described in the above embodiments, after the generating end generates the metadata, it encodes the metadata into a bitstream and sends it to the display end, thereby the display end receives and parses the bitstream to obtain the metadata.

[0213] In this embodiment of the application, the two-dimensional panoramic projection image is divided into multiple regions, that is, the two-dimensional panoramic projection image includes multiple regions, and each region corresponds to a metadata information unit.

[0214] In this embodiment, the metadata information unit includes dynamic mapping information, which contains dynamic mapping parameters. Based on these parameters, pixels can be dynamically mapped to a range. Optionally, the format of the dynamic mapping information can be histogram information or dynamic mapping curve information from the ST2094-40 standard, or dynamic mapping curve information from the ST2094-10 standard; this embodiment does not impose any limitations.

[0215] As can be seen from the description of the above embodiments, the division method of the two-dimensional panoramic projection image can be transmitted through the bitstream. Optionally, when the division method of the two-dimensional panoramic projection image is transmitted through the bitstream, the metadata of the two-dimensional panoramic projection image also includes division method indication information. The division method indication information is used to indicate that the division method of the two-dimensional panoramic projection image is a first division method, a second division method, or a third division method.

[0216] Step 304: The electronic device determines the dynamic mapping information of each pixel within the current viewing angle range based on the dynamic mapping information of multiple regions.

[0217] It should be understood that when a user views a panoramic video, the user (head) cannot see the entire panoramic video content in their current posture, but only a certain range of video content. This range can be understood as the user's field of view. Therefore, the user's field of view is related to their current viewing posture. In this embodiment, dynamic range mapping of the panoramic video involves dynamically mapping the content within the field of view of the user currently viewing the panoramic video.

[0218] In this embodiment, after the electronic device acquires dynamic mapping information for multiple regions, it can determine the dynamic mapping information of each pixel within the current viewing angle range based on the dynamic range mapping information of the multiple regions. Specifically, for a single pixel, several methods for determining the dynamic mapping information of that pixel are described in detail.

[0219] Method 1: The dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a first preset range, where the first preset range is the range centered on the pixel to be processed.

[0220] For example, taking a 3-pixel × 3-pixel range centered on the pixel to be processed as an example, assuming that the first preset range covers pixels in 3 regions (that is, the pixels in the first preset range belong to 3 regions), the dynamic mapping information of these 3 regions will be used as the dynamic mapping information of the pixel to be processed.

[0221] Method 2: The dynamic mapping information of the pixels to be processed within the current field of view is the dynamic mapping information of the region to which the pixel corresponding to the center of the viewpoint within the current field of view belongs in the two-dimensional panoramic projection image.

[0222] It should be understood that when a user watches a panoramic video, the user's current field of view corresponds to a viewpoint center. Assuming the viewpoint center is X, the pixel corresponding to the viewpoint center X in the two-dimensional projection image is X'. Then, the dynamic mapping information of the region to which X' belongs is used as the dynamic mapping information of the pixel to be processed.

[0223] Method 3: The dynamic mapping information of the pixels to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a second preset range. The second preset range is the range centered on the pixel corresponding to the viewpoint center of the current viewing angle range in the two-dimensional panoramic projection image.

[0224] Combining the above method two, taking the viewpoint center X of the current viewing angle range as an example, the pixel corresponding to the viewpoint center X in the two-dimensional projection image is X'. For example, the second preset range can be a 3-pixel × 3-pixel range centered on X'. Assuming that the second preset range covers the pixels of 2 regions (that is, the pixels of the first preset range belong to 2 regions), the dynamic mapping information of these 2 regions will be used as the dynamic mapping information of the pixel to be processed.

[0225] Method 4: The dynamic mapping information of a pixel in the current region is the dynamic mapping information of the current region.

[0226] The dynamic mapping information of the region to which the pixel to be processed belongs is used as the dynamic mapping information of the pixel to be processed.

[0227] Optionally, in practical applications, the generating end can select a suitable method from the above four methods to determine the dynamic mapping information of the pixel to be processed according to actual needs, and then transmit the selected method to the display end in the bitstream (metadata); or the generating end and the display end can pre-determine which method to use, which is not limited in this application embodiment.

[0228] Step 305: The electronic device performs dynamic range mapping on each pixel within the current viewing angle range based on the dynamic mapping information of each pixel within the current viewing angle range, thereby obtaining a two-dimensional panoramic projection image within the current viewing angle range. This two-dimensional panoramic projection image within the current viewing angle range is used for display or subsequent processing.

[0229] Optionally, in this embodiment, the dynamic mapping information is used to map the dynamic range of pixels in a two-dimensional panoramic projection image from a first dynamic range to a second dynamic range; or, the dynamic mapping information is used to map the dynamic range of pixels in a two-dimensional panoramic projection image from a second dynamic range to a first dynamic range; wherein the first dynamic range is greater than the second dynamic range. That is, the dynamic range mapping of pixels can be a mapping from high to low dynamic range, or a mapping from low to high dynamic range, for example, it can include the several cases exemplified in Table 1 of the above embodiments.

[0230] Taking the dynamic mapping information as an example of the dynamic mapping curve, in the embodiments of this application, the dynamic mapping curve may include, but is not limited to, sigmoidal curves, cubic spline curves, Bézier curves, etc.

[0231] For example, taking the sigmoidal curve as an example, this curve is ,in, This represents the pixels before dynamic range mapping. This represents the pixels mapped within the dynamic range.

[0232] The above The expression is:

[0233]

[0234] in, , , , , These are the parameters of the dynamic mapping curve, which are obtained from the metadata.

[0235] Optionally, and It is a normalized value. The normalization can be a normalization of a linear space or a normalization of a nonlinear space. Furthermore, the embodiments of this application do not limit the range of normalization. For example, the range of normalization can be 0-10000 nit or 0.001-100000 nit.

[0236] Optionally, after performing dynamic range mapping on each pixel within the current viewing angle range to obtain a two-dimensional panoramic projection image within the current viewing angle range, the two-dimensional panoramic projection image can be displayed. In one case, the pixels of the two-dimensional panoramic projection image can be configured according to the display capabilities (maximum display value and minimum display value) of the display device. Perform reverse normalization to normalize it to the range of the display device's display capabilities.

[0237] In this embodiment of the application, after determining the dynamic mapping information of each pixel within the current viewing angle range through the steps, some pixels may correspond to multiple dynamic mapping information (e.g., multiple dynamic mapping curves). Therefore, after performing dynamic range mapping on the pixels according to step 305, each dynamic mapping information can obtain a dynamic mapping result. This results in multiple dynamic mapping results for the pixel. In this case, the multiple dynamic mapping results can be processed to obtain the final dynamic mapping result. The processing method may include any of the following:

[0238] Processing method 1: Take the median of multiple dynamic mapping results as the final dynamic mapping result.

[0239] Processing method 2: Perform a weighted average of multiple dynamic mapping results to obtain the final dynamic mapping result.

[0240] Assuming the dynamic mapping information of the pixels to be processed includes n elements, then n dynamic mapping results can be obtained: Result_1, Result_2, ..., Result_n. The weighting coefficients corresponding to the n dynamic mapping results are W1, W2, ..., Wn. n Then the final mapping result R is:

[0241]

[0242] In this embodiment of the application, when processing multiple dynamic mapping results using processing method 2, the weighting coefficients corresponding to each dynamic mapping result can be preset weighting coefficients or weighting coefficients determined based on the current pixel to be processed.

[0243] The weighting coefficients determined based on the current pixel to be processed specifically include: First, calculating the distance between the pixel to be processed and the pixel at the center of the region corresponding to each dynamic mapping information; then, calculating the weighting coefficients based on each distance. For example, if the current pixel to be processed is Y, and the dynamic mapping information of Y includes dynamic mapping information corresponding to three regions, with the pixels at the center of these three regions being Y1', Y2', and Y3' respectively, then the distances D1 between Y and Y1', D2 between Y and Y2', and D3 between Y and Y3' are calculated, and the weighting coefficients are determined based on D1, D2, and D3. Optionally, in this embodiment, the smaller the distance between the pixel to be processed and the pixel at the center of the region corresponding to the dynamic mapping information, the larger the corresponding weighting coefficient.

[0244] For example, one method for determining weighting coefficients W1, W2, and W2 based on D1, D2, and D3 is as follows:

[0245]

[0246]

[0247]

[0248] Optionally, combined Figure 3 ,like Figure 13 As shown, before determining the dynamic mapping information of each pixel in the current region based on the dynamic mapping information of the multiple regions (i.e., before step 304), the dynamic range mapping method for panoramic video provided in this application embodiment further includes steps 306-307.

[0249] Step 306: The electronic device acquires the correspondence between the area and the metadata information unit.

[0250] Step 307: The electronic device determines the metadata information units corresponding to each of the multiple regions from the metadata based on the correspondence between the regions and the metadata information units, thereby determining the dynamic mapping information of each of the multiple regions.

[0251] Optionally, the correspondence between regions and metadata information units can be one-to-one or many-to-one. That is, one region of the two-dimensional panoramic projection image corresponds to one metadata information unit, or multiple regions of the two-dimensional panoramic projection image correspond to one metadata information unit. Optionally, when the brightness of multiple regions of the two-dimensional panoramic projection image differs significantly, a different dynamic mapping information is used for each region, resulting in a one-to-one correspondence between regions and metadata information units. When the brightness of several regions among multiple regions is relatively similar, these regions can use the same dynamic mapping information, resulting in a many-to-one correspondence between regions and metadata information units.

[0252] In this embodiment, the correspondence between a region and a metadata information unit is the correspondence between the index of the metadata information unit and the index of the region; or, the correspondence between a region and a metadata information unit is the correspondence between the index of the metadata unit and one or more pixel coordinates in the region.

[0253] As an example, suppose a two-dimensional panoramic projection image is divided into 6 regions (see reference). Figure 6 Table 2 below is an example of the correspondence between the index of metadata information unit and the index of region.

[0254] Table 2

[0255]

[0256] As shown in Table 2, the correspondence between the index of metadata information units and the index of regions is essentially a traversal order relationship. (Refer to...) Figure 6According to the order of the six regions in the two-dimensional panoramic projection image from left to right and from top to bottom, they are region 4, region 0, region 5, region 3, region 1, and region 2. Therefore, in the correspondence table between regions and metadata information units shown in Table 2, by traversing the six regions of the two-dimensional panoramic projection image from left to right and from top to bottom, the metadata information units corresponding to each of the six regions can be determined sequentially in the metadata.

[0257] In one implementation, the traversal order of multiple regions can be indicated in the bitstream. For example, the traversal order can be indicated in the metadata. The traversal order is not limited to the order from left to right or from top to bottom as mentioned above. The traversal order is specifically related to the location of each region in the two-dimensional panoramic projection image.

[0258] As another example, suppose a two-dimensional panoramic projection image is divided into 6 regions. Table 3 below shows the correspondence between the index of the metadata information unit and the coordinates of one or more pixels in the region.

[0259] Table 3

[0260]

[0261] Referring to Table 3, in one implementation, the pixel coordinates of the regions in Table 3 can also be carried in the metadata. Specifically, the pixel coordinates of each region are carried in the metadata information unit corresponding to each region. After the display end divides the two-dimensional panoramic projection image into multiple regions, for one region, it determines which coordinate in the correspondence shown in Table 3 belongs to that region, and then uses the metadata information unit corresponding to that coordinate as the metadata information unit for that region. For example, for region 1 among the six regions, the relationship between the pixel corresponding to each coordinate in Table 3 and region 1 is determined sequentially. After judgment, the pixel corresponding to coordinate (x4, y4) belongs to region 1. Therefore, the metadata information unit with index 4 corresponding to coordinate (x4, y4) is used as the metadata information unit for region 1.

[0262] Optionally, in the embodiments of this application, the traversal order of multiple regions described above can be a pre-set order, or the generator can determine a traversal order from the candidate traversal orders according to a relevant algorithm, and then carry an identifier in the metadata of the bitstream to transmit to the generator to indicate the traversal order.

[0263] For example, the traversal order to be used can be determined by the following method.

[0264] S111. Calculate the histogram for each region in multiple regions.

[0265] S112. For each of the candidate traversal orders, calculate the difference in histograms of adjacent regions.

[0266] Suppose there are 4 candidate traversal orders, namely traversal order Z1, traversal order Z2, traversal order Z3, and traversal order Z4.

[0267] Taking the example of dividing a 2D panoramic projection image into 6 regions, with the candidate traversal order from left to right and from bottom to top (denoted as Z1), assuming the 6 regions are region 4, region 0, region 5, region 3, region 1, and region 2, we calculate the histogram differences between region 4 and region 0 (denoted as DiffHIS_1), region 0 and region 5 (denoted as DiffHIS_2), region 5 and region 3 (denoted as DiffHIS_3), region 3 and region 1 (denoted as DiffHIS_4), and region 1 and region 2 (denoted as DiffHIS_5). Then, we sum these differences to obtain DiffHIS_Z1. Similarly, we can calculate DiffHIS_Z2, DiffHIS_Z3, and DiffHIS_Z4.

[0268] S113. Among the candidate traversal orders, the traversal method with the smallest difference in histogram is determined as the final traversal method.

[0269] For example, by comparing the values ​​of DiffHIS_Z1, DiffHIS_Z2, DiffHIS_Z3, and DiffHIS_Z4, the difference in the histograms and the traversal order with the smallest difference are determined as the final traversal method.

[0270] Optionally, in the embodiments of this application, a preset algorithm may also be used to determine whether the correspondence between the multiple regions of the above-mentioned two-dimensional panoramic image and the metadata information unit is a one-to-one relationship or a many-to-one relationship.

[0271] For example, the method described below can be used to determine whether the correspondence is one-to-one or many-to-one.

[0272] S1111, Clustering multiple regions.

[0273] The number of clusters can be set according to needs, such as half or a quarter of the number of regions. For example, if the total number of regions is 8, the 8 regions can be divided into 2 categories, which is equivalent to obtaining two larger regions, or the 8 regions can be divided into 4 categories, which is equivalent to obtaining 4 larger regions.

[0274] S1112. Calculate the sum of differences between the histograms of the regions corresponding to each category in the clustering results and the histograms of the two-dimensional panoramic projection image, and calculate the sum of differences between the histograms of the original multiple regions and the histograms of the two-dimensional panoramic image.

[0275] For example, the sum of the differences between the histograms of the regions corresponding to each category in the clustering results and the histograms of the two-dimensional panoramic projection image is denoted as DiffHIS_X, and the sum of the differences between the histograms of the original multiple regions and the histograms of the two-dimensional panoramic image is denoted as DiffHIS_org.

[0276] S1113. Based on the difference between the two histograms, determine whether the correspondence between multiple regions and metadata information units is a one-to-one relationship or a many-to-one relationship.

[0277] For example, when 8 regions are clustered into 2 categories, the sum of the differences between the histograms of the regions corresponding to each category in the clustering results and the histograms of the 2D panoramic projection image is DiffHIS_X2. The sum of the differences between the histograms of the original multiple regions and the histograms of the 2D panoramic image is denoted as DiffHIS_org. One method to determine this is: if DiffHIS_org×(1-T2)>DiffHIS_X2, then the correspondence between the region and the metadata information unit is many to one; if DiffHIS_org×(1-T2)≤DiffHIS_X2, then the correspondence between the region and the metadata information unit is one to one. T2 is a preset threshold.

[0278] Optionally, combined Figure 13 ,like Figure 14 As shown, after step 305, the dynamic range mapping method for panoramic video provided in this application embodiment further includes step 308.

[0279] Step 308: The electronic device performs subsequent processing on the two-dimensional panoramic projection image within the current viewing angle range. The subsequent processing includes: performing three-dimensional transformation on the two-dimensional panoramic projection image within the current viewing angle range to obtain a three-dimensional panoramic image within the current viewing angle range, which is used for display.

[0280] In this embodiment of the application, the two-dimensional panoramic projection image obtained in step 305 can be displayed. Furthermore, the two-dimensional panoramic projection image can also be used for subsequent processing, such as converting it into a three-dimensional panoramic image for display. Furthermore, after obtaining all the three-dimensional panoramic images of the panoramic video, the panoramic video can be played.

[0281] In summary, this application provides a dynamic range mapping method for panoramic video. An electronic device divides a two-dimensional panoramic projection image corresponding to a three-dimensional panoramic image into multiple regions of the two-dimensional panoramic projection image. Each of these regions satisfies a preset condition, which includes at least one of the following: pixels adjacent in the two-dimensional panoramic projection image but not adjacent in the three-dimensional panoramic image are not located in the same region; pixels not adjacent in the three-dimensional panoramic image are not located in the same region. The electronic device acquires metadata of the two-dimensional panoramic projection image, which includes metadata information units corresponding to each of the multiple regions of the two-dimensional panoramic projection image. The metadata information unit corresponding to a region includes the dynamic mapping information of that region. Then, based on the dynamic mapping information of each of the multiple regions, the electronic device determines the dynamic mapping information of each pixel within the current viewing angle range. Based on the dynamic mapping information of each pixel within the current viewing angle range (which is the viewing angle range of the user currently viewing the panoramic video), the electronic device performs dynamic range mapping on each pixel within the current viewing angle range, thereby obtaining a two-dimensional panoramic projection image within the current viewing angle range. This two-dimensional panoramic projection image within the current viewing angle range is used for display or subsequent processing. The dynamic range mapping method for panoramic video provided in this application, combined with the large dynamic range of panoramic video, allows the electronic device to divide the two-dimensional panoramic projection image corresponding to the three-dimensional panoramic image into regions, and then perform dynamic range mapping for different regions. This effectively realizes the dynamic range mapping of panoramic video, thereby improving the display effect of panoramic video.

[0282] Accordingly, this application provides an apparatus for dynamic range mapping of panoramic video. This apparatus can be applied to electronic devices. In this application embodiment, the apparatus can be divided into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0283] When dividing each function into modules according to its corresponding function. Figure 15 This diagram illustrates a possible structure of the dynamic range mapping device for panoramic video involved in the above embodiments. For example... Figure 15As shown, the device includes a segmentation module 1501, an acquisition module 1502, a determination module 1503, and a processing module 1504. The segmentation module 1501 is used to segment the two-dimensional panoramic projection image to obtain multiple regions of the two-dimensional panoramic projection image. The two-dimensional panoramic projection image is the two-dimensional panoramic projection image corresponding to any frame of the three-dimensional panoramic image in the panoramic video. Each of the multiple regions satisfies a preset condition, which includes at least one of the following conditions: pixels that are adjacent in the two-dimensional panoramic projection image but not adjacent in the three-dimensional panoramic image are not located in the same region; pixels that are not adjacent in the three-dimensional panoramic image are not located in the same region. Adjacent pixels in the two-dimensional panoramic projection image include pixels in at least two adjacent projection surfaces of the three-dimensional panoramic image; non-adjacent pixels in the three-dimensional panoramic image include pixels in the three-dimensional panoramic image corresponding to at least two non-adjacent projection surfaces, for example, by performing step 302 in the above method embodiment. The acquisition module 1502 is used to acquire metadata of the two-dimensional panoramic projection image. This metadata includes metadata information units corresponding to multiple regions of the two-dimensional panoramic projection image. Each metadata information unit corresponding to a region includes dynamic mapping information for that region. For example, steps 302 and 303 in the above method embodiment are executed. The determination module 1503 is used to determine the dynamic mapping information of each pixel within the current viewing angle range based on the dynamic mapping information of each of the multiple regions. For example, step 304 in the above method embodiment is executed. The processing module 1504 is used to perform dynamic range mapping on each pixel within the current viewing angle range based on the dynamic mapping information of each pixel within the current viewing angle range, thereby obtaining a two-dimensional panoramic projection image within the current viewing angle range. This two-dimensional panoramic projection image within the current viewing angle range is used for display or subsequent processing. For example, step 305 in the above method embodiment is executed.

[0284] Optionally, the dynamic range mapping device for panoramic video provided in this application embodiment further includes a conversion module 1505; the conversion module 1505 is used to project a three-dimensional panoramic image onto the surface of a polyhedron and unfold the polyhedron into a two-dimensional plane to obtain a two-dimensional panoramic projection image. The polyhedron includes at least one of the following: a regular tetrahedron, a regular hexahedron, a regular octahedron, a regular dodecahedron, or a regular icosahedron, for example, by performing step 301 in the above method embodiment.

[0285] Optionally, the division module 1501 is specifically used to divide the two-dimensional panoramic projection image according to the first division method to obtain multiple regions of the two-dimensional panoramic projection image. The first division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron into one region, for example, by performing step 3021 in the above method embodiment.

[0286] Optionally, the above-mentioned division module 1501 is specifically used to divide the two-dimensional panoramic projection image according to the second division method to obtain multiple regions of the two-dimensional panoramic projection image. The second division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron into one region to obtain multiple intermediate regions, and then divide at least one of the multiple intermediate regions, for example, by executing step 3022 in the above method embodiment.

[0287] Optionally, the division module 1501 is specifically used to divide the two-dimensional panoramic projection image according to a third division method to obtain multiple intermediate regions of the two-dimensional panoramic projection image. The third division method is to perform clustering processing on all pixels of the two-dimensional panoramic projection image and divide pixels belonging to the same class into the same intermediate region. And according to a fourth division method, the intermediate regions that do not meet the preset conditions are divided to obtain multiple regions. For any intermediate region that does not meet the preset conditions, the fourth division method is to divide the intermediate region along the boundary lines of at least two surfaces of the polyhedron contained in the intermediate region, for example, by executing steps 3023 to 3024 in the above method embodiment.

[0288] Optionally, the acquisition module 1502 is further configured to acquire the correspondence between regions and metadata information units, for example, by executing step 306 in the above method embodiment; the determination module 1503 is further configured to determine the metadata information units corresponding to each of the multiple regions from the metadata according to the correspondence between regions and metadata information units, thereby determining the dynamic mapping information of each of the multiple regions, for example, by executing step 307 in the above method embodiment.

[0289] Optionally, the acquisition module 1502 is specifically used to receive metadata of a two-dimensional panoramic projection image from other electronic devices.

[0290] Optionally, the conversion module 1505 is further configured to perform subsequent processing on the two-dimensional panoramic projection image within the current viewing angle range. The subsequent processing includes: performing three-dimensional conversion on the two-dimensional panoramic projection image within the current viewing angle range to obtain a three-dimensional panoramic image within the current viewing angle range. The three-dimensional panoramic image is used for display, for example, by performing step 308 in the above method embodiment.

[0291] Each module of the above-mentioned dynamic range mapping device for panoramic video can also be used to perform other actions in the above method embodiments. All relevant content of each step involved in the above method embodiments can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0292] When using integrated units, Figure 16 A schematic diagram of another possible structure of the dynamic range mapping device for panoramic video involved in the above embodiments is shown. For example... Figure 16 As shown, the dynamic range mapping device for panoramic video provided in this application embodiment may include a processing module 1601 and a communication module 1602. The processing module 1601 can be used to control and manage the operation of the device. For example, the processing module 1601 can be used to support the device in executing steps 301 to 305, 306 to 307, and 308 in the above method embodiments, and / or other processes used in the technology described herein. The communication module 1602 can be used to support communication between the device and other network entities, such as communication with another electronic device (generating end). Optionally, as... Figure 16 As shown, the dynamic range mapping device for panoramic video may also include a storage module 1603 for storing the device's program code and data, such as storing metadata.

[0293] The processing module 1601 may be a processor or a controller (e.g., as described above). Figure 2 The processor 210 shown can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication module 1602 can be a transceiver, transceiver circuit, or communication interface, etc. (e.g., the one described above). Figure 2 The mobile communication module 250 or wireless communication module 260 shown. Storage module 1603 can be a memory (e.g., the one described above). Figure 2 The internal memory 221 shown.

[0294] When the processing module 1601 is a processor, the communication module 1602 is a transceiver, and the storage module 1603 is a memory, the processor, transceiver, and memory can be connected via a bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc.

[0295] For more details on how the modules included in the above-mentioned panoramic video dynamic range mapping device implement the above functions, please refer to the descriptions in the previous method embodiments, which will not be repeated here.

[0296] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0297] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0298] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0299] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0300] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0301] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0302] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0303] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A dynamic range mapping method for panoramic video, characterized in that, Applied to electronic devices, the method includes: The two-dimensional panoramic projection image is divided into multiple regions. The two-dimensional panoramic projection image is the two-dimensional panoramic projection image corresponding to any frame of the three-dimensional panoramic image in the panoramic video. Each of the multiple regions satisfies a preset condition, which includes at least one of the following conditions: adjacent pixels in the two-dimensional panoramic projection image but not adjacent pixels in the three-dimensional panoramic image are not located in the same region; and non-adjacent pixels in the three-dimensional panoramic image are not located in the same region. Adjacent pixels in the two-dimensional panoramic projection image include pixels in at least two adjacent projection planes of the three-dimensional panoramic image, and non-adjacent pixels in the three-dimensional panoramic image include pixels in the three-dimensional panoramic image corresponding to at least two non-adjacent projection planes. The metadata of the two-dimensional panoramic projection image is obtained. The metadata includes metadata information units corresponding to multiple regions of the two-dimensional panoramic projection image. The metadata information unit corresponding to a region includes the dynamic mapping information of the region. The dynamic mapping information includes dynamic mapping parameters for performing dynamic range mapping on the two-dimensional panoramic projection image. The dynamic range represents the ratio of the maximum gray value to the minimum gray value of a pixel in the two-dimensional panoramic projection image. Based on the dynamic mapping information of each of the multiple regions, determine the dynamic mapping information of each pixel within the current viewpoint range; Dynamic range mapping is performed on each pixel within the current viewing angle range based on the dynamic mapping information of each pixel within the current viewing angle range, so as to adjust the dynamic range of the two-dimensional panoramic projection image to the dynamic range supported by the electronic device, thereby obtaining a two-dimensional panoramic projection image within the current viewing angle range, which is used for display or subsequent processing.

2. The method according to claim 1, characterized in that, The method further includes: The three-dimensional panoramic image is projected onto the surface of a polyhedron, and the polyhedron is unfolded into a two-dimensional plane to obtain the two-dimensional panoramic projection image. The polyhedron includes at least one of the following: a regular tetrahedron, a regular hexahedron, a regular octahedron, a regular dodecahedron, or a regular icosahedron.

3. The method according to claim 2, characterized in that, The metadata of the two-dimensional panoramic projection image also includes division method indication information, which indicates whether the division method of the two-dimensional panoramic projection image is a first division method, a second division method, or a third division method.

4. The method according to claim 3, characterized in that, The step of dividing the two-dimensional panoramic projection image into multiple regions includes: The two-dimensional panoramic projection image is divided according to the first division method to obtain multiple regions of the two-dimensional panoramic projection image. The first division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron into one region.

5. The method according to claim 3, characterized in that, The step of dividing the two-dimensional panoramic projection image into multiple regions includes: The two-dimensional panoramic projection image is divided according to the second division method to obtain multiple regions of the two-dimensional panoramic projection image. The second division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron into one region to obtain multiple intermediate regions, and then divide at least one of the intermediate regions.

6. The method according to claim 3, characterized in that, The step of dividing the two-dimensional panoramic projection image into multiple regions includes: The two-dimensional panoramic projection image is divided according to the third division method to obtain multiple intermediate regions of the two-dimensional panoramic projection image. The third division method is to perform clustering processing on all pixels of the two-dimensional panoramic projection image and divide pixels belonging to the same class into the same intermediate region. The intermediate regions that do not meet the preset conditions are divided according to the fourth division method to obtain the multiple regions; for any intermediate region that does not meet the preset conditions, the fourth division method is to divide the intermediate region along the boundary lines of at least two surfaces of the polyhedron contained in the intermediate region.

7. The method according to claim 3, characterized in that, The first division method, the second division method, or the third division method are preset in the electronic device.

8. The method according to any one of claims 1 to 7, characterized in that, Before determining the dynamic mapping information of each pixel in the current region based on the dynamic mapping information of the plurality of regions, the method further includes: Obtain the correspondence between regions and metadata information units; Based on the correspondence between the regions and metadata information units, the metadata information units corresponding to each of the multiple regions are determined from the metadata, thereby determining the dynamic mapping information of each of the multiple regions.

9. The method according to claim 8, characterized in that, The correspondence between the regions and metadata information units is the correspondence between the index of the metadata information unit and the index of the region; or, The correspondence between the region and the metadata information unit is the correspondence between the index of the metadata unit and one or more pixel coordinates in the region.

10. The method according to any one of claims 1 to 7, characterized in that, The dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a first preset range, where the first preset range is the range centered on the pixel to be processed; or... The dynamic mapping information of the pixels to be processed within the current viewing angle range is the dynamic mapping information of the region to which the pixel corresponding to the viewpoint center of the current viewing angle range belongs in the two-dimensional panoramic projection image; or, The dynamic mapping information of the pixels to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a second preset range, where the second preset range is the range centered on the pixel corresponding to the viewpoint center of the current viewing angle range in the two-dimensional panoramic projection image; or... The dynamic mapping information of a pixel in the current region is the dynamic mapping information of the current region.

11. The method according to any one of claims 1 to 7, characterized in that, The dynamic mapping information is used to map the dynamic range of pixels in the two-dimensional panoramic projection image from a first dynamic range to a second dynamic range; or, The dynamic mapping information is used to map the dynamic range of pixels in the two-dimensional panoramic projection image from a second dynamic range to a first dynamic range; wherein the first dynamic range is greater than the second dynamic range.

12. The method according to any one of claims 1 to 7, characterized in that, The metadata for obtaining the two-dimensional panoramic projection image includes: Receive metadata of the two-dimensional panoramic projection image from other electronic devices.

13. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The two-dimensional panoramic projection image within the current viewing angle range is subjected to subsequent processing, which includes: performing a three-dimensional transformation on the two-dimensional panoramic projection image within the current viewing angle range to obtain a three-dimensional panoramic image within the current viewing angle range, which is used for display.

14. A dynamic range mapping device for panoramic video, characterized in that, It includes a partitioning module, an acquisition module, a determination module, and a processing module; The segmentation module is used to divide the two-dimensional panoramic projection image into multiple regions of the two-dimensional panoramic projection image. The two-dimensional panoramic projection image is the two-dimensional panoramic projection image corresponding to any frame of the three-dimensional panoramic image in the panoramic video. Each of the multiple regions satisfies a preset condition, which includes at least one of the following conditions: adjacent pixels in the two-dimensional panoramic projection image but not adjacent pixels in the three-dimensional panoramic image are not located in the same region; and non-adjacent pixels in the three-dimensional panoramic image are not located in the same region. Adjacent pixels in the two-dimensional panoramic projection image include pixels in at least two adjacent projection surfaces of the three-dimensional panoramic image, and non-adjacent pixels in the three-dimensional panoramic image include pixels in the three-dimensional panoramic image corresponding to at least two non-adjacent projection surfaces. The acquisition module is used to acquire metadata of the two-dimensional panoramic projection image. The metadata includes metadata information units corresponding to multiple regions of the two-dimensional panoramic projection image. The metadata information unit corresponding to a region includes the dynamic mapping information of the region. The dynamic mapping information includes dynamic mapping parameters for dynamically mapping the two-dimensional panoramic projection image; the dynamic range represents the ratio of the maximum gray value to the minimum gray value of a pixel in the two-dimensional panoramic projection image. The determining module is used to determine the dynamic mapping information of each pixel within the current viewing angle range based on the dynamic mapping information of each of the multiple regions. The processing module is configured to perform dynamic range mapping on each pixel within the current viewing angle range based on the dynamic mapping information of each pixel within the current viewing angle range, so as to adjust the dynamic range of the two-dimensional panoramic projection image to the dynamic range supported by the electronic device, thereby obtaining a two-dimensional panoramic projection image within the current viewing angle range, which is used for display or subsequent processing.

15. The apparatus according to claim 14, characterized in that, It also includes a conversion module; The conversion module is used to project the three-dimensional panoramic image onto the surface of a polyhedron and unfold the polyhedron into a two-dimensional plane to obtain the two-dimensional panoramic projection image. The polyhedron includes at least one of the following: a regular tetrahedron, a regular hexahedron, a regular octahedron, a regular dodecahedron, or a regular icosahedron.

16. The apparatus according to claim 15, characterized in that, The metadata of the two-dimensional panoramic projection image also includes division method indication information, which indicates whether the division method of the two-dimensional panoramic projection image is a first division method, a second division method, or a third division method.

17. The apparatus according to claim 16, characterized in that, The division module is specifically used to divide the two-dimensional panoramic projection image according to the first division method to obtain multiple regions of the two-dimensional panoramic projection image. The first division method is to divide the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron into one region.

18. The apparatus according to claim 16, characterized in that, The segmentation module is specifically used to segment the two-dimensional panoramic projection image according to the second segmentation method to obtain multiple regions of the two-dimensional panoramic projection image. The second segmentation method is to segment the pixels of the two-dimensional panoramic projection image that belong to the same surface of the polyhedron into one region to obtain multiple intermediate regions, and then segment at least one of the multiple intermediate regions.

19. The apparatus according to claim 16, characterized in that, The segmentation module is specifically used to segment the two-dimensional panoramic projection image according to the third segmentation method to obtain multiple intermediate regions of the two-dimensional panoramic projection image. The third segmentation method is to perform clustering processing on all pixels of the two-dimensional panoramic projection image and divide pixels belonging to the same class into the same intermediate region. Furthermore, the intermediate regions that do not meet the preset conditions are divided according to the fourth division method to obtain the multiple regions; For any intermediate region that does not meet the preset conditions, the fourth division method is to divide the intermediate region along the boundary lines of at least two surfaces of the polyhedron contained in the intermediate region.

20. The apparatus according to claim 16, characterized in that, The first division method, the second division method, or the third division method are preset in the electronic device.

21. The apparatus according to any one of claims 14 to 20, characterized in that, The acquisition module is also used to acquire the correspondence between regions and metadata information units; The determining module is further configured to determine the metadata information unit corresponding to each of the plurality of regions from the metadata according to the correspondence between the regions and the metadata information units, thereby determining the dynamic mapping information of each of the plurality of regions.

22. The apparatus according to claim 21, characterized in that, The correspondence between the regions and metadata information units is the correspondence between the index of the metadata information unit and the index of the region; or, The correspondence between the region and the metadata information unit is the correspondence between the index of the metadata unit and one or more pixel coordinates in the region.

23. The apparatus according to any one of claims 14 to 20, characterized in that, The dynamic mapping information of the pixel to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a first preset range, where the first preset range is the range centered on the pixel to be processed; or... The dynamic mapping information of the pixels to be processed within the current viewing angle range is the dynamic mapping information of the region to which the pixel corresponding to the viewpoint center of the current viewing angle range belongs in the two-dimensional panoramic projection image; or, The dynamic mapping information of the pixels to be processed within the current viewing angle range is the dynamic mapping information of multiple regions within a second preset range, where the second preset range is the range centered on the pixel corresponding to the viewpoint center of the current viewing angle range in the two-dimensional panoramic projection image; or... The dynamic mapping information of a pixel in the current region is the dynamic mapping information of the current region.

24. The apparatus according to any one of claims 14 to 20, characterized in that, The dynamic mapping information is used to map the dynamic range of pixels in the two-dimensional panoramic projection image from a first dynamic range to a second dynamic range; or, The dynamic mapping information is used to map the dynamic range of pixels in the two-dimensional panoramic projection image from a second dynamic range to a first dynamic range; wherein the first dynamic range is greater than the second dynamic range.

25. The apparatus according to any one of claims 14 to 20, characterized in that, The acquisition module is specifically used to receive metadata of the two-dimensional panoramic projection image from other electronic devices.

26. The apparatus according to any one of claims 15 to 20, characterized in that, The conversion module is further configured to perform subsequent processing on the two-dimensional panoramic projection image within the current viewing angle range. The subsequent processing includes: performing three-dimensional conversion on the two-dimensional panoramic projection image within the current viewing angle range to obtain a three-dimensional panoramic image within the current viewing angle range, which is used for display.

27. An electronic device, characterized in that, The method includes a memory and at least one processor connected to the memory, the memory being used to store instructions that, when read by the at least one processor, execute the method as described in any one of claims 1 to 13.

28. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the method according to any one of claims 1 to 13.

Images