Method and apparatus for encoding a spatial image

Abstract

The application discloses a kind of suitable for the encoding method and device of space image. The specific embodiment of method includes: according to the latitude and longitude of multiple sampling points corresponding under different space angles, generate the space picture matrix including multiple space pictures of multiple sampling points collection;According to the degree of change between adjacent space pictures in space picture matrix, determine the target size of the sub-matrix corresponding to different regions in space picture matrix;With the sub-matrix of different regions in space picture matrix corresponding to the space picture matrix, obtain multiple picture groups;According to the key frame and prediction frame respectively included in multiple picture groups, encode the space picture in space picture matrix, generate encoding file.The application is based on the decoupling of space picture in time, using the distribution characteristics of space picture in space, determines the picture group coding structure, improves the compression efficiency of encoding process.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to encoding technology, and more particularly to an encoding method, apparatus, computer-readable medium, and electronic device suitable for spatial images. Background Technology

[0002] Current mainstream video encoding and decoding technologies are all time-series based, meaning that the data of consecutive frames in the bitstream is temporally continuous. This is a very strong prior condition, implying that information from consecutive frames can be safely utilized during the encoding process. Existing video coding technologies are suitable for temporal images, but not for spatial images. Summary of the Invention

[0003] This application provides an encoding method, apparatus, computer-readable medium, and electronic device suitable for spatial images.

[0004] In a first aspect, embodiments of this application provide an encoding method applicable to spatial images, comprising: generating a spatial image matrix including multiple spatial images collected from multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles; determining the target size of sub-matrices corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix; dividing the spatial image matrix into multiple image groups by sub-matrices corresponding to different regions in the spatial image matrix; and encoding the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0005] In some examples, the above method of determining the target size of the submatrix corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix includes: determining the target size of the submatrix corresponding to different regions in the spatial image matrix based on the negative correlation between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the submatrix.

[0006] In some examples, adjacent sampling points among the aforementioned multiple sampling points are arranged at preset degree intervals in the longitude and latitude directions. Based on the negative correlation between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the sub-matrix, the target size of the sub-matrix corresponding to different regions in the spatial image matrix is ​​determined. This includes: in response to determining that the degree of change between adjacent spatial images in the latitude direction is greater than the degree of change between adjacent spatial images in the longitude direction, the length of the sub-matrix is ​​determined to be greater than its width, wherein the length direction of the sub-matrix corresponds to the longitude direction of the sampling point, and the width direction of the sub-matrix corresponds to the latitude direction of the sampling point; in response to determining that the degree of change between adjacent spatial images in the longitude direction is negatively correlated with the latitude value, the length of the sub-matrix is ​​determined to be positively correlated with the latitude value; based on the fact that the length of the sub-matrix is ​​greater than its width and the positive correlation between the length and latitude value of the sub-matrix, the target size of the sub-matrix in different regions of the spatial image matrix is ​​determined.

[0007] In some examples, the above-mentioned generation of a spatial image matrix, which includes multiple spatial images collected from multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles, includes: arranging multiple spatial images with the longitude of the sampling points corresponding to the spatial images as the horizontal axis and the latitude of the sampling points corresponding to the spatial images as the vertical axis, to generate a spatial image matrix.

[0008] In some examples, the above-mentioned encoding of spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file includes: arranging the image groups corresponding to the sampling points of the image groups in ascending order of longitude, generating multiple image group subsequences; arranging the multiple image group subsequences in ascending order of latitude of the sampling points corresponding to the image groups, determining the image group identifiers corresponding to each of the multiple image groups, and generating an image group sequence; arranging the spatial images in each image group in the image group sequence in the order of keyframes first and prediction frames second, determining the image identifiers of the spatial images in each of the multiple image groups, and generating a spatial image sequence; and encoding the spatial image sequence based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0009] In some examples, the above-described process of arranging spatial images in each image group within the image group sequence in the order of keyframes first, then prediction frames, determining the image identifiers of the spatial images in each of the multiple image groups, and generating a spatial image sequence includes: determining the spatial image at the center position of the sub-matrix corresponding to each of the multiple image groups as the initial keyframe; for each image group, based on the initial keyframe of the image group, determining the keyframe of the image group from the direction where the sampling points corresponding to the spatial images in the image group are sparser, and determining the prediction frames adjacent to the keyframes; arranging spatial images in each image group within the image group sequence in the order of keyframes first, then prediction frames, determining the image identifiers of the spatial images in each of the multiple image groups, and generating a spatial image sequence.

[0010] In some examples, the above-mentioned encoding of spatial image sequences based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file includes: for each image group in the multiple image groups, encoding the spatial image sequence by using a reference method in which the prediction frames in the image group uniquely reference the keyframes in the image group to generate an encoded file.

[0011] Secondly, embodiments of this application provide an encoding apparatus suitable for spatial images, comprising: a first generation unit configured to generate a spatial image matrix including multiple spatial images collected by multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles; a determination unit configured to determine the target size of sub-matrices corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix; a partitioning unit configured to partition the spatial image matrix by sub-matrices corresponding to different regions in the spatial image matrix to obtain multiple image groups; and a second generation unit configured to encode the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0012] In some examples, the aforementioned determining unit is further configured to: determine the target size of the submatrix corresponding to different regions in the spatial image matrix based on the negative correlation between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the submatrix.

[0013] In some examples, adjacent sampling points among the aforementioned multiple sampling points are arranged at preset degree intervals in the longitude and latitude directions, and the aforementioned determining unit is further configured to: in response to determining that the degree of change between adjacent spatial images in the latitude direction in the spatial image matrix is ​​greater than the degree of change between adjacent spatial images in the longitude direction, determine that the length of the sub-matrix is ​​greater than its width, wherein the length direction of the sub-matrix corresponds to the longitude direction of the sampling point, and the width direction of the sub-matrix corresponds to the latitude direction of the sampling point; in response to determining that the degree of change between adjacent spatial images in the longitude direction in the spatial image matrix is ​​negatively correlated with the latitude value, determine that the length of the sub-matrix is ​​positively correlated with the latitude value; based on the fact that the length of the sub-matrix is ​​greater than its width, and the positive correlation between the length of the sub-matrix and the latitude value, determine the target size of the sub-matrix in different regions of the spatial image matrix.

[0014] In some examples, the first generation unit described above is further configured to: arrange multiple spatial images with the longitude of the sampling points corresponding to the spatial images as the horizontal axis and the latitude of the sampling points corresponding to the spatial images as the vertical axis, thereby generating a spatial image matrix.

[0015] In some examples, the second generation unit described above is further configured to: arrange image groups corresponding to sampling points at the same latitude in multiple image groups in ascending order of longitude of the sampling points corresponding to the image groups, generating multiple image group subsequences; arrange multiple image group subsequences in ascending order of latitude of the sampling points corresponding to the image groups, determine the image group identifier corresponding to each of the multiple image groups, and generate an image group sequence; arrange the spatial images in each image group in the image group sequence in the order of keyframes first and prediction frames second, determine the image identifier of the spatial images in each of the multiple image groups, and generate a spatial image sequence; and encode the spatial image sequence based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0016] In some examples, the second generation unit described above is further configured to: determine the spatial image at the center position of the sub-matrix corresponding to each of the multiple image groups as the initial keyframe; for each image group in the multiple image groups, based on the initial keyframe of the image group, determine the keyframe of the image group from the direction where the sampling points corresponding to the spatial images in the image group are sparser, and determine the prediction frames adjacent to the keyframes; arrange the spatial images in each image group in the image group sequence in the order of keyframes first and prediction frames second, determine the image identifier of the spatial images in each image group in the multiple image groups, and generate a spatial image sequence.

[0017] In some examples, the second generation unit described above is further configured to: for each of the multiple image groups, encode the spatial image sequence by using a reference method in which the predicted frame in the image group uniquely references the key frame in the image group, and generate an encoded file.

[0018] Thirdly, embodiments of this application provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, it implements the method as described in any implementation of the first aspect.

[0019] Fourthly, embodiments of this application provide an electronic device, including: one or more processors; and a storage device storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect.

[0020] The encoding method and apparatus for spatial images provided in this application generate a spatial image matrix containing multiple spatial images collected from multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles; determine the target size of the sub-matrix corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix; divide the spatial image matrix into multiple image groups by using the sub-matrix corresponding to different regions in the spatial image matrix; and encode the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file. This provides an encoding method suitable for spatial images. During the encoding process, the spatial images are reorganized to obtain a spatial image matrix, and image groups are designated within the spatial image matrix to manage the spatial images in the image groups, achieving temporal decoupling of the spatial images. Furthermore, by utilizing the spatial distribution characteristics of the spatial images, the encoding structure of the image groups is determined, improving the compression efficiency of the encoding process, contributing to improved decoding efficiency, and reducing memory usage during the decoding process. Attached Figure Description

[0021] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0022] Figure 1 This is an exemplary system architecture diagram in which one embodiment of this application can be applied;

[0023] Figure 2 This is a flowchart of one embodiment of the encoding method applicable to spatial images according to this application;

[0024] Figure 3 This is a schematic diagram of the spatial layout of sampling points according to the latitude and longitude sampling method of this embodiment;

[0025] Figure 4 This is a schematic diagram of the spatial image matrix according to this embodiment;

[0026] Figure 5 This is a schematic diagram of sub-matrices of different regions in the spatial image matrix according to this embodiment;

[0027] Figure 6 This is a schematic diagram illustrating a spatial image sequence according to this embodiment;

[0028] Figure 7 This is a schematic diagram illustrating the reference method between predicted frames and keyframes in an image group according to this embodiment;

[0029] Figure 8 This is a schematic diagram illustrating an application scenario of the encoding method for spatial images according to this embodiment;

[0030] Figure 9 This is a flowchart of yet another embodiment of the encoding method applicable to spatial images according to this application;

[0031] Figure 10 This is a schematic diagram of the decoding trajectory according to this embodiment;

[0032] Figure 11 This is a schematic diagram of the screen sliding trajectory according to this embodiment;

[0033] Figure 12 This is a structural diagram of one embodiment of an encoding apparatus for spatial images according to this application;

[0034] Figure 13 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of this application. Detailed Implementation

[0035] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0036] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0037] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0038] Figure 1 An exemplary architecture 100 for encoding methods and apparatus suitable for spatial images, to which the present application can be applied, is shown.

[0039] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The communication connections between terminal devices 101, 102, and 103 form a network topology. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0040] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be hardware or software that supports network connectivity for data interaction and processing. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices that support network connectivity, information acquisition, interaction, display, and processing functions, including but not limited to smartphones, image acquisition devices, tablets, e-book readers, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as multiple software programs or software modules, for example, to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0041] Server 105 can be a server that provides various services, such as receiving spatial images provided by terminal devices 101, 102, and 103, determining the image group encoding structure using the spatial distribution characteristics of the spatial images, and then encoding them to obtain encoded files. As an example, server 105 can be a cloud server.

[0042] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (such as software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0043] It should also be noted that the encoding method for spatial images provided in the embodiments of this application can be executed by a server, by a terminal device, or by a combination of both. Accordingly, the various parts (e.g., various units) of the encoding apparatus for spatial images can be all located in the server, all located in the terminal device, or located separately in the server and the terminal device.

[0044] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Any number of terminal devices, networks, and servers can be included depending on implementation needs. When the electronic devices on which the encoding method for spatial imagery runs do not require data transmission with other electronic devices, the system architecture may consist only of the electronic devices (e.g., servers or terminal devices) on which the encoding method for spatial imagery runs.

[0045] Continue to refer to Figure 2 The flowchart 200, illustrating an embodiment of an encoding method suitable for spatial images, includes the following steps:

[0046] Step 201: Generate a spatial image matrix that includes multiple spatial images collected from multiple sampling points, based on the latitude and longitude corresponding to multiple sampling points at different spatial angles.

[0047] In this embodiment, the execution subject of the encoding method applicable to spatial images (e.g.) Figure 1 The terminal device or server generates a spatial image matrix based on the latitude and longitude corresponding to multiple sampling points at different spatial angles. Each spatial image represents a picture of a target object obtained from a different spatial angle. The target object can be a person, an object, or any other object.

[0048] As an example, multiple sampling points can be evenly set at different spatial angles to capture spatial images of the target object from different spatial angles, which is a uniform sampling method.

[0049] As another example, multiple sampling points can be non-uniformly set around the target object at different spatial angles by pre-setting latitude and longitude intervals, so as to take spatial images of the target object from different spatial angles, that is, latitude and longitude sampling method.

[0050] like Figure 3 As shown, a spatial layout diagram 300 of a latitude and longitude sampling method is illustrated. For the upper hemisphere of the target object, sampling is performed at 10° intervals. Therefore, there are 36 sampling points in the longitude direction and 9 sampling points in the latitude direction, resulting in 324 (36×9) sampling points in the upper hemisphere of the target object. Figure 3 The left sub-figure is a top view of the spatial layout of the sampling points, and the right sub-figure is a front view of the spatial layout of the sampling points.

[0051] Sampling points at different spatial angles correspond to different latitudes and longitudes, thus obtaining spatial images at different spatial angles. Arranging these spatial images in a specific order yields a spatial image matrix. As an example, for each spatial image, it is named after the latitude and longitude of its corresponding sampling point; then, based on the naming information of each spatial image, multiple spatial images are arranged to generate a spatial image matrix. For example, the latitude and longitude of the sampling points corresponding to the spatial images are (10... ° 20 ° If so, the spatial image can be named "10-20.jpg".

[0052] In some optional implementations of this embodiment, the execution entity can perform step 201 as follows: arranging multiple spatial images with the longitude of the sampling points corresponding to the spatial images as the horizontal axis and the latitude of the sampling points corresponding to the spatial images as the vertical axis to generate a spatial image matrix.

[0053] Specifically, using the longitude of the sampling points corresponding to the spatial images as the horizontal axis and the latitude of the sampling points corresponding to the spatial images as the vertical axis, multiple spatial images are arranged in ascending order of longitude and ascending order of latitude to obtain a spatial image matrix.

[0054] Continuing with the above Figure 3 Taking the sampling points shown as an example, the generated spatial image matrix is ​​as follows: Figure 4 As shown. The dimensions in the spatial image matrix range from 0 to 1. ° -80 ° The longitude range is 0. ° -350 ° .

[0055] In this implementation, based on the latitude and longitude of the sampling points, an arrangement method that better reflects the spatial correlation between multiple spatial images is provided, making the generated spatial image matrix more conducive to spatial encoding and thus improving spatial encoding efficiency.

[0056] Step 202: Determine the target size of the submatrix corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix.

[0057] In this embodiment, the execution entity can determine the target size of the sub-matrix corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix.

[0058] The degree of variation between adjacent spatial images is represented by the degree of change in the image content represented by adjacent spatial images, or the degree of change in the position of adjacent sampling points corresponding to each adjacent spatial image. For the degree of variation in image content, for example, image features of each adjacent spatial image can be extracted using a feature extraction network; these image features represent the image content of the spatial image. Then, the degree of variation between adjacent spatial images is determined by calculating the similarity between the image features (e.g., cosine similarity or Euclidean distance). Notably, the similarity between adjacent spatial images is negatively correlated with the degree of variation. Alternatively, the histograms corresponding to each adjacent spatial image can be determined, and the degree of variation between adjacent spatial images can be determined by the distance between the histograms.

[0059] Regarding the degree of positional variation between adjacent sampling points, when the latitude and longitude distance between the sampling points corresponding to adjacent spatial images is large, the degree of variation between adjacent images is greater; when the latitude and longitude distance between the sampling points corresponding to adjacent spatial images is small, the degree of variation between adjacent images is smaller. It can be understood that the degree of variation in the image content of adjacent spatial images collected from adjacent sampling points is often positively correlated with the degree of positional variation between adjacent sampling points.

[0060] As an example, the aforementioned execution entity is equipped with a correspondence table that represents the relationship between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the submatrix corresponding to different regions in the spatial image matrix. Based on the correspondence table and the degree of change between adjacent spatial images in the spatial image matrix, the target size of the submatrix corresponding to different regions of adjacent spatial images in the spatial image matrix is ​​determined.

[0061] As another example, based on the change data of the target size of the submatrix as a function of the degree of change between adjacent spatial images, linear fitting is performed to determine the functional model between the target size of the submatrix and the degree of change between adjacent spatial images; then, based on the determined functional model and the degree of change between adjacent spatial images in the spatial image matrix, the target size of the submatrix corresponding to different regions in the spatial image matrix is ​​determined.

[0062] In some optional implementations of this embodiment, the execution entity can perform step 202 as follows: based on the negative correlation between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the sub-matrix, determine the target size of the sub-matrix corresponding to different regions in the spatial image matrix.

[0063] exist Figure 3 As shown in the latitude and longitude sampling method, the degree of change between adjacent spatial images is not the same. Specifically, the degree of change between adjacent spatial images in the longitude direction decreases as the latitude increases.

[0064] The principle is that the degree of change between adjacent spatial images in the spatial image matrix is ​​negatively correlated with the target size of the submatrix. The smaller the degree of change between adjacent spatial images, the larger the target size of the submatrix of the region where the adjacent spatial images are located in the spatial image matrix; the greater the degree of change between adjacent spatial images, the smaller the target size of the submatrix of the region where the adjacent spatial images are located in the spatial image matrix.

[0065] The submatrix is ​​used to partition the spatial image matrix in subsequent steps, resulting in image groups. For spatial images with low variability, larger image groups are used to contain more spatial images; for spatial images with high variability, smaller image groups are used to contain fewer spatial images, which is more conducive to data compression efficiency in the encoding process.

[0066] In some optional implementations of this embodiment, adjacent sampling points among multiple sampling points are arranged at preset degree intervals in both the longitude and latitude directions. The preset degree intervals can be specifically set according to actual conditions. As an example, continue to refer to... Figure 3 Multiple sampling points are arranged at 10° intervals in both the longitude and latitude directions.

[0067] In this implementation, the execution entity can determine the target size of the submatrix in the following way:

[0068] First, in response to determining that the degree of variation between adjacent spatial images in the latitude direction is greater than the degree of variation between adjacent spatial images in the longitude direction, the length of the submatrix is ​​determined to be greater than its width. The length direction of the submatrix corresponds to the longitude direction of the sampling points, and the width direction corresponds to the latitude direction of the sampling points. Then, in response to determining that the degree of variation between adjacent spatial images in the longitude direction is negatively correlated with the latitude value, the length of the submatrix is ​​determined to be positively correlated with the latitude value. Finally, based on the fact that the length of the submatrix is ​​greater than its width, and the positive correlation between the length and latitude value, the target size of the submatrix in different regions of the spatial image matrix is ​​determined.

[0069] Continue to refer to Figure 5 The diagram 500 illustrates submatrices for different regions within a spatial image matrix. Based on the first principle that the length of a submatrix is ​​greater than its width and the second principle that the length of a submatrix is ​​positively correlated with its latitude, the target sizes of the submatrices are 9×4 and 36×1, respectively. Specifically, the target size of the submatrix corresponding to the region containing the low-latitude spatial image is 9×4, while the target size of the submatrix corresponding to the region containing the high-latitude spatial image is 36×1.

[0070] This implementation provides a specific method for determining the target size of the submatrix. Based on the first principle that the length of the submatrix is ​​greater than its width and the second principle that the length of the submatrix is ​​positively correlated with the dimensional value, it improves the adaptability of the determined submatrix to the spatial distribution characteristics of the spatial image, which helps to further improve coding efficiency.

[0071] Step 203: Divide the spatial image matrix into sub-matrices corresponding to different regions in the spatial image matrix to obtain multiple image groups.

[0072] In this embodiment, the aforementioned execution entity can divide the spatial image matrix into multiple image groups by using sub-matrices corresponding to different regions in the spatial image matrix.

[0073] Continue to refer to Figure 5 The spatial image matrix was divided into nine image groups, GOP1-GOP9, by using sub-matrices corresponding to different regions. Specifically, the sub-matrix corresponding to image groups GOP1-GOP8 is 9×4, and the sub-matrix corresponding to GOP9 is 36×1.

[0074] Step 204: Encode the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0075] In this embodiment, the aforementioned execution entity can encode the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups, and generate an encoded file.

[0076] In the compression encoding process, each spatial image represents a still image. Various compression algorithms are used to reduce data size during actual compression, with IPB frames being the most common. I-frames in IPB frames are also called keyframes or intra-coded frames. Keyframes are usually the first frame of each image group, moderately compressed, and used as reference points for random access to generate still images. A keyframe can be seen as the product of image compression, which removes redundant information from the video. P-frames are called prediction frames or forward predictive coded frames. For the compressed data corresponding to a prediction frame, redundant information identical to that of the compressed data corresponding to the keyframe in the same image group is removed to obtain the coded data corresponding to the prediction frame. A prediction frame represents the difference between itself and its corresponding keyframe. During decoding, the spatial image corresponding to the prediction frame is generated by referring to the corresponding keyframe and the decoded data of the prediction frame.

[0077] In this embodiment, a preset determination method can be used to determine the keyframes and prediction frames in the image group; then, based on the keyframes and prediction frames included in each of the multiple image groups, the spatial images in the spatial image matrix are encoded to generate an encoded file.

[0078] For example, for each image group in multiple image groups, the spatial image corresponding to the sampling point with the smallest latitude and longitude in the image group is used as the keyframe, and the remaining spatial images in the image group are used as prediction frames. The spatial images in the spatial image matrix are then encoded to generate an encoded file.

[0079] In some optional implementations of this embodiment, the execution entity can perform step 204 as follows:

[0080] First, arrange the image groups corresponding to the sampling points of the image groups in ascending order of longitude, and generate multiple image group subsequences.

[0081] Continue to refer to Figure 5 The multiple image group subsequences include a first image group subsequence, a second image group subsequence, and a third image group subsequence. Specifically, the first image group subsequence is “GOP1—>GOP2—>GOP3—>GOP4”, the second image group subsequence is “GOP5—>GOP6—>GOP7—>GOP8”, and the third image group subsequence is GOP9.

[0082] Second, arrange multiple image group sub-sequences according to the latitude of the sampling points corresponding to the image group in ascending order, determine the image group identifier corresponding to each of the multiple image groups, and generate the image group sequence.

[0083] Continue to refer to Figure 5The image group sequence is "GOP1—>GOP2—>GOP3……—>GOP8—>GOP9".

[0084] Third, following the order of keyframes first and prediction frames later, arrange the spatial images in each image group in the image group sequence, determine the image identifier of the spatial images in each image group in multiple image groups, and generate a spatial image sequence.

[0085] Specifically, based on the determined image group sequence, for the spatial images in each image group, the image sequence within the image group is determined in the order of keyframes first and then prediction frames. Then, based on the image sequence corresponding to the image group, the image identifier of the spatial image in each image group is determined, and finally the spatial image sequence is obtained.

[0086] For multiple predicted frames in each image group, the order of these frames can be determined according to a preset method. For example, multiple predicted frames can be arranged in ascending order of latitude and ascending order of longitude of the sampling points corresponding to the spatial images.

[0087] Continue to refer to Figure 6 The diagram illustrates a spatial image sequence. In each image group, the key is labeled "0", and the predicted frames are labeled "1-35".

[0088] Fourth, based on the keyframes and prediction frames included in each of the multiple image groups, the spatial image sequence is encoded to generate an encoded file.

[0089] In this implementation, the spatial image matrix is ​​first arranged to obtain a spatial image sequence. Then, based on the arrangement order of the spatial images in the spatial image sequence, as well as the keyframes and prediction frames in each image group, encoding is performed, which further improves the efficiency and accuracy of the encoding process.

[0090] In some optional implementations of this embodiment, the execution entity can perform the third step as follows: First, determine the spatial image at the center position of the sub-matrix corresponding to each of the multiple image groups as the initial keyframe; then, for each image group in the multiple image groups, based on the initial keyframe of the image group, determine the keyframe of the image group from the direction where the sampling points corresponding to the spatial images in the image group are sparser, and determine the prediction frames adjacent to the keyframes; finally, arrange the spatial images in each image group in the image group sequence in the order of keyframes first and prediction frames second, determine the image identifier of the spatial images in each image group in the multiple image groups, and generate a spatial image sequence.

[0091] Continue with Figure 5Taking the 9×4 submatrix as an example, for the 36 spatial images in the resulting image group, based on the initial keyframe at the center, a keyframe is determined from the direction where the sampling points of the spatial images in that image group are sparser, that is, the direction with smaller latitude, specifically at position (40, 10). Then, for the prediction frames around the keyframe, the sorting information of multiple prediction frames is determined by referring to the left-right symmetry method, and the spatial images in each image group in the image group sequence are arranged to determine the image identifier of the spatial images in the image group. As an example, for the image group corresponding to the 9×4 submatrix, you can refer to... Figure 5 The encoding order in GOP1 determines the image identifier of the spatial image in the image group.

[0092] For a 36×1 submatrix corresponding to an image group, since the variation between adjacent spatial images is very small, any spatial image in it can be determined as a keyframe. For example, the first spatial image in the image group can be determined as a keyframe. Then, the spatial images in the image group are arranged in ascending order of longitude to determine the image identifier of the spatial images in the image group corresponding to the 36×1 submatrix.

[0093] Based on the initial keyframe at the center, keyframes are determined from the direction where the sampling points of the spatial images in the image group are sparser. This makes the changes between the keyframes in the image group and the surrounding predicted frames smaller, which further helps to improve the coding efficiency of the coding process and reduce the data size of the encoded file.

[0094] In some optional implementations of this embodiment, the execution entity can perform the fourth step as follows: for each of the multiple image groups, the spatial image sequence is encoded by using a reference method in which the predicted frame in the image group uniquely references the key frame in the image group, and an encoded file is generated.

[0095] Continue to refer to Figure 7 The diagram 700 illustrates the referencing method between predicted frames and keyframes in a picture group. For each predicted frame in the picture group, a keyframe in that picture group is uniquely referenced to encode the resulting encoded file.

[0096] In this implementation, the spatial image sequence is encoded by using a method where the predicted frame in the image group is uniquely referenced by the key frame in the image group. This reduces the complexity of the relationship between the predicted frame and the key frame in the encoded file, which helps to improve the data determination speed and decoding efficiency during the decoding process.

[0097] See also Figure 8 , Figure 8 This is a schematic diagram 800 illustrating an application scenario of the encoding method for spatial images according to this embodiment. Figure 8In the application scenario, firstly, multiple spatial images of the target object are obtained by sampling from multiple sampling points at different spatial angles using an image acquisition device. The spatial layout of the multiple sampling points is shown in 801. For the upper hemisphere of the target object, sampling is performed at 10° intervals, resulting in 36 sampling points in the longitude direction and 9 sampling points in the latitude direction, for a total of 324 (36×9) sampling points in the upper hemisphere. Then, based on the degree of change between adjacent spatial images in the spatial image matrix, the target size of the sub-matrix corresponding to different regions in the spatial image matrix is ​​determined. Next, the spatial image matrix 802 is divided using sub-matrix 8021 corresponding to different regions in the spatial image matrix, resulting in multiple image groups. Finally, based on the keyframes and prediction frames included in each of the multiple image groups, the spatial images in the spatial image matrix are encoded to generate an encoded file.

[0098] The method provided in the above embodiments of this application generates a spatial image matrix comprising multiple spatial images collected from multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles; determines the target size of the sub-matrix corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix; divides the spatial image matrix into multiple image groups by using the sub-matrix corresponding to different regions in the spatial image matrix; and encodes the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file. This provides an encoding method suitable for spatial images. During the encoding process, the spatial images are reorganized to obtain a spatial image matrix, and image groups are specified in the spatial image matrix to manage the spatial images in the image groups, achieving temporal decoupling of the spatial images. Furthermore, by utilizing the spatial distribution characteristics of the spatial images, the encoding structure of the image groups is determined, improving the compression efficiency of the encoding process, which helps to improve decoding efficiency and reduce memory usage during the decoding process.

[0099] Continue to refer to Figure 9 The illustration shows a schematic flow 900 of an encoding method for spatial images according to this application, comprising the following steps:

[0100] Step 901: Arrange multiple spatial images with the longitude of the sampling points corresponding to the spatial images as the horizontal axis and the latitude of the sampling points corresponding to the spatial images as the vertical axis to generate a spatial image matrix.

[0101] Step 902: In response to determining that the degree of change between adjacent spatial images in the latitude direction in the spatial image matrix is ​​greater than the degree of change between adjacent spatial images in the longitude direction, determine that the length of the submatrix is ​​greater than its width.

[0102] The length direction of the submatrix corresponds to the longitude direction of the sampling point, and the width direction of the submatrix corresponds to the latitude direction of the sampling point.

[0103] Step 903: In response to determining the degree of change between adjacent spatial images in the longitude direction in the spatial image matrix, which is negatively correlated with latitude values, the length of the submatrix is ​​determined to be positively correlated with latitude values.

[0104] Step 904: Based on the fact that the length of the submatrix is ​​greater than its width, and the positive correlation between the length of the submatrix and the latitude value, determine the target size of the submatrix in different regions of the spatial image matrix.

[0105] Step 905: Divide the spatial image matrix into sub-matrices corresponding to different regions in the spatial image matrix to obtain multiple image groups.

[0106] Step 906: Arrange the image groups corresponding to the sampling points of the image groups in ascending order of longitude, and generate multiple image group subsequences.

[0107] Step 907: Arrange multiple image group sub-sequences according to the latitude of the sampling points corresponding to the image group in ascending order, determine the image group identifier corresponding to each of the multiple image groups, and generate the image group sequence.

[0108] Step 908: Determine the spatial image at the center position of the sub-matrix corresponding to each of the multiple image groups as the initial keyframe.

[0109] Step 909: For each of the multiple image groups, based on the initial keyframes of the image group, determine the keyframes of the image group from the direction where the sampling points corresponding to the spatial images in the image group are sparser, and determine the prediction frames adjacent to the keyframes.

[0110] Step 910: Arrange the spatial images in each image group in the image group sequence according to the order of keyframes first and prediction frames second, determine the image identifier of the spatial images in each image group in the multiple image groups, and generate a spatial image sequence.

[0111] Step 911: Encode the spatial image sequence based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0112] As can be seen from this embodiment, with Figure 2Compared with the corresponding embodiments, the flowchart 900 of the encoding method applicable to spatial images in this embodiment specifically describes the process of generating the spatial image matrix, determining the target size of the sub-matrix, determining the spatial image sequence, and determining the keyframes in the image group. It makes full use of the spatial distribution characteristics of spatial images to determine the encoding structure of the image group, further improving the compression efficiency of the encoding process, helping to improve the decoding efficiency, and reducing the memory usage of the decoding process.

[0113] Regarding the encoded files obtained in embodiments 200 and 900 above, an illustrative flow of an embodiment of a decoding method suitable for spatial images is provided, including the following steps:

[0114] The first step is to determine the target latitude and longitude corresponding to the target spatial image desired by the user, based on the obtained operational information.

[0115] In this embodiment, the execution entity of the decoding method applicable to spatial images (e.g., Figure 1 The terminal device or server in the process can determine the target latitude and longitude corresponding to the target spatial image desired by the user based on the acquired operation information.

[0116] The operation information can be the action command corresponding to the user's swipe operation or the voice command corresponding to the voice information. (Continue to refer to...) Figure 10 This shows the user's operation trajectory in the spatial image matrix. Following the operation trajectory, the aforementioned execution entity aims to decode the spatial image data at the corresponding location, obtain and display the target spatial image desired by the user.

[0117] As an example, the aforementioned execution entity can pre-establish a correspondence between the user's operation position on the screen and the target latitude and longitude of the target spatial image desired by the user. This allows for the real-time determination of the target latitude and longitude of the desired spatial image during the user's operation. The target latitude and longitude of the target spatial image are the latitude and longitude of the sampling points corresponding to the target spatial image.

[0118] The second step is to determine the location information of the target spatial image and the keyframes in the target image group to which the target spatial image belongs, in the encoding file, based on the target's latitude and longitude.

[0119] In this embodiment, the aforementioned execution entity can determine the location information of the target spatial image and the keyframes in the target image group to which the target spatial image belongs, within the encoded file, based on the target latitude and longitude.

[0120] Location information includes the location information of the target spatial image in the encoding file and the location information of the keyframes in the target image group to which the target spatial image belongs in the encoding file.

[0121] As an example, the aforementioned execution entity can pre-establish the correspondence between the latitude and longitude of each spatial image involved in the encoding file and the location information of the spatial image in the encoding file. Thus, based on the target latitude and longitude, the location information of the target spatial image in the encoding file is determined; based on the target latitude and longitude, the keyframe latitude and longitude of the keyframes in the target image group to which the target spatial image belongs are determined, thereby determining the location information of the keyframes in the encoding file.

[0122] In some optional implementations of this embodiment, the execution entity can perform the second step as follows:

[0123] First, based on the target latitude and longitude and the keyframe latitude and longitude of the keyframes in each image group in the encoding file, the target image group identifier and the target image identifier of the target spatial image are determined.

[0124] As an example, the aforementioned execution entity can determine the keyframe latitude and longitude of keyframes in each image group in the encoded file, and generate a set of keyframe latitude and longitude. Then, it compares the target latitude and longitude with the keyframe latitude and longitude in the set of keyframe latitude and longitude. Based on the comparison results between the target latitude and longitude and the keyframe latitude and longitude in the set of keyframe latitude and longitude, it determines the target image group to which the target spatial image belongs. Then, it determines the target image group identifier of the target image group and the target image identifier of the target spatial image.

[0125] Continue to refer to Figure 5 The corresponding keyframe latitude and longitude set is as follows:

[0126]

[0127] Then, the location information is determined based on the target image group identifier and the target image identifier.

[0128] During the encoding process to obtain the encoded file, the spatial image identifier of the spatial image and the image group identifier of the image group to which the spatial image belongs are typically encoded. After determining the target image group identifier and the target image identifier, the position information of the target spatial image in the encoded file and the position information of the keyframes in the target image group to which the target spatial image belongs can be determined in the encoded file.

[0129] This implementation provides a specific method for determining the position information of a target spatial image in the encoding file and the position information of keyframes in the target image group to which the target spatial image belongs in the encoding file, thereby improving the efficiency and accuracy of the position information determination process.

[0130] In some optional implementations of this embodiment, the execution entity can determine the target image group identifier and the target image identifier in the following way: First, among the keyframe latitude and longitude coordinates of keyframes in each image group in the encoded file, the image group to which the keyframe corresponding to the keyframe latitude and longitude closest to the target latitude and longitude belongs is taken as the target image group, and the target image group identifier is determined; then, the target image identifier is determined according to the offset between the target latitude and longitude and the keyframe latitude and longitude coordinates corresponding to the keyframes in the target image group.

[0131] As an example, if the target latitude and longitude is (60, 20), and it is closest to the keyframe latitude and longitude (40, 10) in the keyframe latitude and longitude set, then the image group GOP1 to which the keyframe corresponding to the keyframe latitude and longitude (40, 10) belongs is taken as the target image group, and the target image group is identified as 1.

[0132] Then, the offset between the target latitude and longitude and the keyframe latitude and longitude corresponding to the keyframe in the target image group is determined to be (20, 10), and the target image identifier is determined to be 13.

[0133] Continue with Figure 5 Taking the submatrix shown as an example, the offset between the latitude and longitude of the spatial images in image groups GOP1-GOP8 and the latitude and longitude of the keyframes is as follows:

[0134]

[0135] The offset between the latitude and longitude of the spatial images in image group GOP9 and the latitude and longitude of the keyframes is:

[0136]

[0137] According to the arrangement order of spatial images in the image sequence corresponding to the image group during the encoding process, the above offset set is reordered to obtain a sorted offset sequence; based on the sorted offset sequence and the obtained offsets, the target image identifier corresponding to the target spatial image can be determined.

[0138] In this implementation, the target image group identifier is determined based on the comparison between the keyframe latitude and longitude of each keyframe involved in the encoding file and the target latitude and longitude of the target spatial image expected by the user, thereby determining the target image identifier, which improves the universality and accuracy of the identifier information determination process.

[0139] The third step is to decode the data in the encoded file at the location information to obtain the target spatial image.

[0140] In this example, the aforementioned execution entity can decode the data in the encoded file located at the position information to obtain the target spatial image.

[0141] Once the location information of the target data to be decoded is determined, the encoded data at the corresponding location in the encoded file can be decoded, and the target spatial image can be decoded and displayed.

[0142] In some optional implementations of this embodiment, the execution entity can perform the third step as follows:

[0143] First, based on the target image identifier, determine whether the target spatial image is a keyframe in the target image group or a predicted frame in the target image group.

[0144] As an example, if the target image identifier is determined to be the same as the keyframe identifier of a keyframe in the target image group, then the target spatial image is determined to be a keyframe in the target image group; if the target image identifier is determined to be the same as the prediction frame identifier of a prediction frame in the target image group, then the target spatial image is determined to be a prediction frame in the target image group.

[0145] Then, in response to determining that the target spatial image is a predicted frame in the target image group, the keyframes in the target image group at the location represented by the location information and the predicted frames corresponding to the target image identifiers are decoded to obtain the target spatial image.

[0146] When the target spatial image is a predicted frame in the target image group, since the predicted frame references the keyframes in the image group, it is necessary to simultaneously decode the keyframes in the target image group at the location represented by the location information and the predicted frame corresponding to the target image identifier in order to obtain the target spatial image.

[0147] In some optional implementations of this embodiment, the execution entity may also perform the third step as follows: in response to determining that the target spatial image is a keyframe in the target image group, decode the keyframe in the target image group at the position represented by the location information to obtain the target spatial image.

[0148] When the target spatial image is a keyframe in the target image group, the keyframe in the target image group located at the position represented by the position information can be directly decoded to obtain the target spatial image without referring to other spatial images.

[0149] Continue to refer to Figure 11 , showed Figure 10A schematic diagram 1100 of the decoding trajectory of the corresponding operation trajectory is provided. In this embodiment, the target latitude and longitude corresponding to the target spatial image expected by the user are determined based on the acquired operation information; based on the target latitude and longitude, the location information of the target spatial image and the keyframes in the target image group to which the target spatial image belongs are determined in the encoding file; the data in the encoding file at the position represented by the location information is decoded to obtain the target spatial image, thereby providing a decoding method suitable for spatial images. Since the spatial images in the encoding file are decoupled in time, the target latitude and longitude are mapped to the position of the corresponding data in the encoding file, thereby realizing decoding at any position and improving the flexibility of decoding.

[0150] Regarding the encoded files obtained in embodiments 200 and 900 above, an illustrative flow of another embodiment of a decoding method suitable for spatial images is provided, including the following steps:

[0151] The first step is to determine the target latitude and longitude corresponding to the target spatial image desired by the user, based on the obtained operational information.

[0152] The second step is to identify the target image group by taking the keyframe whose latitude and longitude are closest to the target latitude and longitude in the keyframe latitude and longitude of each keyframe in the image group of the encoded file.

[0153] The third step is to determine the target image identifier based on the offset between the target latitude and longitude and the keyframe latitude and longitude corresponding to the keyframes in the target image group.

[0154] The third step involves determining the location information of the target spatial image and the keyframes within the target image group to which the target spatial image belongs, within the encoded file, based on the target image group identifier and the target image identifier.

[0155] The fifth step is to determine whether the target spatial image is a keyframe in the target image group or a predicted frame in the target image group, based on the target image identifier.

[0156] The sixth step, in response to determining that the target spatial image is a predicted frame in the target image group, decodes the keyframes in the target image group at the location represented by the location information and the predicted frames corresponding to the target image identifiers, to obtain the target spatial image.

[0157] The seventh step is to decode the keyframes in the target image group in response to the determination that the target spatial image is a keyframe in the target image group, thereby obtaining the target spatial image.

[0158] As can be seen from this embodiment, compared with the above-described decoding method embodiments, the flow of the decoding method applicable to spatial images in this embodiment specifically illustrates the process of determining location information and the decoding process of the target spatial image, further improving the efficiency and flexibility of the decoding process for spatial images.

[0159] Continue to refer to Figure 12 As an implementation of the methods shown in the above figures, this application provides an embodiment of an encoding apparatus suitable for spatial images, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0160] like Figure 12 As shown, the encoding device for spatial images includes: a first generation unit 1201, configured to generate a spatial image matrix comprising multiple spatial images collected from multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles; a determination unit 1202, configured to determine the target size of sub-matrices corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix; a partitioning unit 1203, configured to partition the spatial image matrix using sub-matrices corresponding to different regions in the spatial image matrix to obtain multiple image groups; and a second generation unit 1204, configured to encode the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0161] In some optional implementations of this embodiment, the determining unit 1202 is further configured to: determine the target size of the submatrix corresponding to different regions in the spatial image matrix based on the negative correlation between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the submatrix.

[0162] In some optional implementations of this embodiment, adjacent sampling points among the plurality of sampling points are arranged at preset degree intervals in the longitude and latitude directions, and the determination unit 1202 is further configured to: in response to determining that the degree of change between adjacent spatial images in the latitude direction in the spatial image matrix is ​​greater than the degree of change between adjacent spatial images in the longitude direction, determine that the length of the sub-matrix is ​​greater than its width, wherein the length direction of the sub-matrix corresponds to the longitude direction of the sampling point, and the width direction of the sub-matrix corresponds to the latitude direction of the sampling point; in response to determining that the degree of change between adjacent spatial images in the longitude direction in the spatial image matrix is ​​negatively correlated with the latitude value, determine that the length of the sub-matrix is ​​positively correlated with the latitude value; based on the fact that the length of the sub-matrix is ​​greater than its width, and the positive correlation between the length of the sub-matrix and the latitude value, determine the target size of the sub-matrix in different regions of the spatial image matrix.

[0163] In some optional implementations of this embodiment, the first generation unit is further configured to: arrange multiple spatial images with the longitude of the sampling points corresponding to the spatial images as the horizontal axis and the latitude of the sampling points corresponding to the spatial images as the vertical axis, and generate a spatial image matrix.

[0164] In some optional implementations of this embodiment, the second generation unit 1201 is further configured to: arrange image groups corresponding to sampling points of the same latitude in multiple image groups in ascending order of longitude of the sampling points corresponding to the image groups, and generate multiple image group sub-sequences; arrange multiple image group sub-sequences in ascending order of latitude of the sampling points corresponding to the image groups, determine the image group identifier corresponding to each of the multiple image groups, and generate an image group sequence; arrange the spatial images in each image group in the image group sequence in the order of keyframes first and prediction frames second, determine the image identifier of the spatial images in each of the multiple image groups, and generate a spatial image sequence; and encode the spatial image sequence according to the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0165] In some optional implementations of this embodiment, the second generation unit 1204 is further configured to: determine the spatial image at the center position of the sub-matrix corresponding to each of the multiple image groups as the initial keyframe; for each image group in the multiple image groups, based on the initial keyframe of the image group, determine the keyframe of the image group from the direction where the sampling points corresponding to the spatial images in the image group are sparser, and determine the prediction frames adjacent to the keyframes; arrange the spatial images in each image group in the image group sequence in the order of keyframes first and prediction frames second, determine the image identifier of the spatial images in each image group in the multiple image groups, and generate a spatial image sequence.

[0166] In some optional implementations of this embodiment, the second generation unit 1204 is further configured to: for each of the multiple image groups, use a reference method in which the predicted frame in the image group uniquely references the key frame in the image group to encode the spatial image sequence and generate an encoded file.

[0167] In this embodiment, the first generation unit in the encoding device suitable for spatial images generates a spatial image matrix containing multiple spatial images collected from multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles; the determination unit determines the target size of the sub-matrix corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix; the partitioning unit partitions the spatial image matrix using the sub-matrix corresponding to different regions in the spatial image matrix to obtain multiple image groups; the second generation unit encodes the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file, thereby providing an encoding method suitable for spatial images. During the encoding process, the spatial images are reorganized to obtain a spatial image matrix, and image groups in the spatial image matrix are specified to manage the spatial images in the image groups, achieving temporal decoupling of the spatial images; furthermore, by utilizing the spatial distribution characteristics of the spatial images, the image group encoding structure is determined, which improves the compression efficiency of the encoding process, helps to improve the decoding efficiency, and reduces the memory usage of the decoding process.

[0168] The following is for reference. Figure 13 It illustrates a device suitable for implementing embodiments of this application (e.g., Figure 1 The diagram shows the structure of the computer system 1300 of the devices 101, 102, 103, and 105. Figure 13 The device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0169] like Figure 13 As shown, the computer system 1300 includes a processor (e.g., CPU, Central Processing Unit) 1301, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1302 or a program loaded from storage portion 1308 into random access memory (RAM) 1303. The RAM 1303 also stores various programs and data required for the operation of the system 1300. The processor 1301, ROM 1302, and RAM 1303 are interconnected via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0170] The following components are connected to I / O interface 1305: an input section 1306 including a keyboard, mouse, etc.; an output section 1307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to I / O interface 1305 as needed. Removable media 1311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1310 as needed so that computer programs read from them can be installed into storage section 1308 as needed.

[0171] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1309, and / or installed from removable medium 1311. When the computer program is executed by processor 1301, it performs the functions defined in the methods of this application.

[0172] It should be noted that the computer-readable medium of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0173] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the client computer, partially on the client computer, as a standalone software package, partially on the client computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the client computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0174] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0175] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, it can be described as: a processor including a first generation unit, a determining unit, a partitioning unit, and a second generation unit. The names of these units do not necessarily limit the specific unit itself; for example, the determining unit can also be described as "a unit that determines the target size of sub-matrices corresponding to different regions in a spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix."

[0176] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the computer device to: generate a spatial image matrix comprising multiple spatial images collected from multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles; determine the target size of sub-matrices corresponding to different regions in the spatial image matrix based on the degree of change between adjacent spatial images in the spatial image matrix; divide the spatial image matrix into multiple image groups by using sub-matrices corresponding to different regions in the spatial image matrix; and encode the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the multiple image groups to generate an encoded file.

[0177] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A coding method suitable for spatial images, comprising: Based on the latitude and longitude corresponding to multiple sampling points at different spatial angles, a spatial image matrix is ​​generated, which includes multiple spatial images collected from the multiple sampling points. Based on the negative correlation between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the submatrix, the target size of the submatrix corresponding to different regions in the spatial image matrix is ​​determined, wherein the degree of change between adjacent spatial images represents the degree of change in the position between adjacent sampling points corresponding to each adjacent spatial image; The spatial image matrix is ​​divided into multiple image groups by sub-matrices corresponding to different regions in the spatial image matrix. Based on the keyframes and prediction frames included in each of the multiple image groups, the spatial images in the spatial image matrix are encoded to generate an encoded file.

2. The method according to claim 1, wherein, The adjacent sampling points among the multiple sampling points are arranged at preset degree intervals in the longitude and latitude directions, and The step of determining the target size of the submatrix corresponding to different regions in the spatial image matrix based on the negative correlation between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the submatrix includes: In response to determining that the degree of change between adjacent spatial images in the latitude direction in the spatial image matrix is ​​greater than the degree of change between adjacent spatial images in the longitude direction, it is determined that the length of the submatrix is ​​greater than its width, wherein the length direction of the submatrix corresponds to the longitude direction of the sampling point, and the width direction of the submatrix corresponds to the latitude direction of the sampling point; In response to determining the degree of change between adjacent spatial images in the longitude direction in the spatial image matrix, which is negatively correlated with latitude values, the length of the submatrix is ​​determined to be positively correlated with latitude values; Based on the fact that the length of the submatrix is ​​greater than its width, and the positive correlation between the length of the submatrix and the latitude value, the target size of the submatrix in different regions of the spatial image matrix is ​​determined.

3. The method according to claim 1, wherein, The step of generating a spatial image matrix, comprising multiple spatial images collected from multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles, includes: The spatial images are arranged with the longitude of the sampling points corresponding to the spatial images as the horizontal axis and the latitude of the sampling points corresponding to the spatial images as the vertical axis to generate the spatial image matrix.

4. The method according to claim 1, wherein, The step of encoding the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the plurality of image groups to generate an encoded file includes: Arrange the image groups corresponding to the sampling points of the image groups in ascending order of longitude, and generate multiple image group subsequences. Arrange the multiple image group sub-sequences according to the latitude of the sampling points corresponding to the image group in ascending order, determine the image group identifier corresponding to each of the multiple image groups, and generate the image group sequence. Arrange the spatial images in each image group in the image group sequence according to the order of keyframes first and prediction frames second, determine the image identifier of the spatial images in each image group in the multiple image groups, and generate a spatial image sequence. The spatial image sequence is encoded based on the keyframes and prediction frames included in each of the multiple image groups to generate the encoded file.

5. The method according to claim 4, wherein, The step of arranging the spatial images in each image group of the image group sequence according to the order of keyframes first and prediction frames second, determining the image identifier of the spatial images in each of the multiple image groups, and generating a spatial image sequence includes: The spatial image at the center position of the sub-matrix corresponding to each of the multiple image groups is determined as the initial keyframe. For each of the plurality of image groups, based on the initial keyframes of the image group, the keyframes of the image group are determined from the direction where the sampling points corresponding to the spatial images in the image group are sparser, and the predicted frames adjacent to the keyframes are determined. Arrange the spatial images in each image group of the image group sequence according to the order of keyframes first and prediction frames second, determine the image identifier of the spatial images in each image group of the multiple image groups, and generate the spatial image sequence.

6. The method according to claim 4 or 5, wherein, The step of encoding the spatial image sequence based on the keyframes and prediction frames included in each of the plurality of image groups to generate the encoded file includes: For each of the plurality of image groups, the spatial image sequence is encoded by using a reference method in which the predicted frame in the image group uniquely references the keyframe in the image group, thereby generating the encoded file.

7. An encoding device suitable for spatial images, comprising: The first generation unit is configured to generate a spatial image matrix including multiple spatial images collected from the multiple sampling points based on the latitude and longitude corresponding to multiple sampling points at different spatial angles. The determining unit is configured to determine the target size of the submatrix corresponding to different regions in the spatial image matrix based on the negative correlation between the degree of change between adjacent spatial images in the spatial image matrix and the target size of the submatrix. The degree of change between adjacent spatial images represents the degree of change between the image content represented by the adjacent spatial images or the degree of change in position between adjacent sampling points corresponding to each adjacent spatial image. The partitioning unit is configured to divide the spatial image matrix into multiple image groups by sub-matrices corresponding to different regions in the spatial image matrix. The second generation unit is configured to encode the spatial images in the spatial image matrix based on the keyframes and prediction frames included in each of the plurality of image groups, and generate an encoded file.

8. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

9. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

Images