Image processing method, device, system, storage medium and program product

Abstract

This application provides an image processing method, device, system, storage medium, and program product. The processing method includes acquiring an image to be encoded and acquiring splitting parameters and encoding parameters. The splitting parameters are used to split the image to be encoded into multiple sub-images. The encoding parameters include encoding types corresponding to the multiple sub-images, including hard encoding and / or soft encoding. Based on the splitting parameters, the image to be encoded is split to obtain multiple first images. Based on the multiple encoding types, the multiple first images are encoded to obtain multiple images to be decoded. The multiple images to be decoded are then sent to a client device. There is a one-to-one correspondence between the multiple first images and the multiple images to be decoded. The image processing method provided by this application can reduce latency and stuttering, improving the user experience.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, device, system, storage medium, and program product. Background Technology

[0002] To efficiently transmit and store video data across different devices and platforms, video streams are typically encoded or decoded.

[0003] In related technologies, the virtual reality (VR) server and client typically perform hard or soft encoding on the image to be processed on the server side, and send the encoded video stream to the client. The client then performs hard or soft decoding on the encoded video stream and displays the decoded video stream.

[0004] However, in the process of realizing this application, the inventors discovered that the prior art has at least the following problems: in the above-mentioned methods, when the image resolution is high and the working efficiency of the encoding and decoding equipment is not high, delays and stuttering often occur, which reduces the user experience. Summary of the Invention

[0005] This application provides an image processing method, device, system, storage medium, and program product to reduce latency and stuttering, and improve user experience.

[0006] In a first aspect, embodiments of this application provide an image processing method applied to a server device, comprising:

[0007] The process involves acquiring an image to be encoded, as well as acquiring splitting parameters and encoding parameters. The splitting parameters are used to split the image to be encoded into multiple sub-images. The encoding parameters include the encoding types corresponding to the multiple sub-images. The encoding types include hard encoding and / or soft encoding.

[0008] The image to be encoded is split according to the splitting parameters to obtain multiple first images;

[0009] The plurality of first images are encoded according to the plurality of encoding types to obtain a plurality of images to be decoded, and the plurality of images to be decoded are sent to the client device; the plurality of first images and the plurality of images to be decoded have a one-to-one correspondence.

[0010] In one possible design, the splitting parameters include at least one set of resolutions and a preset number corresponding to each set of resolutions; each set of resolutions has the same resolution value, and different sets of resolutions have different resolution values; the step of splitting the image to be encoded according to the splitting parameters to obtain multiple first images includes:

[0011] The image to be encoded is split according to the at least one set of resolutions and the preset number corresponding to each of the at least one set of resolutions to obtain multiple first images; the multiple first images include the preset number of images corresponding to each of the resolutions.

[0012] In one possible design, encoding the plurality of first images according to the plurality of encoding types to obtain a plurality of images to be decoded includes:

[0013] If among the plurality of first images there is an image to be hard-coded with a corresponding encoding type of hard encoding, and there is an image to be soft-coded with a corresponding encoding type of soft encoding, then the hard encoding of the image to be hard-coded and the soft encoding of the image to be soft-coded will be performed in parallel to obtain a plurality of images to be decoded.

[0014] In one possible design, the encoding parameters also include the number of hard-encoding paths supported by the server device; the step of performing hard encoding of the image to be hard-encoded and soft encoding of the image to be soft-encoded in parallel to obtain multiple images to be decoded includes:

[0015] If the number of hard-coding paths is multiple, then when hard-coding the image to be hard-coded, the image to be hard-coded is divided into multiple paths and hard-coded in parallel according to the number of hard-coding paths.

[0016] In one possible design, encoding the plurality of first images according to the plurality of encoding types includes:

[0017] Get the encoding format;

[0018] The plurality of first images are encoded according to the plurality of encoding types and the plurality of encoding formats.

[0019] In one possible design, prior to acquiring the image to be encoded and acquiring the splitting and encoding parameters, the process further includes:

[0020] Obtain image combination information; the image combination information includes multiple image groups; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters;

[0021] For each image group in the image combination information, multiple decoding times corresponding to the image group are obtained; each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types;

[0022] For each image subgroup in the image combination information, multiple encoding times corresponding to the image group are obtained; each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types;

[0023] The splitting parameter and the encoding parameter are determined based on the multiple decoding times and the multiple encoding times.

[0024] Secondly, embodiments of this application provide an image processing method applied to a client device, the method comprising:

[0025] The server acquires multiple images to be decoded sent by the server device; the multiple images to be decoded are obtained by the server device based on splitting parameters and encoding parameters; the splitting parameters are used to split the images to be encoded into multiple sub-images; the encoding parameters include the encoding types corresponding to the multiple sub-images respectively.

[0026] Obtain decoding parameters and splicing parameters; the decoding parameters include the decoding types corresponding to the multiple sub-images respectively; the splicing parameters are used to characterize the relative positional relationship of the multiple sub-images;

[0027] According to the multiple decoding types, the multiple images to be decoded are decoded to obtain multiple second images;

[0028] The multiple second images are stitched together according to the stitching parameters to obtain the target image.

[0029] In one possible design, the decoding type includes hardware decoding or software decoding; the step of decoding the plurality of images to be decoded according to the plurality of decoding types to obtain a plurality of second images includes:

[0030] If among the plurality of images to be decoded there is an image to be decoded with a corresponding decoding type of hardware decoding, and there is an image to be decoded with a corresponding decoding type of software decoding, then the hardware decoding of the image to be decoded and the software decoding of the image to be decoded will be performed in parallel to obtain a plurality of second images.

[0031] In one possible design, the decoding parameters also include the number of hardware decoding paths supported by the client device; the parallel processing of hardware decoding of the image to be hardware decoded and software decoding of the image to be software decoded to obtain multiple second images includes:

[0032] If the number of hardware decoding paths is multiple, then when hardware decoding the image to be hardware decoded, the image to be hardware decoded is divided into multiple paths for parallel hardware decoding according to the number of hardware decoding paths.

[0033] In one possible design, the stitching parameters include the position information of each sub-image on the image to be encoded; according to the stitching parameters, the plurality of second images are stitched together to obtain the target image, including:

[0034] Based on the position information of the sub-images corresponding to the plurality of second images on the image to be encoded, the plurality of second images are stitched together to obtain the target image.

[0035] In one possible design, before obtaining the decoding parameters and concatenation parameters, the following steps are also included:

[0036] Obtain image combination information; the image combination information includes multiple image groups; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters;

[0037] For each image group in the image combination information, multiple decoding times corresponding to the image group are obtained; each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types;

[0038] For each image subgroup in the image combination information, multiple encoding times corresponding to the image group are obtained; each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types;

[0039] The splicing parameters and the decoding parameters are determined based on the multiple decoding times and the multiple encoding times.

[0040] Thirdly, embodiments of this application provide a method for determining image encoding / decoding parameters, including:

[0041] Obtain image combination information; the image combination information includes multiple image groups; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters;

[0042] For each image group in the image combination information, multiple decoding times corresponding to the image group are obtained; each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types;

[0043] For each image subgroup in the image combination information, multiple encoding times corresponding to the image group are obtained; each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types;

[0044] Based on the multiple decoding times and multiple encoding times, target splitting parameters, target stitching parameters, target decoding parameters, and target encoding parameters are determined; so that the server device splits and encodes the image to be encoded according to the target splitting parameters and the target decoding parameters to obtain the image to be decoded, and the client device decodes and stitches the image to be decoded according to the target decoding parameters and the target stitching parameters to obtain the target image.

[0045] In one possible design, determining the target splitting parameter, target concatenation parameter, target decoding parameter, and target encoding parameter based on the plurality of decoding times and the plurality of encoding times includes:

[0046] Based on the multiple decoding times and multiple encoding times, the sum of the decoding time and encoding time for each image group in each encoding mode and each decoding mode is determined, and the image group corresponding to the minimum sum is determined as the target image group, the decoding mode corresponding to the minimum sum is determined as the target decoding mode, and the encoding mode corresponding to the minimum sum is determined as the target encoding mode.

[0047] The target splitting parameters and target stitching parameters are determined based on the target image grouping, the target decoding parameters are determined based on the target decoding mode, and the target encoding parameters are determined based on the target encoding mode.

[0048] In one possible design, applied to a server-side device, the step of obtaining multiple decoding times corresponding to each image group in the image combination information includes:

[0049] The image combination information is sent to the client device, so that the client device decodes the images in each image group based on the multiple decoding modes for each image group in the image combination information, and obtains multiple decoding times corresponding to the image group;

[0050] The client device receives multiple decoding times corresponding to the multiple image groups sent by the client device.

[0051] In one possible design, applied to a server-side device, the step of obtaining multiple decoding times corresponding to each image group in the image combination information includes:

[0052] The image combination information is sent to the server device, so that the server device encodes the images in each image group based on the multiple encoding modes for each image group in the image combination information, thereby obtaining multiple encoding times corresponding to the image group;

[0053] The time taken to encode the multiple image groups sent by the server device.

[0054] Fourthly, embodiments of this application provide a server-side device, including:

[0055] The first acquisition module is used to acquire the image to be encoded and to acquire splitting parameters and encoding parameters; the splitting parameters are used to split the image to be encoded into multiple sub-images; the encoding parameters include the encoding types corresponding to the multiple sub-images respectively; the encoding types include hard encoding and / or soft encoding.

[0056] The splitting module is used to split the image to be encoded according to the splitting parameters to obtain multiple first images;

[0057] The encoding module is used to encode the plurality of first images according to the plurality of encoding types to obtain a plurality of images to be decoded, and to send the plurality of images to be decoded to the client device; the plurality of first images and the plurality of images to be decoded have a one-to-one correspondence.

[0058] Fifthly, embodiments of this application provide a client device, including:

[0059] A receiving module is used to acquire multiple images to be decoded sent by a server device; the multiple images to be decoded are obtained by the server device based on splitting parameters and encoding parameters; the splitting parameters are used to split the images to be encoded into multiple sub-images; the encoding parameters include the encoding types corresponding to the multiple sub-images respectively;

[0060] The second acquisition module is used to acquire decoding parameters and splicing parameters; the decoding parameters include the decoding types corresponding to the plurality of sub-images respectively; the splicing parameters are used to characterize the relative positional relationship of the plurality of sub-images;

[0061] A decoding module is used to decode the plurality of images to be decoded according to the plurality of decoding types to obtain a plurality of second images;

[0062] The stitching module is used to stitch the plurality of second images according to the stitching parameters to obtain the target image.

[0063] Sixthly, embodiments of this application provide an image processing parameter determination device, comprising:

[0064] The third acquisition module is used to acquire image combination information; the image combination information includes multiple image groups; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters;

[0065] The fourth acquisition module is used to acquire multiple decoding times corresponding to each image group in the image combination information; each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types.

[0066] The fifth acquisition module is used to acquire multiple encoding times corresponding to each image subgroup in the image combination information; each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types;

[0067] A determining module is used to determine target splitting parameters, target stitching parameters, target decoding parameters, and target encoding parameters based on multiple decoding times and multiple encoding times; so that the server device splits and encodes the image to be encoded according to the target splitting parameters and the target decoding parameters to obtain an image to be decoded, and the client device decodes and stitches the image to be decoded according to the target decoding parameters and the target stitching parameters to obtain a target image.

[0068] In a seventh aspect, embodiments of this application provide a server-side device, including: at least one processor and a memory;

[0069] The memory stores computer-executed instructions;

[0070] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method described in the first aspect above and various possible designs of the first aspect.

[0071] Eighthly, embodiments of this application provide a client device, including: at least one processor and a memory;

[0072] The memory stores computer-executed instructions;

[0073] The at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the method described in the second aspect above and various possible designs of the second aspect.

[0074] In a ninth aspect, embodiments of this application provide an image processing system, including: a server device as described in the seventh aspect and a client device as described in the eighth aspect.

[0075] In a tenth aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the methods described in the first, second, and third aspects above, as well as various possible designs of the first, second, and third aspects.

[0076] In the eleventh aspect, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements the methods described in the first, second, and third aspects above, as well as various possible designs of the first, second, and third aspects.

[0077] This embodiment provides an image processing method, device, system, storage medium, and program product. The method includes acquiring an image to be encoded and acquiring splitting parameters and encoding parameters. The splitting parameters are used to split the image to be encoded into multiple sub-images. The encoding parameters include encoding types corresponding to the multiple sub-images, including hard encoding and / or soft encoding. According to the splitting parameters, the image to be encoded is split to obtain multiple first images. According to the multiple encoding types, the multiple first images are encoded to obtain multiple images to be decoded, and the multiple images to be decoded are sent to a client device. The multiple first images and the multiple images to be decoded have a one-to-one correspondence. The image processing method provided in this application embodiment, by splitting images in a video stream and using hard encoding for some of the split images while using soft encoding for others, can complete the encoding of the video stream through a combination of hard and soft encoding. This achieves full utilization of encoding device resources, reduces latency and stuttering, and improves the user experience. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a schematic diagram illustrating the image processing method and its application scenarios provided in the embodiments of this application;

[0080] Figure 2 A flowchart illustrating the image processing parameter determination method provided in this application embodiment;

[0081] Figure 3 This is a schematic diagram illustrating an application scenario of the image processing parameter determination method provided in the embodiments of this application;

[0082] Figure 4 A schematic diagram of image segmentation provided for an embodiment of this application;

[0083] Figure 5 A schematic flowchart illustrating the image processing method provided in an embodiment of this application;

[0084] Figure 6 A schematic flowchart illustrating the image processing method provided in an embodiment of this application;

[0085] Figure 7 This is a schematic diagram of the structure of the server device provided in the embodiments of this application;

[0086] Figure 8 This is a schematic diagram of the structure of the client device provided in the embodiments of this application;

[0087] Figure 9 This is a schematic diagram of the image processing parameter determination device provided in the embodiments of this application;

[0088] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0089] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0090] It should be noted that the image processing methods, devices, systems, storage media, and program products provided in this application can be used in the field of image processing, or in any field other than image processing. The application fields of the image processing methods, devices, systems, storage media, and program products provided in this application are not limited.

[0091] To address the aforementioned technical problems causing latency and stuttering, the inventors of this application have discovered that images in a video stream can be split into parts, with one part being hard-coded and the other part being software-coded. Then, on the client side, some of the encoded images are hard-decoded while the other part is software-decoded. This allows for the combined use of hardware and software encoding / decoding to fully utilize encoding / decoding equipment resources, reduce latency and stuttering, and improve the user experience.

[0092] Figure 1 This diagram illustrates the image processing method and its application scenarios provided in the embodiments of this application. Figure 1 As shown, server device 101 and client device 102 are connected via a network. Server device 101 can be a computer, tablet, server, cloud server, or other similar device, while client device can be a VR device, tablet, smartwatch, mobile phone, computer, or other similar device.

[0093] In the specific implementation process, the server device 101 acquires the image to be encoded, as well as the splitting parameters and encoding parameters. The splitting parameters are used to split the image to be encoded into multiple sub-images. The encoding parameters include the encoding types corresponding to the multiple sub-images, including hard encoding and / or soft encoding. According to the splitting parameters, the image to be encoded is split to obtain multiple first images. According to the multiple encoding types, the multiple first images are encoded to obtain multiple images to be decoded. The multiple images to be decoded are then sent to the client device 102. The multiple first images and the multiple images to be decoded have a one-to-one correspondence. Client device 102 receives multiple images to be decoded sent by server device 101. These multiple images are obtained based on splitting parameters and encoding parameters. The splitting parameters are used to split the images to be decoded into multiple sub-images. The encoding parameters include the encoding types corresponding to each of the multiple sub-images. Decoding parameters and splicing parameters are obtained. The decoding parameters include the decoding types corresponding to each of the multiple sub-images. The splicing parameters characterize the relative positional relationship of the multiple sub-images. Based on the multiple decoding types, the multiple images to be decoded are decoded to obtain multiple second images. Based on the splicing parameters, the multiple second images are spliced ​​to obtain the target image. The processing and decoding methods provided in this application split the images in the video stream, using hard encoding for some of the split images and soft encoding for others. Then, on the client side, some of the encoded images are hard decoded, while others are soft decoded. Therefore, video stream encoding and decoding can be completed through a combination of hard and soft encoding / decoding, fully utilizing encoding / decoding device resources, reducing latency and stuttering, and improving the user experience.

[0094] It should be noted that, Figure 1 The schematic diagram shown is merely an example. The encoding and decoding methods and scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of the system and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0095] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0096] Figure 2 This is a flowchart illustrating the image processing parameter determination method provided in an embodiment of this application. Figure 2 As shown, the method includes:

[0097] 201. Obtain image combination information; the image combination information includes multiple image groups; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters.

[0098] The execution subject of this embodiment can be any electronic device with data processing capabilities, such as a server device or a client device, or some steps can be executed by the server device and some steps can be executed by the client device.

[0099] The encoding parameters, decoding parameters, splicing parameters, and splitting parameters determined by the image processing parameter determination method provided in this embodiment can be applied to the image processing methods provided in the following embodiments, for example... Figure 5 The illustrated embodiments and Figure 6 The illustrated embodiment.

[0100] In practical applications, such as Figure 3 As shown, image combination information can be stored in the server device. When it is necessary to determine image encoding / decoding parameters, i.e., when a self-test is required, the client can send a request to the server to establish a connection. After the server responds to the request, the connection between the client and the server is completed. Once the connection is established, the server device can pass the pre-stored image combination information as parameters to the client device. The image combination information includes multiple image groups, and each image group can include the encoding format, the number of images, the resolution of each image, and the encoded image.

[0101] Different image groups are obtained by splitting the original image in different ways. For example, based on the total resolution of cloud VR stereo rendering, the image can be split into different resolution combinations, and the sum of the resolutions of the split sub-images is equal to the total resolution of the original image. For example, if the total image resolution is 4096x2048, then after two splitting methods, we can obtain image group A and image group B. Image group A includes two 2048x2048 images, image group B includes four 1024x512 images, and group C includes two 1024x1024 images and one 2048x2048 image.

[0102] 202. For each image group in the image combination information, obtain multiple decoding times corresponding to the image group; each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types.

[0103] In this embodiment, the decoding type can include hardware decoding or software decoding. The application ratio of hardware decoding and software decoding varies in different decoding modes. For example, if an image group includes four sub-images, a, b, c, and d, then decoding mode 1 could be hardware decoding of a and software decoding of b, c, and d; or decoding mode 2 could be hardware decoding of a and b and software decoding of c and d.

[0104] Specifically, continue as follows Figure 3 As shown, the client includes a client self-test module. This module uses the client device's dedicated SDK interface to check if the hardware supports hardware decoding. If supported, it further obtains the number of supported hardware decoding paths. Based on the image combination information, each image combination undergoes parallel decoding detection: full software decoding, full hardware decoding, and a mixture of software and hardware decoding at different ratios. The time consumption for each decoding mode is calculated, and the calculated time consumption is sent back to the server as a parameter.

[0105] For parallel decoding modes that combine software and hardware decoding, an example is: for such... Figure 4The image group D shown contains two 2048x1024 images and one 2048x2048 image, which can be assigned four decoding modes. If hardware decoding is supported, the first mode uses a software decoder for one 2048x1024 image and a hardware decoder for one 2048x1024 and one 2048x2048 image (if the number of hardware decoding paths is >= 2, the two paths are processed in parallel; otherwise, they are processed sequentially), with both software and hardware decoders operating in parallel. The second mode uses a software decoder for one 2048x2048 image and a hardware decoder for both 2048x1024 images. The third mode uses a software decoder for both 2048x1024 images and a hardware decoder for one 2048x2048 image. The fourth mode uses a software decoder for both 2048x1024 and one 2048x2048 image (two images are processed in parallel using multiple threads) and a hardware decoder for one 2048x1024 image. For images with the same resolution, such as two 2048x1024 images x and y, it is possible to perform software decoding on x and hardware decoding on y by default, or to perform hardware decoding on x and software decoding on y. The time consumption of these two forms is the same, that is, only the impact of resolution on the encoding and decoding time is considered, while the impact of image content on the encoding and decoding time is ignored, thereby reducing the amount of computation.

[0106] In some embodiments, the encoding / decoding format can be fixed to the H.265 standard. If multiple encoding / decoding formats are available, the time consumption under different encoding / decoding format conditions can be detected for each image combination to further select the encoding / decoding format based on the time consumption. For example, as shown in Table 1, the decoding times T1 to T5 for different image groups (a and b) under different decoding modes (1 and 2) can be obtained.

[0107] Table 1

[0108] Encoding / decoding formats Image grouping Decoding mode Decoding time m a 1 T1 m a 2 T2 m b 1 T3 m b 2 T4 n a 1 T5

[0109] 203. For each image subgroup in the image combination information, obtain multiple encoding times corresponding to the image group; each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types.

[0110] The encoding type can be hard encoding, soft encoding, or a combination of both.

[0111] In this embodiment, continue as follows Figure 3As shown, the server includes a server self-test module. The server device uses this module, based on its dedicated SDK interface, to detect whether the hardware supports hard coding and the number of supported hard coding paths. Based on the image combination information, parallel coding detection is performed on each image group, including all soft coding, all hard coding, and a mixture of soft and hard coding in different proportions. The time consumption for each coding mode is calculated. As shown in Table 2, assuming the coding format is m, the decoding times for different image groups under different coding modes can be obtained. It should be noted that Table 2 only shows a portion; it may also include the decoding times for other combinations, such as the decoding time T10 corresponding to coding mode 3 for image group a with coding format m.

[0112] Table 2

[0113] Encoding / decoding formats Image grouping Encoding mode Encoding time m a 1 T6 m a 2 T7 m b 1 T8 m b 2 T9

[0114] 204. Based on the multiple decoding times and multiple encoding times, determine the target splitting parameters, target stitching parameters, target decoding parameters, and target encoding parameters; so that the server device splits and encodes the image to be encoded according to the target splitting parameters and the target decoding parameters to obtain the image to be decoded, and the client device decodes and stitches the image to be decoded according to the target decoding parameters and the target stitching parameters to obtain the target image.

[0115] In some embodiments, determining the target splitting parameter, target concatenation parameter, target decoding parameter, and target encoding parameter based on a plurality of decoding times and a plurality of encoding times includes:

[0116] Based on the multiple decoding times and multiple encoding times, the sum of the decoding time and encoding time for each image group in each encoding mode and each decoding mode is determined, and the image group corresponding to the minimum sum is determined as the target image group, the decoding mode corresponding to the minimum sum is determined as the target decoding mode, and the encoding mode corresponding to the minimum sum is determined as the target encoding mode.

[0117] The target splitting parameters and target stitching parameters are determined based on the target image grouping, the target decoding parameters are determined based on the target decoding mode, and the target encoding parameters are determined based on the target encoding mode.

[0118] Specifically, after determining the multiple decoding times of different image groups under different decoding modes and the multiple encoding times of different image groups under different encoding modes, the decoding time and encoding time corresponding to each combination of image group and encoding / decoding mode can be added together to obtain the total time corresponding to that combination. This yields multiple total times, allowing the image group with the minimum total time to be identified as the target image group, the encoding mode as the target encoding mode, and the decoding mode as the target decoding mode. Then, encoding / decoding parameters, i.e., self-checking parameters, are determined based on the target image group, target encoding mode, and target decoding mode. These self-checking parameters can include splitting parameters (e.g., resolution of each image, number of images), stitching parameters (e.g., the coordinate position of the top left corner of each image on the original image), encoding parameters (e.g., the encoding type of each image (hard encoding, soft encoding, or both)), and decoding parameters (e.g., the decoding type of each image).

[0119] For example, by combining the data in Table 1 and Table 2, we can obtain Table 3. As shown in Table 3, we can calculate the total time corresponding to the combination of each image group, encoding mode, and decoding mode. Assuming that the sum of T4 and T9 is the smallest, then we can determine image group b as the target image group, decoding mode 2 as the target decoding mode, and encoding mode 2 as the target encoding mode.

[0120] Table 3

[0121] Encoding / decoding formats Image grouping Decoding mode Encoding mode Decoding time Encoding time Total time m a 1 1 T1 T6 T1+T6 m a 2 2 T2 T7 T2+T7 m b 1 1 T3 T8 T3+T8 m b 2 2 T4 T9 T4+T9

[0122] In some embodiments, the method can be applied to a server device, wherein obtaining multiple decoding times corresponding to each image group in the image combination information may include: sending the image combination information to the client device, so that the client device decodes the images in the image group based on the multiple decoding modes for each image group in the image combination information, thereby obtaining multiple decoding times corresponding to the image group;

[0123] The client device receives multiple decoding times corresponding to the multiple image groups sent by the client device.

[0124] In some embodiments, the method can be applied to a server device, wherein obtaining multiple decoding times corresponding to each image group in the image combination information may include: sending the image combination information to the server device, so that the server device encodes the images in the image group according to the multiple encoding modes for each image group in the image combination information, thereby obtaining multiple encoding times corresponding to the image group; and receiving the multiple encoding times corresponding to the multiple image groups sent by the server device.

[0125] The image processing parameter determination method provided in this application embodiment can fully consider the hardware conditions of server-side devices and client-side devices. Based on the detection results of the self-detection module, it manages the encoding and decoding of video streams through a combination of software encoding and decoding and hardware encoding and decoding, thereby making full use of encoding device resources, reducing latency and stuttering, maximizing compatibility with the hardware conditions of server-side devices and client-side devices, and reducing the requirements and dependence on hardware devices.

[0126] Figure 5 This is a schematic flowchart illustrating the image processing method provided in an embodiment of this application. Figure 5 As shown, the method includes:

[0127] 501. Obtain the image to be encoded and obtain the splitting parameters and encoding parameters; the splitting parameters are used to split the image to be encoded into multiple sub-images; the encoding parameters include the encoding types corresponding to the multiple sub-images respectively; the encoding types include hard encoding and / or soft encoding.

[0128] The execution entity in this embodiment is the server-side device.

[0129] Specifically, the method for determining the splitting parameters and encoding parameters can adopt the image processing parameter determination method described in the above embodiments, such as... Figure 2 The example shown.

[0130] The encoding type can be hard encoding, soft encoding, or a combination of both.

[0131] The splitting parameters and encoding parameters can be pre-stored on the server-side device after they are determined, or they can be pre-stored in other devices such as a database and retrieved from the database after the server-side device starts. This embodiment does not limit this.

[0132] 502. According to the splitting parameters, the image to be encoded is split to obtain multiple first images.

[0133] The splitting parameters can take various forms, and in one feasible implementation, they can include multiple resolutions and the number corresponding to different resolutions. For example, such as... Figure 4As shown, the total resolution is 4096x2048. Therefore, the splitting parameters can include two split resolutions: 2048x1024 and 2048x2048, and also include the number of each resolution: 2 for 2048x1024 and 1 for 2048x2048. In another possible implementation, the splitting parameters can include multiple coordinate values, for example, such as... Figure 4 As shown, when splitting an image into three sub-images, the coordinates of the four corners of each sub-image can be recorded. In one possible implementation, the splitting parameters can include bounding boxes, such as... Figure 4 As shown, the bounding box formed by dashed lines can be enlarged or reduced as the total resolution of the image changes. This allows image segmentation to be performed based on the bounding box regardless of the total resolution of the image to be segmented.

[0134] In some embodiments, the splitting parameters include at least one set of resolutions and a preset number corresponding to each set of resolutions; each set of resolutions has the same resolution value, and different sets of resolutions have different resolution values; splitting the image to be encoded according to the splitting parameters to obtain multiple first images may include: splitting the image to be encoded according to the at least one set of resolutions and the preset number corresponding to each of the at least one set of resolutions to obtain multiple first images; the multiple first images include the preset number of images corresponding to each of the resolutions.

[0135] 503. Encode the plurality of first images according to the plurality of encoding types to obtain a plurality of images to be decoded, and send the plurality of images to be decoded to the client device; the plurality of first images and the plurality of images to be decoded have a one-to-one correspondence.

[0136] Specifically, after the self-test is completed, i.e., after the image processing parameters are determined, the self-test parameters are obtained (which may include splitting parameters (e.g., resolution of each image, number of images), stitching parameters (e.g., the coordinates of the top left corner of each image on the original image), encoding parameters (e.g., encoding type of each image), and decoding parameters (e.g., decoding type of each image)). The server device begins rendering the cloud VR and obtains the first frame image (e.g., a stereo image), and then passes the image to its own image processing module. Based on the self-test parameters (i.e., the determined encoding parameters, splitting parameters, etc.), the image processing module splits the first frame image to obtain new images corresponding to the resolution and number of images in the self-test parameters. Then, according to the allocation method of hard encoding and soft encoding indicated by the encoding mode in the self-test parameters (e.g., which images are hard encoded and which images are soft encoded), the new images are submitted to the hard and soft encoding modules for parallel encoding (parallel encoding here includes both soft and hard encoding being performed simultaneously). Finally, the encoded new image data is packaged and sent to the terminal. Optionally, the data package may include self-test parameters (which may be excluded since they are not used during decoding) and each encoded image.

[0137] In some embodiments, the encoding type includes hard encoding and / or soft encoding. Encoding the plurality of first images according to the plurality of encoding types to obtain a plurality of images to be decoded may include: if there is an image to be hard encoded with a corresponding encoding type of hard encoding and an image to be soft encoded with a corresponding encoding type of soft encoding among the plurality of first images, then the hard encoding of the image to be hard encoded and the soft encoding of the image to be soft encoded are performed in parallel to obtain a plurality of images to be decoded.

[0138] Specifically, the multiple first images correspond to encoding types allocated based on encoding parameters. For example, all of the multiple first images can be hard-coded, all can be soft-coded, or some first images can be hard-coded and some first images can be soft-coded.

[0139] In some embodiments, to improve encoding efficiency, multiple hard encoding paths can be performed in parallel. The encoding parameters also include the number of hard encoding paths supported by the server device. The step of performing hard encoding of the image to be hard encoded and soft encoding of the image to be soft encoded in parallel to obtain multiple images to be decoded includes: if the number of hard encoding paths is multiple, then when hard encoding the image to be hard encoded, the image to be hard encoded is divided into multiple paths and hard encoded in parallel according to the number of hard encoding paths.

[0140] In some embodiments, encoding the plurality of first images according to the plurality of encoding types may include: obtaining an encoding format; and encoding the plurality of first images according to the plurality of encoding types and the encoding format.

[0141] The image processing method provided in this embodiment splits the images in the video stream, and uses hard encoding for one part of the split images and soft encoding for the other part. It can complete the encoding of the video stream by combining hard and soft encoding, so as to make full use of the encoding device resources, reduce latency and stuttering, and improve the user experience.

[0142] Figure 6 This is a schematic flowchart illustrating the image processing method provided in an embodiment of this application. Figure 6 As shown, the method includes:

[0143] 601. Obtain multiple images to be decoded sent by the server device; the multiple images to be decoded are obtained by the server device based on splitting parameters and encoding parameters; the splitting parameters are used to split the images to be encoded into multiple sub-images; the encoding parameters include the encoding types corresponding to the multiple sub-images respectively.

[0144] The execution entity in this embodiment can be a client device.

[0145] Specifically, the server device completes the splitting and encoding of the image to be encoded based on the splitting and encoding parameters in the self-test parameters (based on the encoding and decoding parameters determined in the above embodiments), and obtains multiple images to be decoded, and sends the multiple images to be decoded to the client device.

[0146] 602. Obtain decoding parameters and splicing parameters; the decoding parameters include the decoding types corresponding to the multiple sub-images respectively; the splicing parameters are used to characterize the relative positional relationship of the multiple sub-images.

[0147] Specifically, the decoding parameters and stitching parameters can be packaged together with multiple images to be decoded and sent to the client device by the server device, or they can be sent separately by the server device, or the decoding parameters and stitching parameters can be pre-stored on the client device.

[0148] 603. Decode the multiple images to be decoded according to the multiple decoding types to obtain multiple second images.

[0149] 604. According to the stitching parameters, the plurality of second images are stitched together to obtain the target image.

[0150] For example, the stitching parameters may include positional information of different sub-images relative to the original image, such as the coordinates of the top-left corner of a sub-image in the original image.

[0151] Specifically, based on the decoding parameters (resolution and decoding type description) in the server's self-test parameters, the client decodes each image in parallel using its corresponding decoding type (software decoding or hardware decoding). The decoded images and self-test parameters (including stitching parameters) are then passed to the client's image reconstruction module. The image reconstruction module, based on the stitching parameters in the self-test parameters (such as the coordinates of the top-left corner of each image on the original image), re-renders each decoded image onto the final target image. The client then uses the target image to display it on the screen.

[0152] In some embodiments, the decoding type includes hardware decoding or software decoding; the step of decoding the plurality of images to be decoded according to the plurality of decoding types to obtain a plurality of second images may include: if there is an image to be hardware decoded with a corresponding hardware decoding type and an image to be software decoded with a corresponding software decoding type among the plurality of images to be decoded, then the hardware decoding of the image to be hardware decoded and the software decoding of the image to be software decoded are performed in parallel to obtain a plurality of second images.

[0153] In some embodiments, to improve efficiency, multiple hardware decodings can be performed in parallel. The decoding parameters also include the number of hardware decoding paths supported by the client device. The step of performing hardware decoding on the image to be hardware decoded and software decoding on the image to be software decoded in parallel to obtain multiple second images may include: if the number of hardware decoding paths is multiple, then when performing hardware decoding on the image to be hardware decoded, the image to be hardware decoded is divided into multiple paths for parallel hardware decoding according to the number of hardware decoding paths.

[0154] In some embodiments, the stitching parameters include the position information of each sub-image on the image to be encoded; stitching the plurality of second images according to the stitching parameters to obtain a target image may include: stitching the plurality of second images according to the position information of the sub-images corresponding to the plurality of second images on the image to be encoded to obtain a target image.

[0155] The image processing method provided in this embodiment splits the images in the video stream, hard-encodes one part of the split images and soft-encodes the other part. Then, on the client side, hard-decodes one part of the encoded images and soft-decodes the other part. Thus, the encoding and decoding of the video stream can be completed by combining soft and hard encoding and decoding, so as to make full use of the encoding and decoding equipment resources, reduce latency and stuttering, and improve the user experience.

[0156] Figure 7 This is a schematic diagram of the structure of the server-side device provided in an embodiment of this application. Figure 7As shown, the server device 70 includes: a first acquisition module 701, a splitting module 702, and an encoding module 703.

[0157] The first acquisition module 701 is used to acquire an image to be encoded and to acquire splitting parameters and encoding parameters; the splitting parameters are used to split the image to be encoded into multiple sub-images; the encoding parameters include the encoding types corresponding to the multiple sub-images respectively; the encoding types include hard encoding and / or soft encoding.

[0158] The splitting module 702 is used to split the image to be encoded according to the splitting parameters to obtain multiple first images;

[0159] The encoding module 703 is used to encode the plurality of first images according to the plurality of encoding types to obtain a plurality of images to be decoded, and to send the plurality of images to be decoded to the client device; the plurality of first images and the plurality of images to be decoded have a one-to-one correspondence.

[0160] The server device provided in this application embodiment splits the images in the video stream, and uses hard encoding for one part of the split images and soft encoding for the other part. It can complete the encoding of the video stream by combining hard and soft encoding, so as to make full use of the encoding device resources, reduce latency and stuttering, and improve the user experience.

[0161] In some embodiments, the splitting module 702 is specifically used to: split the image to be encoded according to the at least one set of resolutions and the preset number corresponding to the at least one set of resolutions to obtain a plurality of first images; the plurality of first images include the preset number of images corresponding to each of the resolutions.

[0162] In some embodiments, the encoding type includes hard encoding and / or soft encoding. The encoding module 703 is specifically used to: if there is an image to be hard encoded with a corresponding encoding type of hard encoding and an image to be soft encoded with a corresponding encoding type of soft encoding among the plurality of first images, then the hard encoding of the image to be hard encoded and the soft encoding of the image to be soft encoded are performed in parallel to obtain a plurality of images to be decoded.

[0163] In some embodiments, the encoding parameters further include the number of hard-coding paths supported by the server device; the encoding module 703 is specifically used to: if the number of hard-coding paths is multiple, then when hard-coding the image to be hard-coded, the image to be hard-coded is divided into multiple paths for parallel hard-coding according to the number of hard-coding paths.

[0164] In some embodiments, the encoding module 703 is specifically used for: obtaining an encoding format; and encoding the plurality of first images according to the plurality of encoding types and the encoding format.

[0165] The server-side device provided in this application embodiment can be used to execute the above-described image processing method embodiment with the server-side device as the execution subject. Its implementation principle and technical effect are similar, and will not be repeated here.

[0166] Figure 8 This is a schematic diagram of the structure of a client device provided in an embodiment of this application. Figure 8 As shown, the client device 80 includes: a receiving module 801, a second acquisition module 802, a decoding module 803, and a splicing module 804.

[0167] The receiving module 801 is used to acquire multiple images to be decoded sent by the server device; the multiple images to be decoded are obtained by the server device based on splitting parameters and encoding parameters; the splitting parameters are used to split the images to be encoded into multiple sub-images; the encoding parameters include the encoding types corresponding to the multiple sub-images respectively;

[0168] The second acquisition module 802 is used to acquire decoding parameters and splicing parameters; the decoding parameters include the decoding types corresponding to the plurality of sub-images respectively; the splicing parameters are used to characterize the relative positional relationship of the plurality of sub-images;

[0169] The decoding module 803 is used to decode the plurality of images to be decoded according to the plurality of decoding types to obtain a plurality of second images;

[0170] The stitching module 804 is used to stitch the plurality of second images according to the stitching parameters to obtain the target image.

[0171] The server device provided in this application splits the images in the video stream, hard-encodes one part of the split images and soft-encodes the other part. Then, on the client side, hard-decodes one part of the encoded images and soft-decodes the other part. Thus, the encoding and decoding of the video stream can be completed by combining soft and hard encoding and decoding, so as to make full use of the encoding and decoding equipment resources, reduce latency and stuttering, and improve the user experience.

[0172] In some embodiments, the decoding type includes hardware decoding or software decoding; the decoding module 803 is specifically used to: if there is a hardware decoding image and a software decoding image among the plurality of images to be decoded, the hardware decoding of the hardware decoding image and the software decoding of the software decoding image will be performed in parallel to obtain a plurality of second images.

[0173] In some embodiments, the decoding parameters further include the number of hardware decoding paths supported by the client device; the decoding module 803 is specifically used to: if the number of hardware decoding paths is multiple, then when performing hardware decoding on the image to be hardware decoded, the image to be hardware decoded is divided into multiple paths for parallel hardware decoding according to the number of hardware decoding paths.

[0174] In some embodiments, the stitching parameters include the position information of each sub-image on the image to be encoded; the stitching module 804 is specifically used to: stitch the plurality of second images according to the position information of the sub-images corresponding to the plurality of second images on the image to be encoded, so as to obtain the target image.

[0175] The client device provided in this application embodiment can be used to execute the above-described image processing method embodiment with the client device as the execution subject. Its implementation principle and technical effect are similar, and will not be described again here.

[0176] Figure 9 This is a schematic diagram of the image processing parameter determination device provided in an embodiment of this application. Figure 9 As shown, the image encoding / decoding determination device 90 includes: a third acquisition module 901, a fourth acquisition module 902, a fifth acquisition module 903, and a determination module 904.

[0177] The third acquisition module 901 is used to acquire image combination information; the image combination information includes multiple image groups; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters.

[0178] The fourth acquisition module 902 is used to acquire multiple decoding times corresponding to each image group in the image combination information; each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types.

[0179] The fifth acquisition module 903 is used to acquire multiple encoding times corresponding to each image subgroup in the image combination information; each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types.

[0180] The determining module 904 is used to determine target splitting parameters, target stitching parameters, target decoding parameters, and target encoding parameters based on multiple decoding times and multiple encoding times; so that the server device splits and encodes the image to be encoded according to the target splitting parameters and the target decoding parameters to obtain the image to be decoded, and the client device decodes and stitches the image to be decoded according to the target decoding parameters and the target stitching parameters to obtain the target image.

[0181] The server-side device provided in this application embodiment can fully consider the hardware conditions of the server-side device and the client-side device. Based on the detection results of the self-detection module, it manages the encoding and decoding of the video stream through a combination of software encoding and decoding and hardware encoding and decoding. This achieves full utilization of encoding device resources, reduces latency and stuttering, maximizes compatibility with the hardware conditions of the server-side device and the client-side device, and reduces the requirements and dependence on hardware devices.

[0182] In some embodiments, the determining module 904 is specifically configured to: determine the sum of the decoding time and encoding time of each image group in each encoding mode and each decoding mode based on the plurality of decoding times and the plurality of encoding times; determine the image group corresponding to the minimum sum as the target image group, the decoding mode corresponding to the minimum sum as the target decoding mode, and the encoding mode corresponding to the minimum sum as the target encoding mode; determine target splitting parameters and target stitching parameters based on the target image group, determine target decoding parameters based on the target decoding mode, and determine target encoding parameters based on the target encoding mode.

[0183] In some embodiments, applied to a server device, the fourth acquisition module 902 is specifically used to: send the image combination information to the client device, so that the client device decodes the images in the image group according to the multiple decoding modes for each image group in the image combination information, and obtains multiple decoding times corresponding to the image group; and receive the multiple decoding times corresponding to the multiple image groups sent by the client device.

[0184] In some embodiments, applied to a server device, the fifth acquisition module 903 is specifically used to: send the image combination information to the server device, so that the server device encodes the images in the image group according to the multiple encoding modes for each image group in the image combination information, and obtains multiple encoding times corresponding to the image group; and receive the multiple encoding times corresponding to the multiple image groups sent by the server device.

[0185] The image processing parameter determination device provided in this application embodiment can be used to execute the above-described encoding / decoding parameter determination method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0186] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The device can be a server device or a client device, and can be a computer, VR device, tablet computer, mobile phone, server, etc.

[0187] Device 120 may include one or more of the following components: processing component 1201, memory 1202, power supply component 1203, multimedia component 1204, audio component 1205, input / output (I / O) interface 1206, sensor component 1207, and communication component 1208.

[0188] Processing component 1201 typically controls the overall operation of device 120, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 1201 may include one or more processors 1209 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1201 may include one or more modules to facilitate interaction between processing component 1201 and other components. For example, processing component 1201 may include a multimedia module to facilitate interaction between multimedia component 1204 and processing component 1201.

[0189] Memory 1202 is configured to store various types of data to support the operation of device 120. Examples of such data include instructions for any application or method operating on device 120, contact data, phonebook data, messages, pictures, videos, etc. Memory 1202 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0190] Power supply component 1203 provides power to various components of device 120. Power supply component 1203 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 120.

[0191] Multimedia component 1204 includes a screen that provides an output interface between the device 120 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1204 includes a front-facing camera and / or a rear-facing camera. When the device 120 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0192] Audio component 1205 is configured to output and / or input audio signals. For example, audio component 1205 includes a microphone (MIC) configured to receive external audio signals when device 120 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1202 or transmitted via communication component 1208. In some embodiments, audio component 1205 also includes a speaker for outputting audio signals.

[0193] I / O interface 1206 provides an interface between processing component 1201 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0194] Sensor assembly 1207 includes one or more sensors for providing status assessments of various aspects of device 120. For example, sensor assembly 1207 may detect the on / off state of device 120, the relative positioning of components such as the display and keypad of device 120, changes in the position of device 120 or a component of device 120, the presence or absence of user contact with device 120, the orientation or acceleration / deceleration of device 120, and temperature changes of device 120. Sensor assembly 1207 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1207 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1207 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0195] Communication component 1208 is configured to facilitate wired or wireless communication between device 120 and other devices. Device 120 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1208 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1208 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0196] In an exemplary embodiment, device 120 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0197] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1202 including instructions, which can be executed by a processor 1209 of device 120 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0198] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0199] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0200] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0201] This application also provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method performed by the image encoding device described above.

[0202] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, Applied to server-side devices, the method includes: The process involves acquiring an image to be encoded, as well as acquiring splitting parameters and encoding parameters. The splitting parameters are used to split the image to be encoded into multiple sub-images. The splitting parameters include at least one set of resolutions and a preset number of sub-images for each set of resolutions. Each set of resolutions has the same resolution value, and different sets of resolutions have different resolution values. The encoding parameters include the encoding types corresponding to the multiple sub-images. The encoding types include hard encoding and / or soft encoding. The splitting parameters and the encoding parameters are determined based on the encoding time and decoding time corresponding to the multiple image groups. Sub-images within different image groups are obtained by splitting the original image according to different splitting parameters. Each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode. Each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode. The image to be encoded is split according to the splitting parameters to obtain multiple first images; The plurality of first images are encoded according to multiple encoding types to obtain a plurality of images to be decoded, and the plurality of images to be decoded are sent to the client device; the plurality of first images and the plurality of images to be decoded have a one-to-one correspondence.

2. The method according to claim 1, characterized in that, The step of splitting the image to be encoded according to the splitting parameters to obtain multiple first images includes: The image to be encoded is split according to the at least one set of resolutions and the preset number corresponding to each of the at least one set of resolutions to obtain multiple first images; the multiple first images include the preset number of images corresponding to each of the resolutions.

3. The method according to claim 1, characterized in that, The step of encoding the plurality of first images according to the plurality of encoding types to obtain a plurality of images to be decoded includes: If among the plurality of first images there is an image to be hard-coded with a corresponding encoding type of hard encoding, and there is an image to be soft-coded with a corresponding encoding type of soft encoding, then the hard encoding of the image to be hard-coded and the soft encoding of the image to be soft-coded will be performed in parallel to obtain a plurality of images to be decoded.

4. The method according to claim 3, characterized in that, The encoding parameters also include the number of hard-encoding paths supported by the server device; the process of performing hard encoding on the image to be hard-encoded and soft encoding on the image to be soft-encoded in parallel to obtain multiple images to be decoded includes: If the number of hard-coding paths is multiple, then when hard-coding the image to be hard-coded, the image to be hard-coded is divided into multiple paths and hard-coded in parallel according to the number of hard-coding paths.

5. The method according to any one of claims 1-4, characterized in that, The step of encoding the plurality of first images according to the plurality of encoding types includes: Get the encoding format; The plurality of first images are encoded according to the plurality of encoding types and the plurality of encoding formats.

6. The method according to any one of claims 1-4, characterized in that, Before acquiring the image to be encoded and acquiring the splitting and encoding parameters, the process also includes: Obtain image combination information; the image combination information includes multiple image groups; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters; For each image group in the image combination information, multiple decoding times corresponding to the image group are obtained; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types; For each image group in the image combination information, obtain multiple encoding times corresponding to the image group; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types; The splitting parameter and the encoding parameter are determined based on the multiple decoding times and the multiple encoding times.

7. An image processing method, characterized in that, Applied to a client device, the method includes: The system acquires multiple images to be decoded sent by a server device. These images are obtained by the server device based on splitting parameters and encoding parameters. The splitting parameters are used to split the images to be encoded into multiple sub-images. The splitting parameters include at least one set of resolutions and a preset number of images corresponding to each set of resolutions. Each set of resolutions has the same resolution value, and different sets of resolutions have different resolution values. The encoding parameters include the encoding types corresponding to each of the multiple sub-images. The splitting parameters and the encoding parameters are determined based on the encoding time and decoding time corresponding to each of the multiple image groups. Sub-images within different image groups are obtained by splitting the original image according to different splitting parameters. Each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode. Each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode. Obtain decoding parameters and splicing parameters; the decoding parameters include the decoding types corresponding to the multiple sub-images respectively; the splicing parameters are used to characterize the relative positional relationship of the multiple sub-images; Based on multiple decoding types, the multiple images to be decoded are decoded to obtain multiple second images; The multiple second images are stitched together according to the stitching parameters to obtain the target image.

8. The method according to claim 7, characterized in that, The decoding type includes hardware decoding or software decoding; the process of decoding the multiple images to be decoded according to the multiple decoding types to obtain multiple second images includes: If among the plurality of images to be decoded there is an image to be decoded with a corresponding decoding type of hardware decoding, and there is an image to be decoded with a corresponding decoding type of software decoding, then the hardware decoding of the image to be decoded and the software decoding of the image to be decoded will be performed in parallel to obtain a plurality of second images.

9. The method according to claim 8, characterized in that, The decoding parameters also include the number of hardware decoding paths supported by the client device; the process of performing hardware decoding on the image to be hardware decoded and software decoding on the image to be software decoded in parallel to obtain multiple second images includes: If the number of hardware decoding paths is multiple, then when hardware decoding the image to be hardware decoded, the image to be hardware decoded is divided into multiple paths for parallel hardware decoding according to the number of hardware decoding paths.

10. The method according to any one of claims 7-9, characterized in that, The splicing parameters include the position information of each sub-image on the image to be encoded; according to the splicing parameters, the plurality of second images are spliced ​​to obtain the target image, including: Based on the position information of the sub-images corresponding to the plurality of second images on the image to be encoded, the plurality of second images are stitched together to obtain the target image.

11. The method according to any one of claims 7-9, characterized in that, Before obtaining the decoding parameters and concatenation parameters, the process also includes: Obtain image combination information; the image combination information includes multiple image groups; For each image group in the image combination information, multiple decoding times corresponding to the image group are obtained; each decoding time is determined by the client device decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types; For each image subgroup in the image combination information, multiple encoding times corresponding to the image group are obtained; each encoding time is determined by the server device encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types; The splicing parameters and the decoding parameters are determined based on the multiple decoding times and the multiple encoding times.

12. A method for determining image encoding / decoding parameters, characterized in that, include: Obtain image combination information; the image combination information includes multiple image groups; For each image group in the image combination information, multiple decoding times corresponding to the image group are obtained; each decoding time is determined by the client device in the image processing method as described in any one of claims 7-11 decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types; For each image subgroup in the image combination information, multiple encoding times corresponding to the image group are obtained; each encoding time is determined by the server device in the image processing method as described in any one of claims 1-6 encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types; Based on the multiple decoding times and multiple encoding times, target splitting parameters, target stitching parameters, target decoding parameters, and target encoding parameters are determined; so that the server device splits and encodes the image to be encoded according to the target splitting parameters and the target decoding parameters to obtain the image to be decoded, and the client device decodes and stitches the image to be decoded according to the target decoding parameters and the target stitching parameters to obtain the target image.

13. The method according to claim 12, characterized in that, The step of determining the target splitting parameters, target concatenation parameters, target decoding parameters, and target encoding parameters based on multiple decoding times and multiple encoding times includes: Based on the multiple decoding times and multiple encoding times, the sum of the decoding time and encoding time for each image group in each encoding mode and each decoding mode is determined, and the image group corresponding to the minimum sum is determined as the target image group, the decoding mode corresponding to the minimum sum is determined as the target decoding mode, and the encoding mode corresponding to the minimum sum is determined as the target encoding mode. The target splitting parameters and target stitching parameters are determined based on the target image grouping, the target decoding parameters are determined based on the target decoding mode, and the target encoding parameters are determined based on the target encoding mode.

14. The method according to claim 12 or 13, characterized in that, Applied to server-side devices, the step of obtaining multiple decoding times corresponding to each image group in the image combination information includes: The image combination information is sent to the client device, so that the client device decodes the images in each image group based on multiple decoding modes for each image group in the image combination information, and obtains multiple decoding times corresponding to the image group; The client device receives multiple decoding times corresponding to the multiple image groups sent by the client device.

15. The method according to claim 12 or 13, characterized in that, Applied to server-side devices, the step of obtaining multiple decoding times corresponding to each image group in the image combination information includes: The image combination information is sent to the server device, so that the server device encodes the images in each image group based on multiple encoding modes for each image group in the image combination information, thereby obtaining multiple encoding times corresponding to the image group; The time taken to encode the multiple image groups sent by the server device.

16. A server-side device, characterized in that, include: The first acquisition module is used to acquire the image to be encoded, as well as to acquire the splitting parameters and encoding parameters; The splitting parameters are used to split the image to be encoded into multiple sub-images; The splitting parameters include at least one set of resolutions and a preset number for each set of resolutions; each set of resolutions has the same resolution value, and different sets of resolutions have different resolution values; The encoding parameters include the encoding types corresponding to the multiple sub-images respectively; the encoding types include hard encoding and / or soft encoding; the splitting parameters and the encoding parameters are determined based on the encoding time and decoding time corresponding to the multiple image groups respectively; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters; each decoding time is determined by the client device decoding the image in the image group based on the corresponding decoding mode; each encoding time is determined by the server device encoding the image in the image group based on the corresponding encoding mode. The splitting module is used to split the image to be encoded according to the splitting parameters to obtain multiple first images; The encoding module is used to encode the plurality of first images according to multiple encoding types to obtain a plurality of images to be decoded, and to send the plurality of images to be decoded to the client device; the plurality of first images and the plurality of images to be decoded have a one-to-one correspondence.

17. A client device, characterized in that, include: The receiving module is used to acquire multiple images to be decoded sent by the server device; the multiple images to be decoded are obtained by the server device based on splitting parameters and encoding parameters; The splitting parameters are used to split the image to be encoded into multiple sub-images; The splitting parameters include at least one set of resolutions and a preset number for each set of resolutions; each set of resolutions has the same resolution value, and different sets of resolutions have different resolution values; The encoding parameters include the encoding types corresponding to the multiple sub-images respectively; the splitting parameters and the encoding parameters are determined based on the encoding time and decoding time corresponding to the multiple image groups respectively; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters; each decoding time is determined by the client device decoding the image in the image group based on the corresponding decoding mode; each encoding time is determined by the server device encoding the image in the image group based on the corresponding encoding mode. The second acquisition module is used to acquire decoding parameters and splicing parameters; the decoding parameters include the decoding types corresponding to the plurality of sub-images respectively; The stitching parameters are used to characterize the relative positional relationship of the multiple sub-images; The decoding module is used to decode the multiple images to be decoded according to multiple decoding types to obtain multiple second images; The stitching module is used to stitch the plurality of second images according to the stitching parameters to obtain the target image.

18. An image processing parameter determination device, characterized in that, include: The third acquisition module is used to acquire image combination information; the image combination information includes multiple image groups; the sub-images within different image groups are obtained by splitting the original image according to different splitting parameters; The fourth acquisition module is used to acquire multiple decoding times corresponding to each image group in the image combination information; each decoding time is determined by the client device as described in claim 17 decoding the images in the image group based on the corresponding decoding mode; each decoding mode includes the decoding type used for each sub-image of the image group, and different decoding modes have different combinations of decoding types; The fifth acquisition module is used to acquire multiple encoding times corresponding to each image subgroup in the image combination information; each encoding time is determined by the server device as described in claim 16 encoding the images in the image group based on the corresponding encoding mode; each encoding mode includes the encoding type used for each sub-image of the image group, and different encoding modes have different combinations of encoding types; The determining module is used to determine the target splitting parameters, target concatenation parameters, target decoding parameters, and target encoding parameters based on the multiple decoding times and the multiple encoding times; The server device splits and encodes the image to be encoded according to the target splitting parameters and the target decoding parameters to obtain the image to be decoded, and the client device decodes and stitches the image to be decoded according to the target decoding parameters and the target stitching parameters to obtain the target image.

19. A server-side device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1-15.

20. A client device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1-13, 15.

21. An image processing system, characterized in that... This includes the server-side device as described in claim 19 and the client-side device as described in claim 20.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-15.

23. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-15.

Images