Method for generating image data packet and method for restoring image based on image data packet

By dividing the image into sub-blocks and building image data packets, the problem of restoration failure of image encoding and decoding methods in the prior art in case of packet loss is solved, and the independence of image data packets and rapid and correct restoration is achieved, and the image data packets are adapted to Limitations of data transmission methods such as Beidou short message communication.

CN118741117BActive Publication Date: 2024-11-19CHINA ORDINS GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411195969.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-19
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

Existing image encoding and decoding methods cannot realize instant image restoration in case of packet loss, especially in data transmission methods such as Beidou short message communication, where the packet loss rate is high and the length of a single data packet is limited, resulting in images Restore failed.

Method used

By dividing the target image into sub-blocks, and determining the packet number of each sub-block and the second number under the packet number according to the preset uniform distribution method, the image data packet is constructed and transmitted using a selected transmission method. Even if some image packets are lost, the restoration of other sub-blocks will not be affected, ensuring the independence of image packets.

Benefits of technology

It realizes independent restoration of images in case of packet loss, ensures the independence of image packets and the rapid and correct restoration of images, and adapts to the limitations of data transmission methods such as Beidou short message communication.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118741117B_ABST
    Figure CN118741117B_ABST
Patent Text Reader

Abstract

The disclosed embodiments provide a method for generating an image data packet and a method for restoring an image based on an image data packet. The method for generating an image data packet includes: dividing a target image to obtain sliced ​​sub-blocks and determining a first number for each sliced ​​sub-block; determining the number N of image data packets and the packet number of each image data packet; determining the packet number and the second number corresponding to each sliced ​​sub-block based on the first number; encoding each sliced ​​sub-block to obtain corresponding sub-block encoding data, and sorting and reprocessing the sub-block encoding data corresponding to the sliced ​​sub-blocks with the same packet number according to the second number to determine the target encoding data; constructing N image data packets, and writing the size, number N, corresponding packet number and target encoding data of the target image into one image data packet. The use of this scheme can ensure the independence of the restoration of each image data packet in the subsequent image restoration process as much as possible, and thus realize the restoration of the target image as much as possible.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of image transmission, and in particular to a method for generating an image data packet and a method for restoring an image based on the image data packet. Background Art

[0002] Due to technical principle limitations and numerous interference factors, data transmission methods such as Beidou short message communication have the disadvantages of high packet loss rate and strict limitation of single data packet length. However, due to the irreplaceability of the aforementioned data transmission technology, although its shortcomings are obvious, there are still a large number of application needs, including the need to transmit image data.

[0003] In some application scenarios, when image data is transmitted using the communication method with the aforementioned shortcomings, there is a need for the decoder to restore the image content immediately after receiving the data packet. However, the existing image compression and decoding methods are based on the premise of reliable data packet transmission, and cannot achieve image restoration when the intermediate data packet is lost (that is, part of the image encoding data is missing).

[0004] Some image encoding and decoding methods encode and decode images in a streaming manner. Only after the image area in the previous local area is decoded and restored can the data in the subsequent image area be decoded and restored. In the case that the encoded data packet corresponding to the previous local area is lost, the restoration and display of the subsequent local area cannot be achieved.

[0005] Some other image encoding and decoding methods encode and decode images in a progressive processing method. The progressive encoding method requires encoding the entire image to form base map encoding data. During the decoding process, the base map encoding data needs to be decoded and restored to obtain the base map, and then the decoded data of each local area is filled based on the base map to obtain the decoded image. Since the amount of data after base map encoding is large, multiple data packets are required to carry it. In the case of packet loss of the data packet carrying the base map encoding data, the base map cannot be correctly decoded and restored, and it is impossible to achieve the details of each local area Filling and image content display. Summary of the invention

[0006] In order to solve the problem that the existing image coding and decoding methods cannot adapt to specific data transmission modes, resulting in the inability to achieve instant restoration of image data, the embodiments of the present disclosure provide a method for generating an image data packet and a method for restoring an image based on the image data packet.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for generating an image data packet, comprising:

[0008] Segmenting the target image according to a preset sub-block size to obtain segmented sub-blocks, and determining a first number for each segmented sub-block;

[0009] Determine the number N of image data packets and the packet number of each image data packet according to the upper limit of the data packet byte of the selected transmission mode and the size of the target image;

[0010] Based on the first number, determine the package number corresponding to each sliced ​​sub-block by a preset uniform distribution method, and determine the second number of each sliced ​​sub-block under the corresponding package number;

[0011] Performing encoding processing on each sliced ​​sub-block to obtain corresponding sub-block encoding data, and sorting and reprocessing the sub-block encoding data corresponding to the sliced ​​sub-blocks with the same package number according to the second number to determine the target encoding data;

[0012] N image data packets are constructed, and the size of the target image, the number N, the corresponding packet number and the target coded data are respectively written into one image data packet; the image data packet is transmitted using the selected transmission mode.

[0013] Optionally, determining the package number corresponding to each sliced ​​sub-block by using a preset uniform distribution method includes: using N as a divisor and the first number as a dividend, and using a modulo operation method to determine the package number corresponding to each sliced ​​sub-block.

[0014] Optionally, determining the second number of each sliced ​​sub-block under the corresponding package number includes:

[0015] According to the size of the first number of each sliced ​​sub-block, the second number of each sliced ​​sub-block under the corresponding package number is determined.

[0016] Optionally, before determining the data packet to be formed according to the upper limit of the data packet byte of the selected transmission mode and the size of the target image, the method further includes:

[0017] Determining a quality factor parameter, wherein the quality factor parameter is used to determine a compression degree when compression encoding is performed on the sliced ​​sub-blocks;

[0018] Determining the number of image data packets according to the upper limit of the data packet bytes of the selected transmission mode and the size of the target image includes: determining the number of image data packets according to the upper limit of the data packet bytes of the selected transmission mode, the size of the target image and the quality factor parameter;

[0019] The encoding process is performed on each sliced ​​sub-block to obtain corresponding sub-block encoding data, including:

[0020] Performing discrete cosine transform on the sliced ​​sub-blocks to obtain a first frequency spectrum value matrix;

[0021] Adjusting a preselected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix, wherein a size of the fixed quantization matrix is ​​the same as a size of the first frequency spectrum value matrix;

[0022] Using the elements in the target quantization matrix to quantize the elements at corresponding positions in the first frequency spectrum value matrix to obtain the sub-block coded data;

[0023] The step of writing the size of the target image, the number N, the corresponding packet number and the target coded data into each image data packet comprises:

[0024] The size of the target image, the total number of data packets N, the quality factor parameter, the corresponding packet number and the target encoded data are written into each image data packet.

[0025] Optionally, adjusting the pre-selected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix includes: using Calculating and obtaining the value of each element of the target quantization matrix to construct the target quantization matrix;

[0026] The step of using the elements in the target quantization matrix to quantize the elements at corresponding positions in the first frequency spectrum value matrix to obtain the sub-block coded data comprises: using Quantizing the elements at the corresponding positions to obtain the sub-block coded data;

[0027] in The target quantization matrix The elements at is the quality factor parameter, For a fixed quantization matrix The elements at is the first frequency spectrum value matrix in The elements at Encode data for the sub-block in The element at .

[0028] In a second aspect, an embodiment of the present disclosure provides a method for restoring an image based on an image data packet, comprising:

[0029] Unpack the acquired image data packets to determine the size of the target image, the total number of data packets N, the current packet number and the current encoded data;

[0030] Constructing a canvas to be filled according to the size of the target image and a preset sub-block size, dividing the canvas to be filled into a plurality of areas to be filled according to the preset sub-block size, and determining a third number of each area to be filled;

[0031] Decoding the current coded data to obtain a plurality of sub-block coded data and second serial numbers of the sub-block coded data, and decoding each of the sub-block coded data to obtain a corresponding cut sub-block;

[0032] Based on the current packet number and the second number of the sub-block coded data, restore the first number of each sliced ​​sub-block corresponding to each sub-block coded data; determine the area to be filled corresponding to the third number that is the same as the first number as the target filling area; or

[0033] Based on the third number, determine the allocation package number corresponding to each area to be filled by a preset uniform allocation method, and determine the fourth number of each area to be filled under the corresponding package number; determine the area to be filled whose corresponding allocation package number is the same as the current package number and whose fourth number is the same as the second number as the target filling area;

[0034] Fill the corresponding cut sub-blocks into the corresponding target filling area.

[0035] Optionally, the step of determining the allocation package numbers corresponding to the respective to-be-filled areas by a preset uniform allocation method based on the third number includes:

[0036] Taking N as the divisor and the third number as the dividend, the allocation package number corresponding to each of the to-be-filled areas is determined.

[0037] Optionally, determining the fourth number of each to-be-filled area under the corresponding package number includes:

[0038] According to the order of the third numbers of the areas to be filled, the fourth numbers of the cut sub-blocks under the corresponding allocation package numbers are determined.

[0039] Optionally, based on the current packet number and the second number of the sub-block coded data, restoring the first number of each sliced ​​sub-block corresponding to each sub-block coded data includes:

[0040] Determine a multiple coefficient based on a second number of the sub-block coded data;

[0041] The sum of the result of multiplying the multiple coefficient by N and the current package number is taken as the first number.

[0042] Optionally, the image data packet further includes a quality factor parameter; and the method further includes:

[0043] In the process of unpacking the acquired image data packet, simultaneously acquiring the quality factor parameter;

[0044] Adjusting a preselected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix;

[0045] The decoding process is performed on each of the sub-block coded data to obtain corresponding sliced ​​sub-blocks, including:

[0046] Performing inverse quantization processing based on the elements in the target quantization matrix and the elements at corresponding positions of the sub-block coded data to obtain a second frequency spectrum value matrix;

[0047] An inverse discrete cosine transform is performed based on the second frequency spectrum value matrix to obtain corresponding sliced ​​subblocks.

[0048] Optionally, after filling the sliced ​​sub-blocks into the corresponding target filling areas, the method further includes:

[0049] Determine whether there is an incorrectly filled area in the area to be filled, wherein the incorrectly filled area is an area in which the corresponding cut sub-block is not correctly filled;

[0050] Determining that there is the incorrectly filled area, and determining the most recently filled area according to the position of the incorrectly filled area, wherein the most recently filled area is an area that has been filled with the corresponding cut sub-block;

[0051] The sliced ​​sub-blocks corresponding to the most recently filled area are used as temporary filling data to fill the incorrectly filled area.

[0052] In a third aspect, an embodiment of the present disclosure provides a device for generating an image data packet, including:

[0053] An image segmentation unit, configured to segment the target image into segmented sub-blocks according to a preset sub-block size, and determine a first number for each segmented sub-block;

[0054] a packet quantity determination unit, used to determine the number N of image data packets and the packet number of each image data packet according to the upper limit of the data packet byte of the selected transmission mode and the size of the target image, wherein the number N of image data packets is less than the number of the sliced ​​sub-blocks;

[0055] an allocating unit, configured to determine, based on the first number, a package number corresponding to each sliced ​​sub-block by a preset uniform allocation method, and determine a second number of each sliced ​​sub-block under the corresponding package number;

[0056] an encoding unit, configured to perform encoding processing on each sliced ​​sub-block to obtain corresponding sub-block encoding data, and determine target encoding data based on the sub-block encoding data and the corresponding second number;

[0057] The data packet construction unit is used to construct N image data packets, and write the size of the target image, the number N, the corresponding packet number and the target encoding data into each image data packet respectively; the image data packet is transmitted using the selected transmission method.

[0058] In a fourth aspect, an embodiment of the present disclosure provides a device for restoring an image based on an image data packet, comprising:

[0059] The unpacking unit is used to unpack the acquired image data packet to determine the size of the target image, the total number of data packets N, the current packet number and the current encoded data;

[0060] A canvas construction and division unit 502 is used to construct a canvas to be filled according to the size of the target image and a preset sub-block size, divide the canvas to be filled into a plurality of areas to be filled according to the preset sub-block size, and determine a third number of each area to be filled;

[0061] A decoding unit, configured to decode the current encoded data to obtain a plurality of sub-block encoded data and second serial numbers of the sub-block encoded data, and decode each of the sub-block encoded data to obtain a corresponding sliced ​​sub-block;

[0062] a filling position determining unit, configured to restore the first number of each sliced ​​sub-block based on the current packet number and the second number of the sub-block coded data; determine the to-be-filled area corresponding to the third number which is the same as the first number as the target filling area; or

[0063] Based on the third number, determine the allocation package number corresponding to each area to be filled by a preset uniform allocation method, and determine the fourth number of each area to be filled under the corresponding package number; determine the area to be filled whose corresponding allocation package number is the same as the current package number and whose fourth number is the same as the second number as the target filling area;

[0064] The filling unit is used to fill the corresponding cut sub-blocks into the corresponding area to be filled.

[0065] In a fifth aspect, an embodiment of the present disclosure provides a computing device, comprising a processor and a memory, wherein the memory is used to store a computer program; when the computer program is loaded by the processor, the processor executes the method for generating an image data packet as described above and / or the method for restoring an image based on the image data packet as described above.

[0066] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the method for generating an image data packet as described above and / or the method for restoring an image based on the image data packet as described above.

[0067] By using the image packet generation method and the image restoration method based on the image data packet provided by the embodiments of the present disclosure, even if some image data packets have packet loss problems, it only affects the restoration of the corresponding sliced ​​sub-blocks, and will not affect the restoration of other sliced ​​sub-blocks. In other words, by using the embodiments of the present disclosure, the independence of the restoration of each image data packet in the subsequent image restoration process can be guaranteed as much as possible, thereby realizing the restoration of the target image as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor, including

[0070] Figure 1 is a flow chart of a method for generating an image data packet provided by an embodiment of the present disclosure;

[0071] Figure 2 is a flow chart of a method for restoring an image based on a picture data packet provided by some embodiments of the present disclosure;

[0072] Figure 3 is a flow chart of a method for restoring an image based on a picture data packet provided in some further embodiments of the present disclosure;

[0073] Figure 4 is a schematic diagram of the structure of a device for generating an image data packet provided by an embodiment of the present disclosure;

[0074] Figure 5 is a structural schematic diagram of a device for restoring an image based on an image data packet provided by an embodiment of the present disclosure;

[0075] Figure 6 It is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0076] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0077] The term "including" and its variations used in this document are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. In this document, relational terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0078] In order to solve the problem that the image data packets generated by the existing methods cannot adapt to the packet loss rate of some data transmission methods and the length of a single data packet is strictly limited, resulting in the problem that the recipient cannot quickly and correctly restore the image, the embodiment of the present disclosure provides a new method for generating image data packets, and a method for restoring images based on image data packets that is adapted to the aforementioned method for generating image data packets. The method provided by the embodiment of the present disclosure is executed by a computing device.

[0079] Figure 1 FIG. 1 is a flow chart of a method for generating an image data packet provided by an embodiment of the present disclosure. Figure 1 As shown, the method for generating an image data packet provided by the embodiment of the present disclosure includes S110-S150.

[0080] S110: Divide the target image into sub-blocks according to a preset sub-block size, and determine a first number for each sub-block.

[0081] The target image is an image that needs to be transmitted to the receiving end after encoding. In some embodiments, the target image is an image obtained by processing the original image. For example, when the original image is an image with a high resolution and a large size, the original image can be downsampled to reduce the resolution and size of the original image to form the target image.

[0082] The preset sub-block size is a size that is predetermined according to the subsequent encoding method and is used as a segmentation standard. For example, when encoding each segmented sub-block in the subsequent process, it is necessary to process 8*8 pixel contents at the same time, and the corresponding preset sub-block size should be set to 8*8. The target image is segmented according to the preset sub-block size to obtain segmented sub-blocks, and the sizes of the obtained segmented sub-blocks are all the preset sub-block sizes.

[0083] It should be noted that in some cases the size of a certain dimension of the target image is not an integer multiple of the corresponding dimension size in the preset sub-block size. In this case, the target image needs to be edge-filled to increase the size of the corresponding dimension so that the sizes of the segmented sub-blocks are all the preset sub-block sizes.

[0084] In a specific implementation, the computing device may segment the target image according to a preset segmentation rule and a preset sub-block size to obtain segmented sub-blocks. For example, in some embodiments, the computing device segments the target image line by line to obtain the segmented sub-blocks.

[0085] In the process of or after obtaining the sliced ​​sub-blocks from the target image, the computing device numbers each sliced ​​sub-block and determines the first number of each sliced ​​sub-block. In the embodiment of the present disclosure, the sliced ​​sub-blocks formed by sequential slicing form a set P{P0, P1, P2, ..., Pm}, and the first numbers of the corresponding sliced ​​sub-blocks form a set {0, 1, 2, ..., m}.

[0086] It can be imagined that, according to a predetermined segmentation order rule and a numbering rule for each sub-block, the segmented sub-blocks can be spliced ​​together based on the first numbers of each segmented sub-block to restore the target image.

[0087] S120: Determine the number N of image data packets and the packet number of each image data packet according to the upper limit of the data packet byte of the selected transmission mode and the size of the target image.

[0088] The selected transmission mode is a transmission mode pre-selected for transmitting subsequently generated image data packets. The disclosed embodiments do not limit the type of the selected transmission mode. In some embodiments, the selected transmission mode may be a satellite communication transmission mode such as the Beidou short message of the background technology distance; in other embodiments, the selected transmission mode may be a wired network transmission, a wireless communication transmission mode, etc. Due to transmission protocol and physical layer limitations, a single data packet transmitted by the selected transmission mode has an upper byte limit.

[0089] The number of image data packets N is determined based on the upper byte limit of the data packet of the selected transmission method and the size of the target image. The number of image data packets N is estimated by taking into account the maximum amount of valid data that the data packet of the selected transmission method can carry and the amount of data formed by the subsequent encoding processing of the current size of the target image.

[0090] In practical applications, in order to avoid insufficient number N of data packets, a slightly smaller effective transmission data volume can be determined based on the data packet byte upper limit of the selected transmission method, and a larger encoded data volume can be estimated based on the current storage of the target image. The aforementioned larger encoded data volume is divided by the effective transmission data volume to obtain the number N of image data packets.

[0091] It should be noted here that the number N of image data packets determined is strongly correlated with the encoding method used in the subsequent step. When determining the number of image data packets here, the compression rate of the encoding method used in the subsequent step has been taken into account.

[0092] When the number N of image data packets is determined, the number of each image data packet (that is, the image data packet hereinafter) can also be determined, and the numbers are {0, 1, 2, ..., N-1}.

[0093] In practical applications, the preset sub-block size is small, and the upper limit of the data packet byte of the selected transmission method is not too small. In most cases, the number of data packets N to be formed is less than the number of segmented sub-blocks m. In extreme cases, the number of data packets N to be formed may be equal to the number of segmented sub-blocks m. That is to say, in practical applications, in order to avoid the problem that the encoded data corresponding to a segmented sub-block is divided into two data packets again, and the segmented sub-block cannot be restored because some data packets are lost, the number of data packets N to be formed will not be greater than the number of segmented sub-blocks m. If considered from the above perspective, in some cases, the size of the preset sub-block size may be affected by the upper limit of the data packet byte of the selected transmission method, and cannot be set too large.

[0094] S130: Based on the first number, determine the package number corresponding to each sliced ​​sub-block by a preset uniform allocation method, and determine the second number of each sliced ​​sub-block under the corresponding package number.

[0095] The preset uniform distribution method is a method that is predetermined and used to determine the packet number corresponding to each sliced ​​sub-block, and the sorting number (that is, the second number) under the corresponding packet number. It should be noted that although it is called a uniform distribution method, in some cases, due to the limitation of the sliced ​​sub-blocks and the data packets N to be formed, it cannot achieve completely uniform distribution. That is, the preset uniform distribution method is only a method to achieve uniform distribution as much as possible, and it does not necessarily achieve completely uniform distribution of the sliced ​​sub-blocks.

[0096] In some embodiments, the computing device uses the following method to determine the package number corresponding to each sliced ​​sub-block: determine the number of sliced ​​sub-blocks {x1, x2, ..., xN} corresponding to each package number according to the number of image data packets N and the number of sliced ​​sub-blocks, and then segment the first number in sequence according to the aforementioned number, and assign the package number corresponding to each segment number in turn, thereby correspondingly determining the package number of the corresponding sliced ​​sub-block.

[0097] In some other embodiments, the computing device determines the package number corresponding to each sliced ​​sub-block by a remainder operation, and then determines the second number of each sliced ​​sub-block under the corresponding package number. Specifically, the computing device uses N as the divisor and the first number as the dividend, and uses the remainder operation method to determine the remainder of each sliced ​​sub-block, and uses the package number corresponding to the remainder as the package number of the sliced ​​sub-block. By adopting the remainder budget method, the sliced ​​sub-blocks separated by a certain distance can be assigned to the same package number. This is more conducive to subsequent image restoration, which will be analyzed later.

[0098] Of course, in other embodiments, other methods may be used to determine the package numbers corresponding to the sliced ​​sub-blocks.

[0099] After determining the package number corresponding to each sliced ​​sub-block, it is also necessary to determine the second number of each sliced ​​sub-block in the corresponding package number. In some embodiments, the computing device can determine the second number corresponding to each sliced ​​sub-block according to the order of the first numbers assigned to each sliced ​​sub-block under the package number.

[0100] It should be noted here that determining the packet number and the second number corresponding to each sliced ​​sub-block is used for subsequently combining the encoded data of the sliced ​​sub-block to form target encoded data stored in the data packet to be transmitted.

[0101] S140: performing encoding processing on each sliced ​​sub-block to obtain corresponding sub-block encoding data, and sorting and processing the sub-block encoding data corresponding to the sliced ​​sub-blocks with the same package number according to the second number to determine the target encoding data.

[0102] In the disclosed embodiment, the computing device may use various existing encoding methods to encode each sliced ​​sub-block to obtain corresponding sub-block encoding data. Specifically, it may use a lossy compression method to encode each sliced ​​sub-block, or it may use a lossless compression method to encode each sliced ​​sub-block.

[0103] After obtaining the sub-block coded data corresponding to each sliced ​​sub-block, the computing device will sort the sub-block coded data corresponding to the sliced ​​sub-blocks of the same package number according to the second number to form sorted sub-block coded data, and re-process the sorted sub-block coded data to determine the target coded data. The post-processing method can adopt the existing data coding method.

[0104] Through the above processing, all the data required for forming an image data packet have been obtained, and then the image data packet can be formed by S1450.

[0105] S150: Construct N image data packets, and write the size, quantity N, corresponding packet number and target encoding data of the target image into each image data packet respectively.

[0106] In the disclosed embodiment, the computing device constructs an image data packet according to a preset data packet construction rule, and writes the size, quantity N, corresponding packet number and target encoding data of the target image into the image data packet to obtain N assembled data packets to be transmitted. The aforementioned N image data packets are transmitted using the selected transmission method. Of course, in actual applications, the image data packet is not limited to the aforementioned data content, and other data content can also be added.

[0107] In a specific application, the image and text data packet to be transmitted includes a data packet start mark, a custom information segment, an image information segment, an image frame information segment, compressed image data and a data packet end mark, wherein the custom information segment records the aforementioned target image size, the number of image data packets N, and the corresponding packet number information.

[0108] The method for generating an image data packet of the embodiment of the present disclosure divides the target image into a plurality of sliced ​​sub-blocks, and determines the package number and the second number assigned to each sliced ​​sub-block under the package number. After the sliced ​​sub-blocks are encoded to obtain the corresponding sub-block encoded data, and the sub-block encoded data corresponding to the same encoded sliced ​​sub-blocks are processed according to the second encoding to determine the target encoded data, N image data packets are constructed, and the size, quantity N, corresponding package number and target encoded data of the target image are written into an image data packet to form N image data packets. The N image data packets independently use the selected transmission mode. After the receiving end device receives any image data packet, it can process the image data packet according to the decoding rule corresponding to the aforementioned encoding rule, restore the corresponding sliced ​​sub-block, and fill the sliced ​​sub-block into the corresponding filling area of ​​the canvas. Even if some image data packets have packet loss problems, it only affects the restoration of the corresponding sliced ​​sub-blocks, and will not affect the restoration of other sliced ​​sub-blocks. In other words, the image data packet generation method of the embodiment of the present disclosure can ensure the independence of the restoration of each image data packet in the subsequent image restoration process as much as possible, and thus realize the restoration of the target image as much as possible.

[0109] The following is an analysis of how to encode each sliced ​​sub-block in the embodiment of the present disclosure to obtain encoded data of a sub-block. In some embodiments, the computing device may obtain the sub-block encoded data using the following S141-S142.

[0110] S141: Perform discrete cosine transform on the sliced ​​sub-blocks to obtain a first frequency spectrum value matrix.

[0111] Specifically, the computing device performs fast discrete cosine transform on each of the sliced ​​sub-blocks to obtain a first frequency spectrum value matrix. In a specific implementation, when the sliced ​​sub-blocks are 8*8 sub-blocks, the computing device uses formula 1 to calculate and obtains The corresponding frequency values ​​are assembled to obtain the first frequency spectrum value matrix.

[0112] Formula 1

[0113] , is the pixel value at the corresponding position of the sliced ​​sub-block.

[0114] It should be noted that the above formula is for the sliced ​​sub-blocks to be grayscale images, that is, the target image is a grayscale image. If the target image is a color image, the color image (such as an RGB image) can be transformed into a color space, and the grayscale data of each channel after the color space transformation can be processed using the above formula.

[0115] S142: Use a predetermined fixed quantization matrix to perform quantization processing on elements corresponding to the first frequency spectrum value matrix to obtain sub-block coded data.

[0116] The elements at each position of the predetermined fixed quantization matrix are determined according to the perception characteristics of the human eye to the corresponding frequency. The predetermined fixed quantization matrix is ​​used to quantize the elements corresponding to the first frequency spectrum value matrix to obtain sub-block coded data, which is obtained by dividing the elements in the first frequency spectrum value matrix and the elements at the corresponding positions of the fixed quantization matrix, and rounding down to determine the corresponding quantized data. Then, the quantized data are arranged in order to obtain the sub-block coded data.

[0117] In a specific application, the quantization formula for the computing device is ,in is the first frequency spectrum value matrix in The elements at For a fixed quantization matrix The elements at Encode data for the sub-block in The element at .

[0118] In practical applications, different target images may have different application requirements, and correspondingly different requirements may be placed on the quality of the restored image. When lossy compression is used to compress the target image, different compression ratios may be adaptively selected. Accordingly, in some embodiments, before encoding the sliced ​​sub-blocks to obtain the corresponding sub-block encodings, the following S160 may be performed.

[0119] S160: Determine quality factor parameters.

[0120] As analyzed above, the quality factor parameter is used to determine the compression degree when compressing and encoding the sliced ​​sub-blocks. The size of the quality factor parameter setting directly affects the size of the sub-block encoding data formed by a single sliced ​​sub-block.

[0121] When executing the aforementioned S160, S120 determines the number N of image data packets and the packet number of each image data packet according to the upper limit of the data packet byte of the selected transmission mode and the size of the target image, which is specifically as follows S121.

[0122] S121: Determine the number N of image data packets according to the upper limit of the data packet bytes of the selected transmission mode, the size of the target image, and the quality factor parameter.

[0123] As analyzed above, the size of the quality factor parameter setting directly affects the size of the sub-block coded data formed by a single sliced ​​sub-block, which directly affects the size of the subsequent target coded data. And because the upper limit of the byte of the selected transmission data packet is fixed, the size of the corresponding quality factor parameter setting will also affect the number of image data packets.

[0124] In a specific implementation, the compression ratios of preset sub-blocks under various quality factor parameters can be counted in advance, and the number N of image data packets can be determined according to the aforementioned compression, the upper limit of the data packet bytes of the selected transmission mode and the size of the target image.

[0125] When it is necessary to determine the quality factor parameter and use the quality factor parameter to adjust the compression degree of the sliced ​​sub-blocks, the aforementioned S140 performs encoding processing on each sliced ​​sub-block to obtain the corresponding sub-block encoding data, including the following S143-S145.

[0126] S143: Perform discrete cosine transform on the sliced ​​sub-blocks to obtain a first frequency spectrum value matrix.

[0127] The specific execution process of S143 is the same as that of S141 above, and will not be repeated here.

[0128] S144: Adjust the pre-selected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix.

[0129] The pre-selected fixed quantization matrix is ​​adjusted based on the quality factor parameter, and each element in the fixed quantization matrix is ​​adjusted respectively.

[0130] In some embodiments, the computing device may use Formula 2 to calculate the value of each element in the target quantization matrix and construct the target quantization matrix.

[0131] Formula 2

[0132] in, The target quantization matrix The elements at is the quality factor parameter, For a fixed quantization matrix The element at .

[0133] S145: Using the elements in the target quantization matrix to perform quantization processing on the elements at corresponding positions in the first frequency spectrum value matrix to obtain sub-block coded data.

[0134] In some embodiments, the quantization formula used by the computing device is ,in Encode data for the sub-block in The element at .

[0135] When the quality factor parameter is used to control the compression rate of each sliced ​​sub-block, S150 in the previous text writes the size and quantity N of the target image, the corresponding packet number and the target coding data into each image data packet respectively. Specifically, the size of the target image, the total number of data packets N, the quality factor parameter, the corresponding packet number and the target coding data are written into each image data packet.

[0136] As mentioned above, in S140, the sub-block coded data corresponding to the sliced ​​sub-blocks of the same package number need to be sorted and reprocessed according to the second number to determine the target coded data. In a specific implementation, the computing device can use Huffman coding (that is, using optimal binary tree coding) to re-encode the sub-block coded data sorted according to the second number to obtain the target coded data. In other implementations, the computing device can also use other coding methods to re-encode the sequentially shot sub-block coded data.

[0137] The disclosed embodiment also provides a method for restoring an image based on a picture data packet. Figure 2 is a flow chart of a method for restoring an image based on a picture data packet provided by some embodiments of the present disclosure. Figure 2 As shown, the method for restoring an image based on an image data packet includes S210-S250.

[0138] S210: Unpacking the acquired image data packet to determine the size of the target image, the total number of data packets N, the current packet number and the current encoded data.

[0139] In the disclosed embodiment, after the computing device obtains the image data packet transmitted via the selected transmission method, it unpacks the image data packet according to a predetermined data packet unpacking rule (i.e., an unpacking rule relative to the aforementioned data packet construction rule) to obtain the size of the target image, the total number of data packets N, the current packet number and the current encoded data.

[0140] S220: constructing a canvas to be filled according to the size of the target image and the preset sub-block size, dividing the canvas to be filled into a plurality of areas to be filled according to the preset sub-block size, and determining a third number of each area to be filled.

[0141] In a specific implementation, the computing device may first determine whether the size of the target image is divisible by the preset sub-block size (that is, the length of the target image is divided by the length of the preset sub-block, and the width of the target image is divided by the width of the preset sub-block). If the size of the target image is divisible by the preset sub-block size, the canvas to be filled is constructed according to the size of the target image, so that the size of the canvas to be filled is the size of the target image. If the size of the target image is not divisible by the preset sub-block size, the minimum enlargement processing is first performed based on the size of the target image to obtain an enlarged size, and the enlarged size is made divisible by the preset sub-block size, and then the enlarged size is used to construct the canvas to be filled.

[0142] After obtaining the filled canvas, the computing device uses the corresponding method in S110 in the above-mentioned image data packet generation method to divide the canvas to be filled according to the preset sub-block size to obtain multiple areas to be filled, and determines the third number of each area to be filled according to the corresponding method in S110.

[0143] It can be imagined that the third number of the area to be filled corresponding to the position is the same as the first number of the cut sub-block at the corresponding position in the previous text.

[0144] S230: Decode the current coded data to obtain a plurality of sub-block coded data and second serial numbers of the sub-block coded data, and decode each sub-block coded data to obtain a corresponding sliced ​​sub-block.

[0145] In the embodiment of the present disclosure, the computing device uses a decoding method corresponding to the encoding method in S140 above to decode the current encoded data to obtain a plurality of sub-block encoded data. After obtaining the plurality of sub-block encoded data, the second serial number of each sub-block encoded data can be determined according to the sorting data of each sub-block encoded data. Then, the decoding method corresponding to the encoding method for the cut sub-block above is used to decode the sub-block encoded data to obtain the corresponding cut sub-block.

[0146] It should be noted here that: if a lossless encoding method is adopted in S140, then the corresponding lossless decoding method is adopted in S230, and the pixel grayscale of the sliced ​​sub-block obtained in S230 is the same as the pixel grayscale of the sliced ​​sub-block before encoding; and if a lossy encoding method is adopted in S140, the pixel grayscale of the sliced ​​sub-block restored in S240 may not be the same as the pixel grayscale of the sliced ​​sub-block before encoding.

[0147] S240: Restore the first number of each sliced ​​sub-block based on the current package number and the second number of the sub-block coded data, and determine the to-be-filled area corresponding to the third number that is the same as the first number as the target filling area.

[0148] In the disclosed embodiment, based on the current packet number and the number of the sub-block coded data, the first number of each sliced ​​sub-block corresponding to each sub-block coded data is restored by using a method opposite to the above S130 to determine the first number of each sliced ​​sub-block.

[0149] For example, if S130 determines that the number of sliced ​​sub-blocks corresponding to each package number is {x1, x2, ..., xN}, and the first number is segmented and cut in the order of the aforementioned number, and the second code is determined according to the first number sorting of the sliced ​​sub-blocks, then the computing device determines the first number interval range corresponding to the current package number decoding according to the aforementioned method, and determines the first number of each sliced ​​sub-block within the first number interval range according to the second numbering rule.

[0150] If the remainder calculation method is used in S130 to determine the package number corresponding to each sliced ​​sub-block, and the second number under the package number is determined according to the first number sequence, the computing device determines the corresponding first number according to the inverse method of the above method. Specifically, the multiplication coefficient is determined according to the second number of the sub-block coded data, and the sum of the result of multiplying the multiplication coefficient by N and the current package number is used as the first number.

[0151] According to the above analysis, the position of the sliced ​​sub-block in the target image can be determined based on the first number.

[0152] S250: Fill the corresponding sliced ​​sub-blocks into the corresponding target filling area.

[0153] As analyzed above, the first number can determine the position of the sliced ​​sub-block in the target image, and accordingly filling it into the to-be-filled area corresponding to the third number which is the same as the first number can achieve restoration of the target image at the corresponding position. Therefore, the computing device uses the to-be-filled area corresponding to the third number which is the same as the first number as the target filling area, and fills the corresponding sliced ​​sub-block into the target filling area.

[0154] By adopting the embodiment of the present disclosure, after receiving an image data packet, the sliced ​​sub-blocks of the target image and the target filling area corresponding to the sliced ​​sub-blocks can be decoded according to the above S210-S250, and the corresponding sliced ​​sub-blocks are filled into the target filling area to achieve partial restoration of the target image, and this local area will not be unable to be restored immediately due to the influence of other factors.

[0155] In a specific implementation, when the computing device executes the aforementioned S230 to decode the current encoded data and obtains multiple sub-block encoded data and second numbers of the sub-block encoded data, it can use a method opposite to the previous reprocessing method to perform reverse processing to obtain multiple sub-block encoded data, and determine the corresponding second numbers according to the sorting order of the multiple sub-block encoded data.

[0156] In some embodiments, the computing device may obtain corresponding sliced ​​sub-blocks based on the sub-block coded data according to S231 - S232 as follows.

[0157] S231: Perform inverse quantization processing based on the elements in the fixed quantization matrix and the elements at corresponding positions of the sub-block coded data to obtain a second frequency spectrum value matrix.

[0158] In a specific implementation, the elements in the fixed quantization matrix are multiplied by the elements at the corresponding positions in the sub-block coded data to obtain the inverse quantization elements at the corresponding positions. The inverse quantization elements are arranged according to the corresponding positions to form a second frequency spectrum value matrix.

[0159] S232: Perform inverse discrete cosine transform based on the second frequency spectrum value matrix to obtain corresponding sliced ​​sub-blocks.

[0160] After obtaining the second frequency spectrum value matrix, the inverse discrete cosine transform is used, that is, the inverse transform formula of the above formula 1 is used to obtain the pixel value at each position of the sliced ​​sub-block, and then the pixel values ​​at each position are combined to form the sliced ​​sub-block.

[0161] In some other embodiments, the image data packet also includes a quality factor parameter. Accordingly, the quality factor parameter is obtained at the same time during the execution of the aforementioned S210. In this case, the computing device first needs to adjust the pre-selected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix. In a specific implementation, the method of adjusting the fixed quantization matrix based on the quality factor parameter to obtain the target quantization matrix is ​​the same as the above, and will not be repeated here.

[0162] When the target quantization matrix is ​​obtained, the computing device obtains the corresponding sliced ​​subblocks based on the subblock coding data according to the following S233-S234.

[0163] S233: Perform inverse quantization processing based on the elements in the target quantization matrix and the elements at corresponding positions of the sub-block coded data to obtain a second frequency spectrum value matrix.

[0164] S234: Perform inverse discrete cosine transform based on the second frequency spectrum value matrix to obtain corresponding sliced ​​sub-blocks.

[0165] As analyzed above, in actual applications, due to problems with the selected transmission method itself, some image data packets may be lost, resulting in the inability to restore the corresponding sliced ​​sub-blocks, which means that the corresponding area of ​​the target image cannot be displayed. In some cases, the pixel content of the adjacent area of ​​the aforementioned sliced ​​sub-block that cannot be restored may be the same or similar to the pixel content of this area. Temporarily placing the pixel content of the adjacent area in the aforementioned sliced ​​sub-block that cannot be restored may allow the target image to display more content, and is also beneficial for users to make subsequent judgments based on the displayed image. To this end, in some embodiments, after executing the aforementioned S250, the following S260-S280 may also be executed.

[0166] S260: Determine whether there is an incorrectly filled area in the area to be filled; if so, execute S270; if not, the execution ends.

[0167] The incorrectly filled area is the area where the corresponding sliced ​​sub-block is not filled. It should be noted that in actual applications, the incorrectly filled area may have been filled with the sliced ​​sub-block corresponding to the adjacent filled area, that is, from the display point of view, the incorrectly filled area has displayed pixel content, but this pixel content is not the content of the sliced ​​sub-block that should be correctly filled in the incorrectly filled area.

[0168] S270: Determine the most recently filled area according to the position of the incorrectly filled area.

[0169] The most recently filled region is a region that has been filled with the corresponding sliced ​​sub-block. In a specific embodiment, if the incorrectly filled region has at least two most recently filled regions, the most recently filled region with the smallest first number may be used as the selected most recently filled region.

[0170] S280: Fill the incorrectly filled area with the sliced ​​sub-blocks corresponding to the most recently filled area as temporary filling data.

[0171] It should be noted that if the receiving computing device receives a new image data packet again, and the sliced ​​sub-blocks obtained after unpacking are the sliced ​​sub-blocks corresponding to the correct filling area, it will take the incorrect filling area as the new target filling area and fill the aforementioned sliced ​​sub-blocks into the new target filling area. If the receiving computing device receives a data packet with a repeated packet number, it can directly delete the subsequently received data packet to avoid performing repeated operations on this image data packet.

[0172] As analyzed above, in some embodiments, the method for generating the image data packet adopts a modulo operation to determine the packet number corresponding to each sliced ​​sub-block, and thus determines the image data packet to which it is assigned. This method assigns several adjacent sliced ​​sub-blocks to different image data packets. Accordingly, when executing the aforementioned S270, more and more suitable recently filled areas can be provided, making the adjacent filling method more reasonable.

[0173] Figure 3 1 is a flow chart of a method for restoring an image based on a picture data packet provided in some embodiments of the present disclosure. Figure 3 As shown, the method for restoring an image based on an image data packet includes S310-S350.

[0174] S310: Unpack the acquired image data packet to determine the size of the target image, the total number of data packets N, the current packet number and the current encoded data.

[0175] S320: constructing a canvas to be filled according to the size of the target image and the preset sub-block size, dividing the canvas to be filled into a plurality of areas to be filled according to the preset sub-block size, and determining a third number of each area to be filled.

[0176] The implementation process of S310-S320 is the same as that of S210-S220 in the previous text, and will not be repeated here.

[0177] S330: Based on the third number, determine the allocation package number corresponding to each area to be filled by a preset uniform allocation method, and determine the fourth number of each area to be filled under the corresponding package number.

[0178] In a specific implementation, the implementation process of S330 is the same as the method of determining the package number corresponding to each sliced ​​sub-block and determining the second number of the sliced ​​sub-block under the corresponding package number in the above S120, which will not be repeated here.

[0179] S340: Decode the current coded data to obtain a plurality of sub-block coded data and second serial numbers of the sub-block coded data, and decode each sub-block coded data to obtain a corresponding sliced ​​sub-block.

[0180] The implementation process of S340 is the same as that of S230 above, and will not be repeated here.

[0181] S350: Determine that the corresponding allocated package number is the same as the current package number, and the to-be-filled area whose fourth number is the same as the second number is the target filling area.

[0182] Different from the idea of ​​reversely determining the first number of each sliced ​​sub-block in the aforementioned embodiment, in the embodiment disclosed herein, after determining the package number of a certain area to be filled and the fourth number under the corresponding package number, it is checked whether the sliced ​​sub-block of the corresponding package number is received. If the corresponding sliced ​​sub-block is received, the target filling area corresponding to each sliced ​​sub-block can be determined according to the fourth number and the sorting order of the sliced ​​sub-blocks.

[0183] In one embodiment, the current packet number and the packet number of the sliced ​​sub-block are determined by modulo calculation, and the sliced ​​sub-blocks are sorted according to the size of the first number. Accordingly, the target area corresponding to each sliced ​​sub-block is determined according to the above rule.

[0184] S360: Fill the corresponding sliced ​​sub-blocks into the corresponding target filling area.

[0185] The implementation process of S360 is the same as that of S250 above, and will not be repeated here. In addition, in this solution, after executing S350, the above S260-S280 may also be executed.

[0186] The disclosed embodiment also provides a device for generating an image data packet. Figure 4 FIG. 1 is a schematic diagram of the structure of a device for generating an image data packet provided by an embodiment of the present disclosure. Figure 4 As shown, the device 400 for generating an image data packet includes an image cutting unit 401 , a packet amount determining unit 402 , an allocating unit 403 , an encoding unit 404 and a data packet constructing unit 405 .

[0187] The image segmentation unit 401 is used to segment the target image according to a preset sub-block size to obtain segmented sub-blocks, and determine a first number for each segmented sub-block.

[0188] The packet quantity determination unit 402 is used to determine the number N of image data packets and the packet number of each image data packet according to the upper limit of the data packet byte of the selected transmission mode and the size of the target image. The number N of image data packets is less than the number of sliced ​​sub-blocks.

[0189] The allocating unit 403 is used to determine the package number corresponding to each sliced ​​sub-block based on the first number by a preset uniform allocation method, and determine the second number of each sliced ​​sub-block under the corresponding package number.

[0190] The encoding unit 404 is used to perform encoding processing on each sliced ​​sub-block to obtain corresponding sub-block encoding data, and determine target encoding data based on the sub-block encoding data and the corresponding second number.

[0191] The data packet construction unit 405 is used to construct N image data packets, and write the size and quantity N of the target image, the corresponding packet number and the target encoding data into each image data packet respectively; the image data packet is transmitted using the selected transmission mode.

[0192] In some embodiments, the allocating unit 403 uses N as the divisor and the first number as the dividend, and adopts a modulo operation method to determine the package number corresponding to each sliced ​​sub-block.

[0193] In some embodiments, the allocating unit 403 determines the second number of each sliced ​​sub-block under the corresponding package number according to the size of the first number of each sliced ​​sub-block.

[0194] In some embodiments, the packet quantity determination unit 402 determines the number of image data packets according to the upper limit of the data packet bytes of the selected transmission mode, the size of the target image and the pre-selected quality factor parameter. The encoding unit 404 performs discrete cosine transform on the sliced ​​sub-blocks to obtain a first frequency spectrum value matrix; adjusts the pre-selected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix, the size of the fixed quantization matrix is ​​the same as the size of the first frequency spectrum value matrix; uses the elements in the target quantization matrix to quantize the elements at the corresponding positions in the first frequency spectrum value matrix to obtain sub-block encoding data. The data packet construction unit 405 writes the size of the target image, the total number of data packets N, the quality factor parameter, and the corresponding packet number and target encoding data into each image data packet.

[0195] The disclosed embodiment also provides a device for restoring an image based on an image data packet. Figure 5 FIG. 1 is a schematic diagram of a structure of a device for restoring an image based on an image data packet provided by an embodiment of the present disclosure. Figure 5 As shown, the device 500 for restoring an image based on an image data packet includes an unpacking unit 501, a canvas construction and division unit 502, a decoding unit 503, a filling position determination unit 504 and a filling unit 505.

[0196] The unpacking unit 501 is used to unpack the acquired image data packets to determine the size of the target image, the total number of data packets N, the current packet number and the current encoded data.

[0197] The canvas construction and division unit 502 is used to construct a canvas to be filled according to the size of the target image and the preset sub-block size, divide the canvas to be filled into a plurality of areas to be filled according to the preset sub-block size, and determine a third number of each area to be filled;

[0198] The decoding unit 503 is used to decode the current coded data to obtain a plurality of sub-block coded data and second serial numbers of the sub-block coded data, and to decode each sub-block coded data to obtain a corresponding sliced ​​sub-block.

[0199] The filling position determination unit 504 is used to restore the first number of each sliced ​​sub-block based on the current package number and the second number of the sub-block encoding data; determine the area to be filled corresponding to the third number that is the same as the first number as the target filling area; or, based on the third number, determine the allocation package number corresponding to each area to be filled by a preset uniform allocation method, and determine the fourth number of each area to be filled under the corresponding package number; determine the area to be filled whose corresponding allocation package number is the same as the current package number and whose fourth number is the same as the second number as the target filling area.

[0200] The filling unit 505 is used to fill the corresponding cut sub-blocks into the corresponding area to be filled.

[0201] In some embodiments, the filling position determination unit 504 uses N as the divisor and the third number as the dividend to determine the allocation package number corresponding to each area to be filled, and determines the fourth number of each cut sub-block under the corresponding allocation package number according to the size of the third number of each area to be filled.

[0202] In some embodiments, the padding position determination unit 504 determines a multiplication coefficient based on the second number of the sub-block coded data; and takes the sum of a result of multiplying the multiplication coefficient by N and the current packet number as the first number.

[0203] In some embodiments, the unpacking unit 501 simultaneously obtains the quality factor parameter during the unpacking process of the acquired image data packet. The decoding unit 503 adjusts the pre-selected fixed quantization matrix based on the quality factor parameter to obtain the target quantization matrix; then performs inverse quantization processing based on the elements in the target quantization matrix and the elements at the corresponding positions of the sub-block coded data to obtain the second frequency spectrum value matrix; performs inverse discrete cosine transform based on the second frequency spectrum value matrix to obtain the corresponding sliced ​​sub-blocks.

[0204] In some embodiments, the filling unit 505 is also used to determine whether there is an incorrectly filled area in the area to be filled, where the incorrectly filled area is an area where the corresponding sliced ​​sub-block is not correctly filled; determine whether there is an incorrectly filled area, and determine the most recently filled area according to the position of the incorrectly filled area, where the most recently filled area is an area that has been filled with the corresponding sliced ​​sub-block; and fill the incorrectly filled area with the sliced ​​sub-block corresponding to the most recently filled area as temporary filling data.

[0205] The embodiment of the present disclosure also provides a computing device for implementing the aforementioned method. Figure 6 Schematic diagram of the structure of the computing device provided by the embodiment of the present disclosure. Figure 6 , which shows a schematic diagram of the structure of a computing device 600 suitable for implementing the embodiments of the present disclosure. Figure 6The computing device shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0206] like Figure 6 As shown, the computing device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory ROM 602 or a program loaded from a storage device 608 to a random access memory RAM 603. Various programs and data required for the operation of the computing device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output I / O interface 605 is also connected to the bus 604.

[0207] Typically, the following devices may be connected to the I / O interface 605: input devices 605 including, for example, a touch screen, a touch pad, a camera, a microphone, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication devices 609 may allow the computing device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The computing device 600 is shown with various devices, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0208] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0209] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable storage medium, a computer-readable signal medium, or any combination of the above two.

[0210] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0211] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0212] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0213] The computer-readable medium may be included in the computing device, or may exist independently without being incorporated into the computing device.

[0214] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the tester's computer, partially on the tester's computer, as a separate software package, partially on the tester's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the tester's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0215] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system based on hardware that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0216] The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The name of the unit does not constitute a limitation on the unit itself in some cases. The functions described above in this article may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: field programmable gate array (FPGA), application specific integrated circuit (ASIC), application specific standard product (ASSP), system on chip (SOC), complex programmable logic device (CPLD), etc.

[0217] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating an image data packet, characterized in that: include: Segmenting the target image according to a preset sub-block size to obtain segmented sub-blocks, and determining a first number for each segmented sub-block; Determine the number N of image data packets and the packet number of each image data packet according to the upper limit of the data packet byte of the selected transmission mode and the size of the target image; Based on the first number, determine the package number corresponding to each sliced ​​sub-block by a preset uniform distribution method, and determine the second number of each sliced ​​sub-block under the corresponding package number; Performing encoding processing on each sliced ​​sub-block to obtain corresponding sub-block encoding data, and sorting and reprocessing the sub-block encoding data corresponding to the sliced ​​sub-blocks with the same package number according to the second number to determine the target encoding data; Constructing N image data packets, writing the size of the target image, the number N, the corresponding packet number and the target encoded data into one image data packet respectively; the image data packet is transmitted using the selected transmission method; Before determining the data packet to be formed according to the upper limit of the data packet byte of the selected transmission mode and the size of the target image, the method further includes: Determining a quality factor parameter, wherein the quality factor parameter is used to determine a compression degree when compression encoding is performed on the sliced ​​sub-blocks; Determining the number of image data packets according to the upper limit of the data packet bytes of the selected transmission mode and the size of the target image includes: determining the number of image data packets according to the upper limit of the data packet bytes of the selected transmission mode, the size of the target image and the quality factor parameter; The encoding process is performed on each sliced ​​sub-block to obtain corresponding sub-block encoding data, including: Performing discrete cosine transform on the sliced ​​sub-blocks to obtain a first frequency spectrum value matrix; Adjusting a preselected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix, wherein a size of the fixed quantization matrix is ​​the same as a size of the first frequency spectrum value matrix; Using the elements in the target quantization matrix to quantize the elements at corresponding positions in the first frequency spectrum value matrix to obtain the sub-block coded data; The step of writing the size of the target image, the number N, the corresponding packet number and the target coded data into each image data packet comprises: Writing the size of the target image, the total number of data packets N, the quality factor parameter, the corresponding packet number and the target encoded data into each image data packet; The method of adjusting the pre-selected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix comprises: using Calculating and obtaining the value of each element of the target quantization matrix to construct the target quantization matrix; The step of using the elements in the target quantization matrix to quantize the elements at corresponding positions in the first frequency spectrum value matrix to obtain the sub-block coded data comprises: using Quantizing the elements at the corresponding positions to obtain the sub-block coded data; in The target quantization matrix The elements at is the quality factor parameter, For a fixed quantization matrix The elements at is the first frequency spectrum value matrix in The elements at Encode data for the sub-block in The element at .

2. The method for generating an image data packet according to claim 1, characterized in that: The step of determining the package numbers corresponding to the sliced ​​sub-blocks by using a preset uniform distribution method includes: With N as the divisor and the first number as the dividend, a remainder operation method is used to determine the package number corresponding to each sliced ​​sub-block.

3. The method for generating an image data packet according to claim 2, characterized in that: The determining of the second number of each sliced ​​sub-block under the corresponding package number includes: According to the size of the first number of each sliced ​​sub-block, the second number of each sliced ​​sub-block under the corresponding package number is determined.

4. A method for restoring an image based on an image data packet, characterized in that: include: Unpack the acquired image data packets to determine the size of the target image, the total number of data packets N, the current packet number and the current encoded data; Constructing a canvas to be filled according to the size of the target image and a preset sub-block size, dividing the canvas to be filled into a plurality of areas to be filled according to the preset sub-block size, and determining a third number of each area to be filled; Decoding the current coded data to obtain a plurality of sub-block coded data and second serial numbers of the sub-block coded data, and decoding each of the sub-block coded data to obtain a corresponding cut sub-block; Based on the current packet number and the second number of the sub-block coded data, restore the first number of each sliced ​​sub-block corresponding to each sub-block coded data; determine the to-be-filled area corresponding to the third number that is the same as the first number as the target filling area; or, Based on the third number, determine the allocation package number corresponding to each area to be filled by a preset uniform allocation method, and determine the fourth number of each area to be filled under the corresponding package number; determine the area to be filled whose corresponding allocation package number is the same as the current package number and whose fourth number is the same as the second number as the target filling area; Filling the corresponding cut sub-blocks into the corresponding target filling area; The image data packet also includes a quality factor parameter; the method further includes: in the process of unpacking the acquired image data packet, simultaneously acquiring the quality factor parameter; Adjusting a preselected fixed quantization matrix based on the quality factor parameter to obtain a target quantization matrix; The decoding process is performed on each of the sub-block coded data to obtain corresponding sliced ​​sub-blocks, including: Performing inverse quantization processing based on the elements in the target quantization matrix and the elements at corresponding positions of the sub-block coded data to obtain a second frequency spectrum value matrix; An inverse discrete cosine transform is performed based on the second frequency spectrum value matrix to obtain corresponding sliced ​​subblocks.

5. The method for restoring an image based on an image data packet according to claim 4, characterized in that: The step of determining the allocation package numbers corresponding to the respective to-be-filled areas based on the third number by a preset uniform allocation method includes: Taking N as the divisor and the third number as the dividend, the allocation package number corresponding to each of the to-be-filled areas is determined.

6. The method for restoring an image based on an image data packet according to claim 5, characterized in that: The step of determining the fourth number of each to-be-filled area under the corresponding package number includes: According to the order of the third numbers of the areas to be filled, the fourth numbers of the cut sub-blocks under the corresponding allocation package numbers are determined.

7. The method for restoring an image based on an image data packet according to claim 4, characterized in that: Restoring the first number of each sliced ​​sub-block corresponding to each sub-block coded data based on the current packet number and the second number of the sub-block coded data includes: Determine a multiple coefficient based on a second number of the sub-block coded data; The sum of the result of multiplying the multiple coefficient by N and the current package number is taken as the first number.

8. The method for restoring an image based on an image data packet according to any one of claims 4 to 7, characterized in that: After filling the sliced ​​sub-blocks into the corresponding target filling area, the following steps are also included: Determine whether there is an incorrectly filled area in the area to be filled, wherein the incorrectly filled area is an area in which the corresponding cut sub-block is not correctly filled; Determining that there is the incorrectly filled area, and determining the most recently filled area according to the position of the incorrectly filled area, wherein the most recently filled area is an area that has been filled with the corresponding cut sub-block; The sliced ​​sub-blocks corresponding to the most recently filled area are used as temporary filling data to fill the incorrectly filled area.

9. A computing device, characterized in that It comprises a processor and a memory, wherein the memory is used to store a computer program; when the computer program is loaded by the processor, the processor executes the method for generating an image data packet as described in any one of claims 1 to 3 and / or the method for restoring an image based on an image data packet as described in any one of claims 4 to 8.