Image processing method, device, equipment, storage medium and program product

Abstract

The embodiment of the present application provides an image processing method, device, equipment, storage medium and program product, the method comprising: receiving coding information sent by a server; the coding information includes a coded image corresponding to the original image; obtaining an operation instruction; determining an image to be decoded in the coded image according to the operation instruction and the coding information; the image to be decoded is a coded image corresponding to a target original image in a marked area in the original image; decoding the image to be decoded to obtain a target original image, and displaying the target original image in the marked area. The image processing method, device, equipment, storage medium and program product provided by the embodiment of the present application are used to save memory space of a terminal device, avoid freezing or crashing of information applications, etc.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, equipment, storage medium and program product. Background Art

[0002] Currently, information applications can include long images with rich text and graphic content, and terminal devices can display the long images included in the information applications to users.

[0003] In the related technology, the terminal device corresponding to the information application can send encoded information corresponding to the long image to the terminal device. When the user browses the information application through the terminal device, the terminal device can decode the encoded information, obtain the long image, and then display the long image to the user.

[0004] In actual applications, users usually only browse part of the images in a long picture (such as the beginning part of the image). Therefore, if the terminal device decodes the encoded information to obtain a long picture, it will waste more memory space of the terminal device, and then cause the information application to freeze or crash. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, apparatus, device, storage medium and program product to solve the problem that a terminal device wastes a large amount of memory space, thereby causing information applications to freeze or crash.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to a terminal device, and the method includes:

[0007] Receiving the coded information sent by the server; the coded information includes a coded image corresponding to the original image;

[0008] Get operation instructions;

[0009] According to the operation instruction and the coding information, an image to be decoded is determined in the coded image; the image to be decoded is a coded image corresponding to a target original image in a calibrated area in the original image;

[0010] The image to be decoded is decoded to obtain the target original image, and the target original image is displayed in the calibration area.

[0011] Optionally, the operation instruction includes a starting position, a first height and a first width of the calibration area, and the coding information includes a sequence parameter set; determining the image to be decoded in the coded image according to the operation instruction and the coding information includes: determining whether the starting position, the first height and the first width are all preset values;

[0012] If so, the sequence parameter set is parsed to obtain decoding information, which includes the second height and second width of the encoded image; and the image to be decoded is determined in the encoded image based on the starting position, the first height, the first width, the second height and the second width.

[0013] Optionally, determining the image to be decoded in the encoded image according to the starting position, the first height, the first width, the second height and the second width includes: judging whether the calibration area is located in the encoded image according to the starting position, the first height, the first width, the second height and the second width;

[0014] If yes, then determine the image to be decoded in the encoded image according to the starting position, the first height and the second height.

[0015] Optionally, judging whether the calibration area is located in the encoded image according to the starting position, the first height, the first width, the second height, and the second width includes:

[0016] If the starting position is located in the encoded image, the first height is smaller than the second height, and the first width is smaller than the second width, then the calibration area is located in the encoded image;

[0017] Otherwise, the marked area is not in the encoded image.

[0018] Optionally, determining the image to be decoded in the encoded image according to the starting position, the first height and the second height includes: determining the area to be decoded according to the starting position, the first height and the second height; and determining the image in the area to be decoded in the encoded image as the image to be decoded.

[0019] Optionally, the starting position includes a height starting position; based on the starting position, the first height and the second height, determining the area to be decoded, including: determining the minimum boundary value based on the height starting position; determining the maximum boundary value based on the first height, the second height and the height starting position; and determining the CTU row from the minimum boundary value to the maximum boundary value in the encoded image as the image to be decoded.

[0020] Optionally, the minimum boundary value is determined based on the starting height position, including: shifting the starting height position right by a third preset value; the third preset value is a preset value corresponding to the encoding method corresponding to the encoded image; and determining the maximum value of the first preset value and the difference between the starting height position after right shifting and the second preset value as the minimum boundary value.

[0021] Optionally, the maximum boundary value is determined based on the first height, the second height and the starting height position, including: shifting the third preset value left by the tenth preset value; determining the difference between the sum of the starting height position, the first height, the third preset value after the left shift, and the tenth preset value; shifting the difference right by the third preset value; and determining the minimum value between the second height and the difference after the right shift as the maximum boundary value.

[0022] Optionally, decoding the image to be decoded to obtain the target original image includes: acquiring decoding reference information; and decoding the image to be decoded according to the decoding reference information to obtain the target original image.

[0023] Optionally, the decoding information includes a switch identifier of the filter processing tool; obtaining the decoding reference information includes: judging whether the filter processing tool is turned on according to the switch identifier;

[0024] If yes, determining a fourth preset value of CTU rows located above the image to be decoded and a fifth preset value of CTU rows located below the image to be decoded in the encoded image as a first reference decoded image, and determining decoding reference information according to the first reference decoded image;

[0025] Otherwise, the sixth preset value of CTU rows located above the image to be decoded and the seventh preset value of CTU rows located below the image to be decoded in the encoded image are determined as the second reference decoded image, and the decoding reference information is determined based on the second reference decoded image.

[0026] Optionally, the fourth preset value of CTU rows located above the image to be decoded includes the first CTU row and the second CTU row, and the fifth preset value of CTU rows located below the image to be decoded includes the third CTU row; according to the first reference decoded image, the decoding reference information is determined, including: respectively decoding the first CTU row, the second CTU row and the third CTU row to obtain a first original pixel set, a second original pixel set and a third original pixel set; using a filtering processing tool to perform in-loop filtering on the first original pixel set and the second original pixel set, respectively, to obtain a first processed pixel set and a second processed pixel set; according to the first processed pixel set, the second processed pixel set and the third original pixel set, the decoding reference information is determined.

[0027] Optionally, the first processed pixel set and the second processed pixel set respectively include N rows of pixel data, where N is an integer greater than or equal to 2; decoding reference information is determined based on the first processed pixel set, the second processed pixel set and the third original pixel set, including: using the last M rows of pixel data in the N rows of pixel data in the second processed pixel set to replace the last M rows of pixel data in the N rows of pixel data in the first processed pixel set, to obtain a third processed pixel set corresponding to the first processed pixel set; M is a positive integer less than N; and the second processed pixel set, the third processed pixel set and the third original pixel set are determined as decoding reference information.

[0028] Optionally, the sixth preset value of CTU rows located above the image to be decoded includes a first CTU row, and the seventh preset value of CTU rows located below the image to be decoded includes a second CTU row; according to the second reference decoded image, decoding reference information is determined, including: respectively decoding the first CTU row and the second CTU row to obtain a first original pixel set and a second original pixel set; the first original pixel set includes N rows of original pixel data; using the last M rows of original pixel data of the N rows of original pixel data in the first original pixel set to replace the first M rows of original pixel data of the N rows of original pixel data in the first original pixel set to obtain a second pixel set; and determining the second pixel set and the second original pixel set as decoding reference information.

[0029] Optionally, according to the decoding reference information, the image to be decoded is decoded to obtain a target original image, including: according to the decoding reference information, the image to be decoded is decoded to obtain reconstructed pixel values; and the reconstructed pixel values ​​are stored in a preset area to obtain a reconstructed image; the preset area has the same size as the image to be decoded; and the target original image is determined in the reconstructed image according to the starting position, the first height, and the first width.

[0030] Optionally, a target original image is determined in a reconstructed image according to a starting position, a first height, and a first width, including: determining a top offset as the difference between an initial height position and a left shift value after a third preset value is shifted left by a minimum boundary value; determining a sum of a maximum boundary value and a ninth preset value, and determining a bottom offset as the minimum value of a value after the third preset value is shifted left by the sum value, and a difference between a second height and the initial height position and the first height; determining a left offset as the width starting position of a starting position; determining a right offset as the difference between a second width and the sum of a left offset and a first width; a top offset, a bottom offset, a left offset, and a right offset indicate an image to be deleted in the reconstructed image; and deleting the image to be deleted in the reconstructed image to obtain a target original image.

[0031] Optionally, the area to be decoded is determined according to the starting position, the first height and the second height, including: parsing the image parameter set included in the encoding information to obtain the position area and the number of slices of the image slices included in the encoded image; if the number of slices is equal to a preset value, the area to be decoded is determined according to the starting position, the first height and the second height; if the number of slices is not equal to the preset value, the image slices that do not overlap with the calibrated area are deleted in the encoded image, and the area to be decoded is determined in the encoded image after the image slices are deleted according to the starting position, the first height and the second height.

[0032] Optionally, the operation instruction is an instruction corresponding to a sliding operation; according to the operation instruction and the coding information, the image to be decoded is determined in the encoded image, including: detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining a calibrated area in the current display interface; according to the calibrated area and the coding information, determining the image to be decoded corresponding to the calibrated area in the encoded image.

[0033] In a second aspect, an embodiment of the present application provides an image processing device, which is applied to a terminal device, and the device includes:

[0034] A receiving module, used for receiving the coded information sent by the server; the coded information includes a coded image corresponding to the original image;

[0035] An acquisition module, used for acquiring operation instructions;

[0036] A determination module, used to determine the image to be decoded in the encoded image according to the operation instruction and the encoding information; the image to be decoded is the encoded image corresponding to the target original image in the marked area of ​​the original image;

[0037] A processing module, used for decoding the image to be decoded to obtain a target original image;

[0038] The display module is used to display the original image of the target in the calibration area.

[0039] Optionally, the operation instruction includes a starting position, a first height and a first width of the calibration area, and the encoding information includes a sequence parameter set; the determination module is specifically used to: determine whether the starting position, the first height and the first width are all preset values;

[0040] If so, the sequence parameter set is parsed to obtain decoding information, which includes the second height and second width of the encoded image; and the image to be decoded is determined in the encoded image based on the starting position, the first height, the first width, the second height and the second width.

[0041] Optionally, the determination module is specifically used to: determine whether the calibration area is located in the encoded image according to the starting position, the first height, the first width, the second height and the second width;

[0042] If yes, then determine the image to be decoded in the encoded image according to the starting position, the first height and the second height.

[0043] Optionally, the determination module is specifically used for:

[0044] If the starting position is located in the encoded image, the first height is smaller than the second height, and the first width is smaller than the second width, then the calibration area is located in the encoded image;

[0045] Otherwise, the marked area is not in the encoded image.

[0046] Optionally, the determination module is specifically used to: determine the area to be decoded according to the starting position, the first height and the second height; and determine the image in the area to be decoded in the encoded image as the image to be decoded.

[0047] Optionally, the starting position includes a height starting position; the determination module is specifically used to: determine a minimum boundary value based on the height starting position; determine a maximum boundary value based on the first height, the second height and the height starting position; and determine a CTU row from the minimum boundary value to the maximum boundary value in the encoded image as an image to be decoded.

[0048] Optionally, the determination module is specifically used to: shift the starting height position right by a third preset value; the third preset value is a preset value corresponding to the encoding method corresponding to the encoded image; and determine the maximum value of the first preset value and the difference between the starting height position after right shifting and the second preset value as the minimum boundary value.

[0049] Optionally, the determination module is specifically used to: shift the third preset value left by the tenth preset value; determine the difference between the sum of the height starting position, the first height, the third preset value after the left shift, and the tenth preset value; shift the difference right by the third preset value; and determine the minimum value between the second height and the difference after the right shift as the maximum boundary value.

[0050] Optionally, the processing module is specifically used to: obtain decoding reference information; and decode the image to be decoded according to the decoding reference information to obtain a target original image.

[0051] Optionally, the decoded information includes a switch identifier of the filter processing tool; the processing module is specifically used to: determine whether the filter processing tool is turned on according to the switch identifier;

[0052] If yes, determining a fourth preset value of CTU rows located above the image to be decoded and a fifth preset value of CTU rows located below the image to be decoded in the encoded image as a first reference decoded image, and determining decoding reference information according to the first reference decoded image;

[0053] Otherwise, the sixth preset value of CTU rows located above the image to be decoded and the seventh preset value of CTU rows located below the image to be decoded in the encoded image are determined as the second reference decoded image, and the decoding reference information is determined based on the second reference decoded image.

[0054] Optionally, the fourth preset value of CTU rows located above the image to be decoded includes a first CTU row and a second CTU row, and the fifth preset value of CTU rows located below the image to be decoded includes a third CTU row; the processing module is specifically used to: decode the first CTU row, the second CTU row, and the third CTU row, respectively, to obtain a first original pixel set, a second original pixel set, and a third original pixel set; use a filtering processing tool to perform in-loop filtering on the first original pixel set and the second original pixel set, respectively, to obtain a first processed pixel set and a second processed pixel set; determine decoding reference information based on the first processed pixel set, the second processed pixel set, and the third original pixel set.

[0055] Optionally, the first processed pixel set and the second processed pixel set respectively include N rows of pixel data, where N is an integer greater than or equal to 2; the processing module is specifically used to: use the last M rows of pixel data in the N rows of pixel data in the second processed pixel set to replace the last M rows of pixel data in the N rows of pixel data in the first processed pixel set, to obtain a third processed pixel set corresponding to the first processed pixel set; M is a positive integer less than N; and determine the second processed pixel set, the third processed pixel set and the third original pixel set as decoding reference information.

[0056] Optionally, the sixth preset value of CTU rows located above the image to be decoded includes a first CTU row, and the seventh preset value of CTU rows located below the image to be decoded includes a second CTU row; the processing module is specifically used to: decode the first CTU row and the second CTU row respectively to obtain a first original pixel set and a second original pixel set; the first original pixel set includes N rows of original pixel data; use the last M rows of original pixel data of the N rows of original pixel data in the first original pixel set to replace the first M rows of original pixel data of the N rows of original pixel data in the first original pixel set to obtain a second pixel set; and determine the second pixel set and the second original pixel set as decoding reference information.

[0057] Optionally, the processing module is specifically used to: decode the image to be decoded according to the decoding reference information to obtain reconstructed pixel values; and store the reconstructed pixel values ​​in a preset area to obtain a reconstructed image; the preset area has the same size as the image to be decoded; and determine the target original image in the reconstructed image according to the starting position, the first height, and the first width.

[0058] Optionally, the processing module is specifically used to: determine the difference between the initial height position and the left shift value after the third preset value is shifted left by the minimum boundary value as the top offset; determine the sum of the maximum boundary value and the ninth preset value, and determine the minimum value of the value after the third preset value is shifted left by the sum value, and the difference between the second height and the initial height position and the first height as the bottom offset; determine the width starting position of the starting position as the left offset; determine the difference between the second width and the sum of the left offset and the first width as the right offset; the top offset, bottom offset, left offset and right offset indicate the image to be deleted in the reconstructed image; delete the image to be deleted in the reconstructed image to obtain the target original image.

[0059] Optionally, the determination module is specifically used to: parse the image parameter set included in the encoding information to obtain the position area and the number of slices of the image slices included in the encoded image; if the number of slices is equal to a preset value, determine the area to be decoded based on the starting position, the first height, and the second height; if the number of slices is not equal to the preset value, delete the image slices that do not overlap with the calibrated area in the encoded image, and determine the area to be decoded in the encoded image after deleting the image slices based on the starting position, the first height, and the second height.

[0060] Optionally, the operation instruction is an instruction corresponding to a sliding operation; the determination module is specifically used to: detect a pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determine a calibration area in the current display interface; and determine an image to be decoded corresponding to the calibration area in the encoded image according to the calibration area and the encoding information.

[0061] In a third aspect, an embodiment of the present application further provides a terminal device, comprising: a processor, and a memory communicatively connected to the processor;

[0062] Memory stores computer-executable instructions;

[0063] The processor executes the computer-executable instructions stored in the memory to implement any method in the first aspect.

[0064] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement any one of the methods in the first aspect.

[0065] In a fifth aspect, an embodiment of the present application provides a computer program product, including: a computer program; and a method for implementing any one of the first aspects when the computer program is executed by a processor.

[0066] The embodiment of the present application provides an image processing method, apparatus, device, storage medium and program product, wherein the image processing method comprises: receiving coding information sent by a server; the coding information includes a coded image corresponding to the original image; obtaining an operation instruction; determining an image to be decoded in the coded image according to the operation instruction and the coding information; the image to be decoded is a coded image corresponding to a target original image in a marked area in the original image; decoding the image to be decoded to obtain the target original image, and displaying the target original image in the marked area. In the above method, according to the operation instruction and the coding information, the image to be decoded is determined in the coded image, the image to be decoded is decoded to obtain the target original image, and the target original image is displayed in the marked area, so that the terminal device can only decode part of the coded image (i.e., the image to be decoded) in the coded image to obtain the target original image that the user wants to browse, thereby saving the memory space of the terminal device and avoiding the information application from freezing or crashing. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0068] Figure 1 A schematic diagram of a scenario provided for an embodiment of the present application;

[0069] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0070] Figure 3 The method flow for determining the image to be decoded provided in the embodiment of the present application Figure 1 ;

[0071] Figure 4 A schematic diagram for determining an image to be decoded is provided for an embodiment of the present application;

[0072] Figure 5 Another schematic diagram for determining an image to be decoded is provided for an embodiment of the present application;

[0073] Figure 6 A flow chart of a method for obtaining a target original image provided in an embodiment of the present application;

[0074] Figure 7 A schematic diagram of determining decoding reference information provided by an embodiment of the present application;

[0075] Figure 8 A schematic diagram of another method for determining decoding reference information provided by an embodiment of the present application;

[0076] Fig. 9 A schematic diagram of obtaining a reconstructed image provided in an embodiment of the present application;

[0077] Fig.10 A schematic diagram of determining a target original image in a reconstructed image is provided for an embodiment of the present application;

[0078] Fig.11 The method flow for determining the image to be decoded provided in the embodiment of the present application Figure 2 ;

[0079] Fig.12 A schematic diagram of a coded image including multiple tiles provided in an embodiment of the present application;

[0080] Fig.13 A schematic diagram of a coded image including multiple slices provided in an embodiment of the present application;

[0081] Fig.14 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0082] Fig.15 A hardware schematic diagram of a terminal device provided in an embodiment of the present application.

[0083] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0084] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0085] First, the prior art is briefly described.

[0086] In the prior art, when a user browses a long image (or high-definition video) included in an information application through a terminal device, the terminal device decodes the encoding information corresponding to the long image (or high-definition video) sent by the terminal device to obtain the completed long image (or high-definition video) and displays the completed long image (or high-definition video).

[0087] In practice, through investigation and research on users' browsing of long images (or high-definition videos), it was found that users usually browse only the beginning or middle part of the long image (or high-definition video) with a high probability. Therefore, if the method of the terminal device in the prior art is used to decode the encoded information to obtain the completed long image (or high-definition video), the terminal device will waste more memory space, and then cause the information application to freeze or crash.

[0088] In the present application, in order to save memory space of the terminal device and thus avoid freezing or crashing of the information application, the inventors have come up with the idea of: during the user's browsing process, the terminal device only decodes the encoded image to obtain the part of the image that the user wants to browse in the long image (or high-definition video), thereby saving the memory space of the terminal device and thus avoiding freezing or crashing of the information application.

[0089] Combine the following Figure 1 , the application scenario of the image processing method provided in the embodiment of the present application is described.

[0090] Figure 1 A schematic diagram of a scenario provided in an embodiment of the present application. Figure 1 As shown, it includes: a terminal device and at least one terminal device.

[0091] An information application may be installed in a terminal device, and the terminal device is a terminal device corresponding to the information application.

[0092] In order to save transmission traffic for sending the original image from one terminal device to another, the terminal device usually encodes the visible original image, obtains encoding information corresponding to the original image, and sends the encoding information to the terminal device.

[0093] After receiving the coding information, the terminal device determines the image to be decoded according to the operation instruction and the coding information, and then decodes the image to be decoded to obtain the target original image and displays the target original image, wherein the target original image is the part of the original image that the user wants to browse.

[0094] In the above process, the terminal device determines the image to be decoded according to the operation instruction and the coding information, and then decodes the image to be decoded to obtain the target original image, which can save the memory space of the terminal device and avoid the information application from freezing or crashing.

[0095] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0096] Figure 2 Flow chart of the image processing method provided in the embodiment of the present application. Figure 2 As shown, the method includes:

[0097] S201, receiving coding information sent by a server; the coding information includes a coded image corresponding to the original image.

[0098] Optionally, the execution subject of the embodiment of the present application may be a terminal device, or an image processing device arranged in the terminal device, and the image processing device may be implemented by a combination of software and / or hardware.

[0099] The above-mentioned terminal device can be a wireless terminal or a wired terminal. The wireless terminal can be a device that provides voice and / or other business data connectivity to the user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. The wireless terminal can communicate with one or more core network devices via a radio access network (Radio Access Network, referred to as RAN). The wireless terminal can be a mobile terminal, such as a mobile phone (or a "cellular" phone, or a smart phone) and a computer with a mobile terminal. For example, the wireless terminal can also be a personal communication service (Personal Communication Service, referred to as PCS) phone, a cordless phone, a session initiation protocol (Session Initiation Protocol, referred to as SIP) phone, a wireless local loop (Wireless Local Loop, referred to as WLL) station, a personal digital assistant (Personal Digital Assistant, referred to as PDA) and other devices. The wireless terminal can also be called a system, a user terminal (User Terminal), a user device (User Device or User Equipment), which is not limited here. Optionally, the above-mentioned terminal device can also be a tablet computer and other devices.

[0100] The coding information may include a coded image corresponding to the original image, a slice parameter set (SPS), and a picture parameter set (PPS).

[0101] The original image is a long image or a high-definition image visible to the user, for example, the original image is a visible image in a format such as RGB or YUV.

[0102] S202, obtaining an operation instruction.

[0103] The operation instruction may be an instruction corresponding to the user's sliding operation.

[0104] For example, when the terminal device is a smart phone, the sliding operation may be a sliding operation performed by the user on a touch screen of the terminal device.

[0105] Optionally, the sliding operation may be continuous or paused.

[0106] S203: Determine an image to be decoded in the encoded image according to the operation instruction and the encoding information.

[0107] The image to be decoded is a coded image corresponding to the target original image in the calibrated area of ​​the original image.

[0108] Optionally, the size of the image to be decoded may be the same as or different from the size of the calibration area.

[0109] When the size of the image to be decoded is different from the size of the calibration area, the size of the image to be decoded is larger than the size of the calibration area. For example, the height of the image to be decoded is equal to the first height of the calibration area, and the width of the image to be decoded is greater than or equal to the first width of the calibration area. For example, the height of the image to be decoded is greater than the first height of the calibration area, and the width of the image to be decoded is greater than or equal to the first width of the calibration area.

[0110] Optionally, S203 may be implemented in the following two ways (way B1 and way B2).

[0111] Mode B1, determining whether the operation instruction includes a starting position, a first height, and a first width of the calibration area are all preset values;

[0112] If yes, the SPS in the coded information is parsed to obtain decoded information; the decoded information includes the second height and the second width of the coded image; and the image to be decoded is determined in the coded image according to the starting position, the first height, the first width, the second height and the second width;

[0113] Otherwise, perform full frame decoding to obtain the original image.

[0114] The preset value may be 0 or other values, and the value of the preset value is not limited here.

[0115] Optionally, when the starting position is located in the encoded image, the first height is less than or equal to the second height, and the first width is less than or equal to the second width, it is determined that the image to be decoded is determined in the encoded image.

[0116] The first height being less than or equal to the second height indicates that a Coding Tree Unit (CTU) row corresponding to the first height in the calibration region is included in a CTU row corresponding to the second height in the coded image.

[0117] The first width being less than or equal to the second width indicates that a CTU row corresponding to the first width in the calibration area is included in a CTU row corresponding to the second width in the encoded image.

[0118] Specifically, the method of performing full-frame decoding to obtain the original image can be referred to in the prior art, which will not be described in detail here.

[0119] Mode B2, the operation instruction is an instruction corresponding to a sliding operation; S203 specifically includes: detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining a calibration area in the current display interface; and determining an image to be decoded corresponding to the calibration area in the coded image according to the calibration area and the encoding information.

[0120] Furthermore, the above-mentioned method B1 can be used to implement the method B2 of determining the image to be decoded corresponding to the marked area in the coded image according to the marked area and the coding information.

[0121] S204, decoding the image to be decoded to obtain a target original image, and displaying the target original image in the calibrated area.

[0122] Optionally, when the size of the image to be decoded is the same as the size of the calibration area, the image to be decoded is decoded to obtain the target original image. Optionally, when the size of the image to be decoded is different from the size of the calibration area, the image to be decoded is decoded to obtain a decoded image, and the decoded image is cropped according to the size of the calibration area to obtain the target original image.

[0123] Figure 2 The image processing method provided in the embodiment includes: determining the image to be decoded in the encoded image according to the operation instruction and the encoding information, decoding the image to be decoded to obtain the target original image, and displaying the target original image in the calibrated area, so that the terminal device can only decode part of the encoded image (i.e., the image to be decoded) in the encoded image to obtain the target original image that the user wants to browse, thereby saving the memory space of the terminal device, avoiding the information application from being stuck or crashing, etc., improving the performance of the information application in the terminal device, and thus improving the user's application experience.

[0124] Based on the above embodiments, Figure 3 Taking the case where the size of the image to be decoded is larger than the size of the calibration area as an example, the specific execution process of determining the image to be decoded in the encoded image according to the starting position, the first height, the first width, the second height and the second width is described.

[0125] Figure 3 The method flow for determining the image to be decoded provided in the embodiment of the present application Figure 1 .like Figure 3 As shown, based on the above S203, the method includes:

[0126] S301, obtaining a starting position, a first height, a first width, a second height, and a second width.

[0127] Optionally, after the operation instruction is obtained, the starting position, the first height and the first width of the calibration area may be stored.

[0128] Optionally, after parsing the sequence parameter set included in the coding information to obtain the second height and the second width of the coded image, the second height and the second width may be stored.

[0129] When S301 is executed, the stored start position, first height, first width, second height, and second width may be obtained.

[0130] Optionally, after determining that the starting position, the first height, and the first width are not preset values, directly

[0131] S302: Determine whether the calibration area is located in the encoded image according to the starting position, the first height, the first width, the second height, and the second width.

[0132] If yes, execute S303, otherwise execute S304.

[0133] Optionally, if the starting position is located in the encoded image, the first height is smaller than the second height, and the first width is smaller than the second width, then the calibration area is located in the encoded image;

[0134] Otherwise, the marked area is not in the encoded image.

[0135] S303 , determining a to-be-decoded image in the encoded image according to the starting position, the first height, and the second height.

[0136] Optionally, the specific execution method of S303 may include the following two methods (method A1 and method A2).

[0137] Method A1, the starting position includes the height starting position; according to the height starting position, the minimum boundary value is determined; according to the first height, the second height and the height starting position, the maximum boundary value is determined; the CTU row from the minimum boundary value to the maximum boundary value in the encoded image is determined as the image to be decoded.

[0138] Optionally, the minimum boundary value may be determined by:

[0139] Shift the starting height position right by a third preset value; the third preset value is a preset value corresponding to the encoding method corresponding to the encoded image; and determine the minimum value of the first preset value and the difference between the starting height position after right shift and the second preset value as the minimum boundary value.

[0140] That is, the minimum boundary value can be obtained through the following formula (1):

[0141] Min_crop_ctu_row = max(F1, (Y0 >> C) - F2) (1);

[0142] Wherein, Min_crop_ctu_row is the minimum boundary value, max is the operation of taking the larger value, F1 is the first preset value, Y0 is the starting position of the height, >> is the right shift symbol, - is the minus sign, C is the third preset value, and F2 is the second preset value. Optionally, F1 can be 0 or other positive integers, and F2 can be 2 or other positive integers.

[0143] Optionally, the following method can be used to determine the maximum boundary value:

[0144] Shift the third preset value to the left by the tenth preset value; determine the difference between the sum of the starting position of the height, the first height, and the third preset value after left shift, and the tenth preset value; shift the difference to the right by the third preset value; determine the minimum value between the second height and the difference after right shift as the maximum boundary value.

[0145] That is, the maximum boundary value can be obtained through the following formula (2):

[0146] Max_crop_ctu_row = min(H2, (Y0 + H1 + (F3 << C) - F3) >> C) (2);

[0147] Wherein, Max_crop_ctu_row is the maximum boundary value, min is the operation of taking the smaller value, H2 is the second height, H1 is the first height, F3 is the tenth preset value, and << is the left shift symbol. Optionally, F3 can be 0 or other positive integers.

[0148] It should be noted that in the above formula (2), F3 << C means shifting the third preset value to the left by the tenth preset value, Y0 + H1 + (F3 << C) represents the above sum value, Y0 + H1 + (F3 << C) - F3 is the above difference, and (Y0 + H1 + (F3 << C) - F3) >> C means shifting the above difference to the right by the third preset value.

[0149] Figure 4 This is a schematic diagram provided by an embodiment of the present application for determining an image to be decoded. As Figure 4 shown, it includes: an encoded image 41, a calibration area 42, and an image to be decoded 43. Among them, the minimum boundary value above the calibration area 42 is obtained by using formula (1), and the maximum boundary value below the calibration area 42 is obtained by using formula (2). In Figure 4 the image to be decoded 43 includes: the CTU row where the minimum boundary value is located, the CTU row where the maximum boundary value is located, and the CTU rows between the minimum boundary value and the maximum boundary value.

[0150] Method A2, the starting position includes a height starting position; the encoded image that meets the first preset condition in the encoded image is determined as the image to be decoded, wherein the first preset condition is the first height, and the sum of the first preset height and the second preset height is less than the second height.

[0151] Figure 5 Another schematic diagram for determining an image to be decoded is provided for an embodiment of the present application. Figure 5 As shown, it includes: a coded image 51, a calibration area 52 and an image to be decoded 53. The first preset height is located above the first height, and the second preset height is located below the first height. Figure 5 In the example, the image 53 to be decoded includes: the CTU row spanned by the sum of the first preset height, the first height, and the second preset height.

[0152] S304, perform full frame decoding to obtain the original image.

[0153] exist Figure 3 In an embodiment, in the process of determining the image to be decoded in the encoded image based on the starting position, the first height and the second height, the CTU row from the minimum boundary value to the maximum boundary value in the encoded image can be determined as the image to be decoded, so that the image to be decoded includes the encoded image corresponding to the target original image, thereby improving the accuracy of the target original image obtained by decoding the image to be decoded.

[0154] Based on the above embodiments, Figure 6 The specific execution process of the above S204 is described below by taking the to-be-decoded image as an example of a CTU behavior from the minimum boundary value to the maximum boundary value in the encoded image.

[0155] Figure 6 The flowchart of the method for obtaining the target original image provided by the embodiment of the present application. Figure 6 As shown, the method includes:

[0156] S601: Obtain decoding reference information.

[0157] Optionally, the decoding information includes a switch identifier of a filtering processing tool; S601 specifically includes: judging whether the filtering processing tool is turned on according to the switch identifier; if so, determining the fourth preset value of CTU rows located above the image to be decoded and the fifth preset value of CTU rows located below the image to be decoded in the encoded image as the first reference decoded image, and determining the decoding reference information according to the first reference decoded image; otherwise, determining the sixth preset value of CTU rows located above the image to be decoded and the seventh preset value of CTU rows located below the image to be decoded in the encoded image as the second reference decoded image, and determining the decoding reference information according to the second reference decoded image.

[0158] Optionally, the filter processing tool may be any at least one of an adaptive loop filter tool (adaptive loop filter), a deblocking filter tool (deblocking filter), and a sample adaptive offset tool (SAO).

[0159] Optionally, when the filter processing tool includes a deblocking filter tool and a sample adaptive compensation tool, the decoded information includes a first switch identifier corresponding to the deblocking filter tool and a second switch identifier corresponding to the sample adaptive compensation tool.

[0160] Further, according to the first switch flag, it is determined whether the deblocking filter tool is turned on, and according to the second switch flag, it is determined whether the sample adaptive compensation tool is turned on.

[0161] For example, if the first switch identifier is an on identifier, it is determined that the deblocking filter tool is on; if the first switch identifier is an off identifier, it is determined that the deblocking filter tool is off.

[0162] Optionally, the fourth preset value may be 2, the fifth preset value may be 1, the sixth preset value may be 1, and the seventh preset value may be 1. Of course, the fourth preset value, the fifth preset value, the sixth preset value, and the seventh preset value may also be other positive integers.

[0163] Optionally, when the fourth preset value is 2 and the fifth preset value is 1, the fourth preset value of CTU rows located above the image to be decoded includes a first CTU row and a second CTU row, and the fifth preset value of CTU rows located below the image to be decoded includes a third CTU row; and determining the decoding reference information according to the first reference decoded image includes:

[0164] Decoding the first CTU row, the second CTU row, and the third CTU row respectively to obtain a first original pixel set, a second original pixel set, and a third original pixel set;

[0165] Using a filtering processing tool to perform in-loop filtering processing on the first original pixel set and the second original pixel set respectively, to obtain a first processed pixel set and a second processed pixel set;

[0166] Decoding reference information is determined according to the first processed pixel set, the second processed pixel set and the third original pixel set.

[0167] The first original pixel set is obtained by decoding the first CTU row, the second original pixel set is obtained by decoding the second CTU row, and the third original pixel set is obtained by decoding the third CTU row.

[0168] The first processed pixel set is obtained by performing in-loop filtering on the first original pixel set, and the second processed pixel set is obtained by performing in-loop filtering on the second original pixel set.

[0169] Optionally, when the filtering processing tool includes a deblocking filtering tool and a sample adaptive compensation tool, the deblocking filtering tool is used to perform in-loop filtering on the first original pixel set and the second original pixel set respectively to obtain the first processed pixel set and the second processed pixel set, including:

[0170] Using a deblocking filter tool, respectively performing in-loop filtering on the first original pixel set and the second original pixel set to obtain a first processed pixel set and a second processed pixel set; or,

[0171] Using a sample adaptive compensation tool, respectively performing in-loop filtering processing on the first original pixel set and the second original pixel set to obtain a first processed pixel set and a second processed pixel set; or,

[0172] The first original pixel set is processed by in-loop filtering using a deblocking filter tool to obtain a first processed pixel set, and the second original pixel set is processed by in-loop filtering using a sample adaptive compensation tool to obtain a second processed pixel set.

[0173] Optionally, the first processed pixel set and the second processed pixel set respectively include N rows of pixel data, where N is an integer greater than or equal to 2; and determining the decoding reference information according to the first processed pixel set, the second processed pixel set and the third original pixel set comprises:

[0174] The last M lines of pixel data in the N lines of pixel data in the second processing pixel set are used to replace the last M lines of pixel data in the N lines of pixel data in the first processing pixel set, so as to obtain a third processing pixel set corresponding to the first processing pixel set; M is a positive integer less than N;

[0175] The second processed pixel set, the third processed pixel set and the third original pixel set are determined as decoding reference information.

[0176] Optionally, N can be 64, 32, 16, 4, etc.

[0177] Combine the following Figure 7 A block diagram for determining decoding reference information based on a first reference decoded image in the present application is described.

[0178] Figure 7 A schematic diagram of determining decoding reference information provided by an embodiment of the present application. Figure 7 As shown, the first CTU row, the second CTU row and the third CTU row are decoded respectively to obtain the first original pixel set, the second original pixel set and the third original pixel set.

[0179] Loop filtering is performed on the first original pixel set and the second original pixel set respectively to obtain a first processed pixel set and a second processed pixel set.

[0180] A third processed pixel set is obtained according to the first processed pixel set and the second processed pixel set.

[0181] Then, the third processed pixel set, the second processed pixel set, and the third original pixel set are determined as decoding reference information.

[0182] Optionally, when the sixth preset value and the seventh preset value are both 1, the sixth preset value of CTU rows located above the image to be decoded includes a first CTU row, and the seventh preset value of CTU rows located below the image to be decoded includes a second CTU row;

[0183] Determining decoding reference information according to the second reference decoded image includes:

[0184] Decoding the first CTU row and the second CTU row respectively to obtain a first original pixel set and a second original pixel set; the first original pixel set and the second original pixel set respectively include N rows of original pixel data;

[0185] The last M lines of original pixel data in the N lines of original pixel data in the first original pixel set are used to replace the first M lines of original pixel data in the N lines of original pixel data in the first original pixel set, so as to obtain a second pixel set;

[0186] The second pixel set and the second original pixel set are determined as decoding reference information.

[0187] Combine the following Figure 8 A block diagram for determining decoding reference information based on a second reference decoded image in the present application is described. Figure 8 A schematic diagram of another method for determining decoding reference information provided by an embodiment of the present application. Figure 8 As shown, the first CTU row, the second CTU row and the third CTU row are decoded respectively to obtain a first original pixel set and a second original pixel. The second pixel set is obtained according to the first original pixel set. The second pixel set and the second original pixel are determined as decoding reference information.

[0188] S602: Decode the image to be decoded according to the decoding reference information to obtain a target original image.

[0189] Optionally, S602 specifically includes: decoding the image to be decoded according to the decoding reference information to obtain reconstructed pixel values; storing the reconstructed pixel values ​​in a preset area to obtain a reconstructed image; the preset area has the same size as the image to be decoded; and determining the target original image in the reconstructed image according to the starting position, the first height, and the first width.

[0190] Optionally, determining a target original image in the reconstructed image according to a starting position, a first height, and a first width includes: determining a top offset by taking a difference between a height starting position and a shifted value obtained by shifting a third preset value by a minimum boundary value to the left;

[0191] determining a bottom offset by taking a minimum value between a value obtained by shifting the sum of a third preset value shifted by a maximum boundary value to the left and a ninth preset value, and a difference between a second height and a sum of the height starting position, a first height;

[0192] determining a left offset by taking a width starting position of the starting position;

[0193] determining a right offset by taking a difference between a second width and a sum of the left offset and a first width; the top offset, the bottom offset, the left offset, and the right offset indicate an image to be deleted in the reconstructed image;

[0194] deleting the image to be deleted in the reconstructed image to obtain the target original image.

[0195] Optionally, the top offset can be determined by the following formula 3:

[0196] luma_offset_top = Y0 - (Min_crop_ctu_row << C) Formula 3;

[0197] where luma_offset_top is the top offset.

[0198] Optionally, the bottom offset can be determined by the following formula 4:

[0199] luma_offset_bottom = min(((Max_crop_ctu_row + 1) << C), H2 - (Y0 + H1)) (4);

[0200] where luma_offset_bottom is the bottom offset, and + is the plus sign.

[0201] Optionally, the left offset can be determined by the following formula 5:

[0202] luma_offset_left = X0 (5);

[0203] where luma_offset_left is the left offset, and X0 is the width starting position.

[0204] Optionally, the right offset can be determined by the following formula 6:

[0205] luma_offset_right=src->width-(luma_offset_left+W1) (6);

[0206] Among them, luma_offset_right is the right offset, and src->width indicates the second width.

[0207] Fig. 9 A schematic diagram of obtaining a reconstructed image provided in an embodiment of the present application. Figure 7 On the basis of Fig. 9 As shown, the third processed pixel set and the second processed pixel set are stored in sequence above the preset area, and the third original pixel set is stored below the preset area. The reconstructed pixel values ​​obtained by decoding the image to be decoded according to the decoding reference information determined by the third processed pixel set, the second processed pixel set and the third original pixel are stored in the preset area to obtain a reconstructed image.

[0208] The reconstructed image includes reconstructed pixel values.

[0209] Fig.10 A schematic diagram of determining a target original image in a reconstructed image is provided for an embodiment of the present application. Fig.10 As shown, it includes: a reconstructed image 1001, a target original image 1002, and an image to be deleted 1003.

[0210] The image 1003 is an image to be deleted in the reconstructed image indicated by the top offset, the bottom offset, the left offset and the right offset. Further, after deleting the image 1003 in the reconstructed image, the target original image 1002 is obtained.

[0211] In this application, the memory address of the target original image can also be determined. For example, Fig.10 In the example, the memory address of the upper left corner of the reconstructed image is p_src0, and the memory interval of the pixel is stride. Then the memory address of the upper left corner of the target original image is p_src0+luma_offset_top*stride+luma_offset_left.

[0212] Based on the above embodiment, the present application also provides another method for determining the area to be decoded according to the starting position, the first height and the second height. Fig.11 Provide explanation.

[0213] Fig.11 The method flow for determining the image to be decoded provided in the embodiment of the present application Figure 2 .like Fig.10 As shown, the method includes:

[0214] S111, parsing the image parameter set included in the coding information to obtain the location area and number of image slices included in the coded image.

[0215] Optionally, the image slice may be an image slice or an image tile.

[0216] It should be noted that slice divides the coded image into strips, and tile divides the coded image into rectangles.

[0217] S112, determining whether the number of fragments is equal to a preset value.

[0218] If yes, execute S113, otherwise execute S114.

[0219] S113, determining a to-be-decoded area according to the starting position, the first height, and the second height.

[0220] It should be noted that the execution process of S113 can be found in Figure 3 S303 in the description will not be described in detail here.

[0221] S114, deleting image slices that do not overlap with the calibrated area in the coded image, and determining a to-be-decoded area in the coded image after deleting the image slices according to the starting position, the first height, and the second height.

[0222] Combine the following Fig.12 S114 is exemplarily described by taking the image fragments as tiles as an example.

[0223] Fig.12 A schematic diagram of a coded image including multiple tiles provided in an embodiment of the present application. Fig.12 As shown, the coded image includes multiple tiles. For example, the multiple tiles include T1-T10. Among them, image tiles T1-T4 and T7-T10 are 8 image tiles that do not overlap with the calibration area in the coded image.

[0224] The image slices T1-T4 and T7-T10 which do not overlap with the calibration area can be deleted from the coded image to obtain a remaining coded image, and then the area to be decoded can be determined in the remaining coded image according to the starting position, the first height and the second height.

[0225] It should be noted that the method for determining the area to be decoded in the remaining coded image according to the starting position, the first height and the second height is similar to the above S303 and will not be described in detail here.

[0226] Combine the following Fig.13 S114 is exemplarily described by taking the image slice as an example.

[0227] Fig.13 A schematic diagram of a coded image including multiple slices provided in an embodiment of the present application. Fig.12 As shown, the coded image includes multiple slices. For example, the multiple slices include S1-S5. Among them, the image slices S1, S2 and S5 are three image slices that do not overlap with the calibration area in the coded image.

[0228] The image slices S1, S2 and S5 which do not overlap with the calibration area may be deleted from the coded image to obtain a remaining coded image, and then the area to be decoded may be determined in the remaining coded image according to the starting position, the first height and the second height.

[0229] It should be noted that the method for determining the area to be decoded in the remaining coded image according to the starting position, the first height and the second height is similar to the above S303 and will not be described in detail here.

[0230] According to the method design of the present application, for the image to be decoded, the completed CTU row included in the image to be decoded needs to be decoded, that is, the first width in the present application has no effect on the decoding speed and memory. In conjunction with Table 1, compared with full-frame decoding (i.e., full-frame decoding of the encoded image (with height H2)), the image to be decoded with heights of H2 / 2, H2 / 4, and H2 / 8 (i.e., the first height) is decoded to obtain the decoding time and the occupied memory size, as shown in Table 1 below.

[0231] Table 1

[0232] Resolution Height Decoding time (milliseconds, percentage) Memory (Mbytes, %) 1080(H2) 82.679，100％ 6.09，100％ 540(H2 / 2) 51.000，61.68％ 4.15，68.14％ 270(H2 / 4) 31.008，37.50％ 3.03，49.75％ 135(H2 / 8) 21.383，25.86％ 2.46，40.39％

[0233] It should be noted that Table 1 shows the test results obtained based on the test data with the starting height position and the starting width position both being 0 and the first width being 1920 as an example.

[0234] In Table 1, 1080 (H2) is used to indicate full frame decoding. When full frame decoding is performed, the decoding duration is 82.679 milliseconds (considered as 100% of the reference duration percentage), and the memory occupied size is 6.09 megabytes (considered as 100% of the reference memory occupied percentage).

[0235] For example, when decoding the image to be decoded with a first height of 540 (ie, height H2 / 2), the decoding time is 51.000 milliseconds (61.68% of 82.679 milliseconds), and the memory occupied is 4.15 megabytes (68.14% of 6.09 megabytes).

[0236] Furthermore, it can be seen from Table 1 that the smaller the first height is, the shorter the decoding time is and the smaller the memory occupied is.

[0237] It should be noted that the image processing method provided in this application can be applied to codec standards including but not limited to the heif codec standard or a certain codec standard.

[0238] Fig.14 This is a schematic diagram of the structure of the image processing device provided in the embodiment of the present application. Fig.14 As shown, the image processing device 10 includes:

[0239] The receiving module 11 is used to receive the coded information sent by the server; the coded information includes the coded image corresponding to the original image;

[0240] An acquisition module 12, used for acquiring an operation instruction;

[0241] The determination module 13 is used to determine the image to be decoded in the encoded image according to the operation instruction and the encoding information; the image to be decoded is the encoded image corresponding to the target original image in the marked area of ​​the original image;

[0242] A processing module 14 is used to decode the image to be decoded to obtain a target original image;

[0243] The display module 15 is used to display the original image of the target in the calibration area.

[0244] The image processing device provided in the embodiment of the present application can execute the above-mentioned image processing method, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0245] Optionally, the operation instruction includes a starting position, a first height and a first width of the calibration area, and the encoding information includes a sequence parameter set; the determination module 13 is specifically used to: determine whether the starting position, the first height and the first width are all preset values;

[0246] If so, the sequence parameter set is parsed to obtain decoding information, which includes the second height and second width of the encoded image; and the image to be decoded is determined in the encoded image based on the starting position, the first height, the first width, the second height and the second width.

[0247] Optionally, the determination module 13 is specifically used to: determine whether the calibration area is located in the encoded image according to the starting position, the first height, the first width, the second height and the second width;

[0248] If yes, then determine the image to be decoded in the encoded image according to the starting position, the first height and the second height.

[0249] Optionally, the determination module 13 is specifically configured to:

[0250] If the starting position is located in the encoded image, the first height is smaller than the second height, and the first width is smaller than the second width, then the calibration area is located in the encoded image;

[0251] Otherwise, the marked area is not in the encoded image.

[0252] Optionally, the determination module 13 is specifically configured to: determine the area to be decoded according to the starting position, the first height, and the second height; and determine the image in the area to be decoded in the encoded image as the image to be decoded.

[0253] Optionally, the starting position includes a height starting position; the determination module 13 is specifically used to: determine the minimum boundary value according to the height starting position; determine the maximum boundary value according to the first height, the second height and the height starting position; and determine the CTU row from the minimum boundary value to the maximum boundary value in the encoded image as the image to be decoded.

[0254] Optionally, the determination module 13 is specifically used to: shift the starting height position right by a third preset value; the third preset value is a preset value corresponding to the encoding method corresponding to the encoded image; and determine the maximum value of the first preset value and the difference between the starting height position after the right shift and the second preset value as the minimum boundary value.

[0255] Optionally, the determination module 13 is specifically used to: shift the third preset value left by the tenth preset value; determine the difference between the sum of the height starting position, the first height, the third preset value after the left shift, and the tenth preset value; shift the difference right by the third preset value; and determine the minimum value between the second height and the difference after the right shift as the maximum boundary value.

[0256] Optionally, the processing module 14 is specifically used to: obtain decoding reference information; and decode the image to be decoded according to the decoding reference information to obtain a target original image.

[0257] Optionally, the decoded information includes a switch identifier of the filter processing tool; the processing module 14 is specifically used to: determine whether the filter processing tool is turned on according to the switch identifier;

[0258] If yes, determining a fourth preset value of CTU rows located above the image to be decoded and a fifth preset value of CTU rows located below the image to be decoded in the encoded image as a first reference decoded image, and determining decoding reference information according to the first reference decoded image;

[0259] Otherwise, the sixth preset value of CTU rows located above the image to be decoded and the seventh preset value of CTU rows located below the image to be decoded in the encoded image are determined as the second reference decoded image, and the decoding reference information is determined based on the second reference decoded image.

[0260] Optionally, the fourth preset value of CTU rows located above the image to be decoded includes the first CTU row and the second CTU row, and the fifth preset value of CTU rows located below the image to be decoded includes the third CTU row; the processing module 14 is specifically used to: decode the first CTU row, the second CTU row and the third CTU row, respectively, to obtain a first original pixel set, a second original pixel set and a third original pixel set; use a filtering processing tool to perform in-loop filtering on the first original pixel set and the second original pixel set, respectively, to obtain a first processed pixel set and a second processed pixel set; determine the decoding reference information according to the first processed pixel set, the second processed pixel set and the third original pixel set.

[0261] Optionally, the first processed pixel set and the second processed pixel set respectively include N rows of pixel data, where N is an integer greater than or equal to 2; the processing module 14 is specifically used to: use the last M rows of pixel data in the N rows of pixel data in the second processed pixel set to replace the last M rows of pixel data in the N rows of pixel data in the first processed pixel set, to obtain a third processed pixel set corresponding to the first processed pixel set; M is a positive integer less than N; and determine the second processed pixel set, the third processed pixel set and the third original pixel set as decoding reference information.

[0262] Optionally, the sixth preset value of CTU rows located above the image to be decoded includes a first CTU row, and the seventh preset value of CTU rows located below the image to be decoded includes a second CTU row; the processing module 14 is specifically used to: decode the first CTU row and the second CTU row respectively to obtain a first original pixel set and a second original pixel set; the first original pixel set includes N rows of original pixel data; use the last M rows of original pixel data of the N rows of original pixel data in the first original pixel set to replace the first M rows of original pixel data of the N rows of original pixel data in the first original pixel set to obtain a second pixel set; and determine the second pixel set and the second original pixel set as decoding reference information.

[0263] Optionally, the processing module 14 is specifically used to: decode the image to be decoded according to the decoding reference information to obtain reconstructed pixel values; and store the reconstructed pixel values ​​in a preset area to obtain a reconstructed image; the preset area has the same size as the image to be decoded; and determine the target original image in the reconstructed image according to the starting position, the first height, and the first width.

[0264] Optionally, the processing module 14 is specifically used to: determine the difference between the initial height position and the left shift value after the third preset value is shifted left by the minimum boundary value as the top offset; determine the sum of the maximum boundary value and the ninth preset value, and determine the minimum value of the value after the third preset value is shifted left by the sum value, and the difference between the second height and the initial height position and the first height as the bottom offset; determine the width starting position of the starting position as the left offset; determine the difference between the second width and the sum of the left offset and the first width as the right offset; the top offset, bottom offset, left offset and right offset indicate the image to be deleted in the reconstructed image; delete the image to be deleted in the reconstructed image to obtain the target original image.

[0265] Optionally, the determination module 13 is specifically used to: parse the image parameter set included in the encoding information to obtain the position area and the number of slices of the image slices included in the encoded image; if the number of slices is equal to a preset value, determine the area to be decoded based on the starting position, the first height, and the second height; if the number of slices is not equal to the preset value, delete the image slices that do not overlap with the calibrated area in the encoded image, and determine the area to be decoded in the encoded image after deleting the image slices based on the starting position, the first height, and the second height.

[0266] Optionally, the operation instruction is an instruction corresponding to a sliding operation; the determination module 13 is specifically used to: detect a pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determine a calibration area in the current display interface; and determine an image to be decoded corresponding to the calibration area in the encoded image according to the calibration area and the encoding information.

[0267] The image processing device provided in the embodiment of the present application can execute the above-mentioned image processing method, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0268] Fig.15 This is a hardware diagram of a terminal device provided in an embodiment of the present application. Fig.15 As shown, the terminal device 20 may include: a transceiver 21 , a memory 22 , and a processor 23 .

[0269] The transceiver 21 may include: a transmitter and / or a receiver. The transmitter may also be referred to as a transmitter, a transmitter, a transmission port, a transmission interface, or the like, and the receiver may also be referred to as a receiver, a receiver, a reception port, a reception interface, or the like. Exemplarily, the transceiver 21, the memory 22, and the processor 23 are interconnected via a bus 404.

[0270] The memory 22 is used to store computer executable instructions;

[0271] The processor 23 is used to execute the computer execution instructions stored in the memory 22, so that the processor 23 executes the above-mentioned image processing method.

[0272] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the above-mentioned image processing method is implemented.

[0273] An embodiment of the present application also provides a computer program product, including a computer program, which can implement the above-mentioned image processing method when executed by a processor.

[0274] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The above-mentioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the above-mentioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc and any combination thereof.

[0275] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0276] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0277] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0278] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0279] In the present application, the term "include" and its variations may refer to non-restrictive inclusion; the term "or" and its variations may refer to "and / or". The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In the present application, "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship.

[0280] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0281] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image processing method, It is characterized in that Applied to a terminal device, the method comprises: Receiving encoding information sent by a server; the encoding information includes an encoded image corresponding to the original image; Acquire an operation instruction, wherein the operation instruction includes a starting position, a first height, and a first width of the calibration area, and the encoding information includes a sequence parameter set; According to the operation instruction and the encoding information, an image to be decoded is determined in the encoded image; the image to be decoded is an encoded image corresponding to a target original image in a marked area in the original image, and the sequence parameter set is parsed to obtain decoding information, wherein the decoding information includes a second height and a second width of the encoded image; and is determined in the encoded image according to the starting position, the first height, the first width, the second height and the second width; Decode the image to be decoded to obtain the target original image, and display the target original image in the calibration area; Wherein, the decoding information includes a switch identifier of a filtering processing tool; the decoding of the image to be decoded to obtain a target original image includes: judging whether the filtering processing tool is turned on according to the switch identifier; if so, determining a fourth preset value of CTU rows located above the image to be decoded and a fifth preset value of CTU rows located below the image to be decoded in the encoded image as a first reference decoded image, and determining decoding reference information according to the first reference decoded image; otherwise, determining a sixth preset value of CTU rows located above the image to be decoded and a seventh preset value of CTU rows located below the image to be decoded in the encoded image as a second reference decoded image, and determining decoding reference information according to the second reference decoded image; decoding the image to be decoded according to the decoding reference information to obtain the target original image, and displaying the target original image in the calibration area.

2. The method according to claim 1, It is characterized in that The step of determining the image to be decoded in the encoded image according to the operation instruction and the encoding information includes: Determine whether the starting position, the first height, and the first width are all preset values; If so, the sequence parameter set is parsed to obtain decoding information, which includes the second height and second width of the encoded image; and based on the starting position, the first height, the first width, the second height and the second width, the image to be decoded is determined in the encoded image.

3. The method according to claim 2, It is characterized in that The step of determining the image to be decoded in the encoded image according to the starting position, the first height, the first width, the second height, and the second width includes: Determining whether the calibration area is located in the encoded image according to the starting position, the first height, the first width, the second height, and the second width; If so, the image to be decoded is determined in the encoded image according to the starting position, the first height and the second height.

4. The method according to claim 3, It is characterized in that The determining, according to the starting position, the first height, the first width, the second height, and the second width, whether the calibration area is located in the encoded image includes: If the starting position is located in the encoded image, the first height is smaller than the second height, and the first width is smaller than the second width, then the calibration area is located in the encoded image; Otherwise, the marked area is not in the encoded image.

5. The method according to any one of claims 2 to 4, It is characterized in that The step of determining the image to be decoded in the encoded image according to the starting position, the first height, and the second height includes: Determine a to-be-decoded area according to the starting position, the first height, and the second height; An image in the to-be-decoded area within the encoded image is determined as the to-be-decoded image.

6. The method according to claim 5, It is characterized in that The starting position includes a height starting position; The step of determining the area to be decoded according to the starting position, the first height, and the second height includes: Determining a minimum boundary value according to the height starting position; Determining a maximum boundary value according to the first height, the second height and the height starting position; The CTU row from the minimum boundary value to the maximum boundary value in the encoded image is determined as the image to be decoded.

7. The method according to claim 6, It is characterized in that The step of determining the minimum boundary value according to the height starting position includes: The height starting position is shifted rightward by a third preset value; the third preset value is a preset value corresponding to the encoding mode corresponding to the encoded image; The maximum value of the first preset value and the difference between the starting height position after the right shift and the second preset value is determined as the minimum boundary value.

8. The method according to claim 7, It is characterized in that The determining of the maximum boundary value according to the first height, the second height and the height starting position includes: Shifting the third preset value leftward by a tenth preset value; Determine a difference between a sum of the height starting position, the first height, the third preset value after left shift, and the tenth preset value; Shifting the difference rightward by the third preset value; The minimum value between the second height and the difference after right shift is determined as the maximum boundary value.

9. The method according to claim 1, It is characterized in that The fourth preset value of CTU rows located above the image to be decoded includes a first CTU row and a second CTU row, and the fifth preset value of CTU rows located below the image to be decoded includes a third CTU row; The determining, according to the first reference decoded image, decoding reference information comprises: Decoding the first CTU row, the second CTU row, and the third CTU row respectively to obtain a first original pixel set, a second original pixel set, and a third original pixel set; Using the filtering processing tool to perform in-loop filtering on the first original pixel set and the second original pixel set, respectively, to obtain a first processed pixel set and a second processed pixel set; Decoding reference information is determined according to the first processed pixel set, the second processed pixel set and the third original pixel set.

10. The method according to claim 9, It is characterized in that The first processing pixel set and the second processing pixel set respectively include N lines of pixel data, where N is an integer greater than or equal to 2; The step of determining decoding reference information according to the first processed pixel set, the second processed pixel set, and the third original pixel set comprises: The last M lines of pixel data in the N lines of pixel data in the second processing pixel set are used to replace the last M lines of pixel data in the N lines of pixel data in the first processing pixel set, so as to obtain a third processing pixel set corresponding to the first processing pixel set; M is a positive integer less than N; The second processed pixel set, the third processed pixel set and the third original pixel set are determined as the decoding reference information.

11. The method according to claim 1, It is characterized in that The sixth preset value of CTU rows located above the image to be decoded includes a first CTU row, and the seventh preset value of CTU rows located below the image to be decoded includes a second CTU row; The determining decoding reference information according to the second reference decoded image includes: Decoding the first CTU row and the second CTU row respectively to obtain a first original pixel set and a second original pixel set; the first original pixel set includes N rows of original pixel data; The last M lines of original pixel data in the N lines of original pixel data in the first original pixel set are used to replace the first M lines of original pixel data in the N lines of original pixel data in the first original pixel set, so as to obtain a second pixel set; The second pixel set and the second original pixel set are determined as decoding reference information.

12. The method according to claim 1, It is characterized in that Decoding the image to be decoded according to the decoding reference information to obtain the target original image includes: Decoding the image to be decoded according to the decoding reference information to obtain a reconstructed pixel value; and storing the reconstructed pixel values ​​in a preset area to obtain a reconstructed image; the preset area has the same size as the image to be decoded; The target original image is determined in the reconstructed image according to the starting position, the first height, and the first width.

13. The method according to claim 12, It is characterized in that The step of determining the target original image in the reconstructed image according to the starting position, the first height, and the first width includes: Determine the difference between the initial height position and the left shift value after the third preset value is shifted left by the minimum boundary value as the top offset; Determine the sum of the maximum boundary value and the ninth preset value, and determine the minimum value of the third preset value shifted left by the sum and the difference between the second height, the initial height position and the first height as the bottom offset; Determine the width starting position of the starting position as the left offset; The difference between the second width and the sum of the left offset and the first width is determined as the right offset; the top offset, the bottom offset, the left offset and the right offset indicate an image to be deleted in the reconstructed image; The image to be deleted is deleted from the reconstructed image to obtain a target original image.

14. The method according to claim 5, It is characterized in that The step of determining the area to be decoded according to the starting position, the first height, and the second height includes: Parsing the image parameter set included in the coding information to obtain the location area and number of image slices included in the coded image; If the number of slices is equal to a preset value, determining the area to be decoded according to the starting position, the first height, and the second height; If the number of slices is not equal to a preset value, image slices that do not overlap with the calibrated area are deleted from the encoded image, and the area to be decoded is determined in the encoded image after deleting the image slices based on the starting position, the first height, and the second height.

15. The method according to claim 1, It is characterized in that The operation instruction is an instruction corresponding to the sliding operation; The step of determining the image to be decoded in the encoded image according to the operation instruction and the encoding information includes: Detecting a pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining a calibration area in the current display interface; According to the marked area and the encoding information, an image to be decoded corresponding to the marked area is determined in the encoded image.

16. An image processing device, It is characterized in that Applied to a terminal device, the device comprises: A receiving module, used for receiving the coded information sent by the server; the coded information includes a coded image corresponding to the original image; An acquisition module, used for acquiring an operation instruction, wherein the operation instruction includes a starting position, a first height and a first width of the calibration area, and the encoding information includes a sequence parameter set; a determination module, configured to determine an image to be decoded in the encoded image according to an operation instruction and the encoding information; the image to be decoded is an encoded image corresponding to a target original image in a marked area in the original image, and is obtained by parsing the sequence parameter set, wherein the decoding information includes a second height and a second width of the encoded image; and is determined in the encoded image according to the starting position, the first height, the first width, the second height and the second width; A processing module is used to decode the image to be decoded, obtain the target original image, and display the target original image in the calibration area; The decoding information includes a switch identifier of a filtering processing tool; the processing module decodes the image to be decoded to obtain a target original image, specifically including: judging whether the filtering processing tool is turned on according to the switch identifier; if so, determining a fourth preset value of CTU rows located above the image to be decoded and a fifth preset value of CTU rows located below the image to be decoded in the encoded image as a first reference decoded image, and determining decoding reference information according to the first reference decoded image; otherwise, determining a sixth preset value of CTU rows located above the image to be decoded and a seventh preset value of CTU rows located below the image to be decoded in the encoded image as a second reference decoded image, and determining decoding reference information according to the second reference decoded image; decoding the image to be decoded according to the decoding reference information to obtain the target original image; A display module is used to display the target original image in the calibration area.

17. A terminal device, It is characterized in that include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 15.

18. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 15 when executed by a processor.

19. A computer program product, It is characterized in that include: Computer program; when the computer program is executed by a processor, it implements the method described in any one of claims 1 to 15.

Images