Image data processing method and device, equipment and medium

Abstract

The invention relates to the technical field of image processing, in particular to an image data processing method and device, equipment and a medium, which are used for solving the problem of low image quality existing in a traditional data compression scheme, and the method comprises the following steps: obtaining to-be-processed video frame data, the video frame data is image data corresponding to any video frame in the video stream to be displayed; for each processing unit of the video frame data, executing the following operations: performing in-line segmentation on each pixel based on the pixel value of each pixel in any processing unit of the video frame data, and performing compression packaging on the pixel information of each pixel based on a plurality of obtained segments to obtain a data packet corresponding to the processing unit, the pixel positions of the pixels in each segment are continuous and the pixel values are the same; sending each data packet of the video frame data to a receiving end to display an image corresponding to the video frame data; therefore, the transmission rate is improved, transmission resources are saved, and the compression quality of each obtained data packet is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular, to an image data processing method, apparatus, device, and medium. Background Art

[0002] Image processing is a technology that uses a computer to analyze images to achieve the desired results. Image processing generally refers to digital image processing. A digital image is a large two-dimensional array obtained by shooting with devices such as industrial cameras, video cameras, and scanners. The elements of this array are called pixels. Image coding and compression technology is a kind of image processing technology. Image coding and compression technology can reduce the amount of data (i.e., the number of bits) describing the image, so as to save image transmission and processing time and reduce the occupied memory capacity.

[0003] Under the related technology, image coding and compression usually compresses the pixel information of each pixel in the current frame data based on the correlation between pixels or between frames, so as to achieve the purpose of reducing the amount of data and increasing the transmission rate. However, in actual applications, in scenarios where high compression rates are prioritized, the compression quality is often easily overlooked. For example, the data compression method based on the correlation between pixels or between frames as described above is a lossy compression method, resulting in low image quality after restoration. Summary of the Invention

[0004] Embodiments of the present application provide an image data processing method, apparatus, device, and medium, which are used to increase the transmission rate, save transmission resources, and ensure the compression quality of each obtained data packet.

[0005] The specific technical solutions provided by the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides an image data processing method, including:

[0007] Obtain video frame data to be processed, where the video frame data is image data corresponding to any video frame in a video stream to be displayed;

[0008] Perform the following operations on each processing unit of the video frame data: Based on the pixel values of the pixels in any processing unit of the video frame data, segment the pixels within a row, and based on the obtained multiple segments, compress and package the pixel information of the pixels to obtain a data packet corresponding to any processing unit, where the pixel positions of the pixels within each segment are continuous and the pixel values are the same;

[0009] Send the data packets of the video frame data to a receiving end to display the image corresponding to the video frame data.

[0010] An image data processing method provided by an embodiment of the present application can improve the transmission rate and save transmission resources by compressing and packing each processing unit in the video frame data to be processed. At the same time, before compressing and packing each processing unit, each pixel in each processing unit is segmented within a row, where the pixel positions of the pixels in each segment are continuous and the pixel values are the same. And when compressing and packing the processing unit, specifically based on the obtained multiple segments, the pixel information of each pixel included in the processing unit is compressed and packed, which can achieve lossless compression of the video frame data, thereby ensuring the compression quality of each obtained data packet, and further ensuring the quality of the image corresponding to the video frame data displayed by the receiving end based on the received data packets.

[0011] In a possible implementation manner, the segmenting each pixel within a row based on the pixel values of each pixel in any processing unit of the video frame data includes:

[0012] The following operations are respectively performed on each pixel in any processing unit of the video frame data:

[0013] For any pixel among the pixels, if the pixel value of the any pixel is the same as the pixel value of the previous pixel, then the any pixel is assigned to the segment where the previous pixel is located;

[0014] If the pixel value of the any pixel is different from the pixel value of the previous pixel, then the any pixel is assigned to a new segment.

[0015] In the above method, segmenting the pixels in each processing unit according to the pixel values can ensure that subsequent data compression based on any segment is lossless compression, so that after the receiving end receives the compressed data packets, the original image can be restored, thereby improving the image quality.

[0016] In a possible implementation manner, the compressing and packing the pixel information of each pixel based on the obtained multiple segments to obtain the data packet corresponding to any processing unit includes:

[0017] Selecting N segments with longer segment lengths from the multiple segments according to the segment length, where N is a positive integer;

[0018] Based on the number of bits required for the N segments, the number of bits required for the other segments in the multiple segments, and the number of bits required for the original pixel data in any processing unit, the compression ratio of any processing unit is obtained;

[0019] After determining that the compression ratio is greater than the compression ratio threshold, the following operations are performed for each of the N segments: Compress the segment pixel values, start position, and end position corresponding to any one of the N segments to obtain the segment compression data corresponding to the any one segment;

[0020] Package the segment compression data corresponding to the N segments with the pixel values of each pixel in the other segments to obtain the compression data packet corresponding to any one processing unit.

[0021] In the above method, by setting the maximum number of target segments for data compression, the compression quality can be guaranteed; at the same time, by setting the compression ratio threshold, after the compression ratio of the processing unit after data compression for the N segments is greater than the compression ratio threshold, the compression operation is only implemented, which can ensure that the candidate transmits the compression data packet, and can indeed improve the transmission rate and save transmission resources.

[0022] In a possible implementation, the number of bits required for the N segments is obtained by the following method:

[0023] Determine the sum of the start position, end position, and the number of bits required for the segment pixel values corresponding to any one of the N segments as the number of bits required for the any one segment, where the start position is the pixel position of the first pixel in the any one segment, and the end position is the pixel position of the last pixel in the any one segment;

[0024] Determine the product value of the number of bits required for the any one segment and N as the number of bits required for the N segments.

[0025] In the above method, based on the segment pixel values, start position, and end position of each segment, calculate the number of bits required for the segment, so as to obtain the number of bits required for the N segments, calculate whether the current compression method can obtain the expected compression ratio, and further guide the subsequent processing flow to improve the transmission rate.

[0026] In a possible implementation, after determining that the compression ratio is greater than the compression ratio threshold, it further includes:

[0027] Generate a transmission control signal based on the compression ratio and a preset transmission protocol;

[0028] Replace the preset transmission control signal with the transmission control signal;

[0029] The step of sending each data packet of the video frame data to the receiving end includes:

[0030] When transmitting the compression data packet, control the opening of each target transmission link based on the transmission control signal;

[0031] Send the compressed data packets to the receiving end through the respective target transmission links;

[0032] After the opening duration reaches the duration included in the transmission control signal, close the respective target transmission links.

[0033] In the above method, after determining that the compression ratio is greater than the compression ratio threshold, a transmission control signal is generated based on the compression ratio to control the operation and radio interface state of the transmission link (connection layer) between the transmitter and the receiver, thereby achieving the purpose of saving resources.

[0034] In a possible implementation manner, after obtaining the compression ratio of any processing unit, the method further includes:

[0035] After determining that the compression ratio is not greater than the compression ratio threshold, pack the original pixel data in any processing unit to obtain an original data packet corresponding to any processing unit;

[0036] The step of sending each data packet of the video frame data to the receiving end includes:

[0037] When transmitting the original data packet, based on a preset transmission control signal, control multiple candidate transmission links to be turned on;

[0038] Transmit the original data packet to the receiving end through the multiple candidate transmission links;

[0039] After the opening duration reaches the duration included in the preset transmission control signal, close the multiple candidate transmission links.

[0040] In a second aspect, an embodiment of the present application provides an image data processing device, including:

[0041] An acquisition module, configured to acquire video frame data to be processed, where the video frame data is image data corresponding to any video frame in a video stream to be displayed;

[0042] A segmentation and packaging module, configured to perform the following operations for each processing unit of the video frame data: based on the pixel values of each pixel in any processing unit of the video frame data, perform in-line segmentation on each pixel, and based on the obtained multiple segments, compress and package the pixel information of each pixel to obtain a data packet corresponding to any processing unit, where the pixel positions of each pixel in each segment are continuous and the pixel values are the same;

[0043] A sending module, configured to send each data packet of the video frame data to the receiving end to display the image corresponding to the video frame data.

[0044] In a possible implementation manner, the segmentation and packaging module is specifically configured to:

[0045] For each pixel in any processing unit for the video frame data, perform the following operations respectively:

[0046] For any one of the pixels, if the pixel value of the any one pixel is the same as the pixel value of the previous pixel, then classify the any one pixel into the segment where the previous pixel is located;

[0047] If the pixel value of the any one pixel is different from the pixel value of the previous pixel, then classify the any one pixel into a new segment.

[0048] In a possible implementation manner, the segmenting and packing module is specifically configured to:

[0049] Select N segments with longer segment lengths from the multiple segments according to the segment length, where N is a positive integer;

[0050] Based on the number of bits required for the N segments, the number of bits required for other segments in the multiple segments, and the number of bits required for the original pixel data in any processing unit, obtain the compression ratio of any processing unit;

[0051] After determining that the compression ratio is greater than the compression ratio threshold, for each of the N segments, perform the following operations: Compress the segment pixel value, start position, and end position corresponding to any one of the N segments to obtain the segment compression data corresponding to any one of the N segments;

[0052] Pack the segment compression data corresponding to the N segments and the pixel values of each pixel in the other segments to obtain the compression data packet corresponding to any processing unit.

[0053] In a possible implementation manner, obtain the number of bits required for the N segments through the following method:

[0054] Determine the sum of the start position, end position, and the number of bits required for the segment pixel value corresponding to any one of the N segments as the number of bits required for any one of the N segments, where the start position is the pixel position of the first pixel in any one of the N segments, and the end position is the pixel position of the last pixel in any one of the N segments;

[0055] Determine the product value of the number of bits required for any one of the N segments and N as the number of bits required for the N segments.

[0056] In a possible implementation manner, after determining that the compression ratio is greater than the compression ratio threshold, the segmenting and packing module is further configured to:

[0057] Generate a transmission control signal based on the compression ratio and a preset transmission protocol;

[0058] Replace the preset transmission control signal with the transmission control signal;

[0059] The sending module is specifically configured to:

[0060] When transmitting the compressed data packet, based on the transmission control signal, control each target transmission link to be turned on;

[0061] Send the compressed data packet to the receiving end through each of the target transmission links;

[0062] After the turn-on duration reaches the duration included in the transmission control signal, turn off each of the target transmission links.

[0063] In a possible implementation manner, after obtaining the compression ratio of any processing unit, the segmentation and packaging module is further configured to:

[0064] After determining that the compression ratio is not greater than the compression ratio threshold, pack the original pixel data in any processing unit to obtain an original data packet corresponding to any processing unit;

[0065] The sending module is specifically configured to:

[0066] When transmitting the original data packet, based on the preset transmission control signal, control multiple candidate transmission links to be turned on;

[0067] Transmit the original data packet to the receiving end through the multiple candidate transmission links;

[0068] After the turn-on duration reaches the duration included in the preset transmission control signal, turn off the multiple candidate transmission links.

[0069] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0070] A memory for storing computer programs or instructions;

[0071] A processor for executing the computer programs or instructions in the memory, so that the method according to any one of the first aspects is executed.

[0072] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, when the instructions in the storage medium are executed by a processor, enabling the processor to execute the method according to any one of the first aspects.

[0073] In addition, for the technical effects brought about by any one of the implementation manners in the second aspect to the fourth aspect, reference may be made to the technical effects brought about by different implementation manners in the first aspect, which will not be elaborated herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1A FIG. is a schematic diagram of an optional application scenario in an embodiment of the present application;

[0075] Figure 1B FIG. is a schematic structural diagram of a display device in an embodiment of the present application;

[0076] Figure 2 FIG. is another schematic diagram of an optional application scenario in an embodiment of the present application;

[0077] Figure 3 FIG. is a schematic flowchart of an image data processing method in an embodiment of the present application;

[0078] Figure 4 FIG. is a schematic flowchart of an in-line segmentation method in an embodiment of the present application;

[0079] Figure 5 FIG. is a schematic flowchart of a data compression process of a compressed data packet in an embodiment of the present application;

[0080] Figure 6 FIG. is a schematic flowchart of a method for determining the number of bits required for N segments in an embodiment of the present application;

[0081] Figure 7 FIG. is a schematic diagram of the format of segmented compressed data in an embodiment of the present application;

[0082] Figure 8 FIG. is a schematic diagram of the format of a compressed data packet in an embodiment of the present application;

[0083] Figure 9A FIG. is a schematic flowchart of a method for determining a transmission control signal of a compressed data packet in an embodiment of the present application;

[0084] Figure 9B FIG. is a schematic diagram of an application scenario of a sending end and a receiving end in an embodiment of the present application;

[0085] Figure 9C FIG. is a schematic diagram for comparing the working states of the connection layer and the radio interface before and after in-line compression in an embodiment of the present application;

[0086] Figure 10 FIG. is a schematic flowchart of a transmission control process of a compressed data packet in an embodiment of the present application;

[0087] Figure 11 FIG. is a schematic logical architecture diagram of an image data processing device in an embodiment of the present application;

[0088] Figure 12 This is a schematic diagram of the physical architecture of the electronic device in the embodiments of the present application. Detailed implementation manners

[0089] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0090] It should be noted that the terms "first", "second", "third", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data used may be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0091] Next, a brief introduction to the design concept of the embodiments of the present application will be given.

[0092] The present application relates to the technical field of image processing, and mainly relates to an image data processing method, device, equipment and medium.

[0093] Under the related technology, image coding and compression is usually based on the correlation between pixels or frames, and the pixel information of each pixel in the current frame data is compressed to achieve the purpose of reducing the data volume and improving the transmission rate. However, in actual applications, in scenarios where high compression rate is given priority, the compression quality is often easily ignored. For example, the data compression method based on the correlation between pixels or frames as described above is a lossy compression method, which is likely to result in low image quality after restoration.

[0094] Correspondingly, in scenarios where low compression rate is given priority, the image quality is guaranteed, but the transmission rate is easily ignored, resulting in unnecessary waste of transmission resources. For example, in the point-to-point transmission method under the related technology, that is, after the sending end receives the data of a pixel, it immediately sends the data of the pixel to the receiving end. When not considering the loss during the transmission process, the image displayed at the receiving end is the original image, and there is a certain degree of waste in both the transmission rate and the transmission resources.

[0095] In view of this, to solve the above problems existing in the traditional data compression solution, an embodiment of the present application provides an image data processing method, which includes: obtaining video frame data to be processed, where the video frame data is the image data corresponding to any video frame in the video stream to be displayed; performing the following operations on each processing unit of the video frame data: based on the pixel values of the pixels in any processing unit of the video frame data, segmenting the pixels within a row, and based on the obtained multiple segments, compressing and packing the pixel information of each pixel to obtain a data packet corresponding to the processing unit, where the pixel positions of the pixels within each segment are continuous and the pixel values are the same; sending the data packets of the video frame data to a receiving end to display the image corresponding to the video frame data.

[0096] In this way, by compressing and packing each processing unit in the video frame data to be processed, the transmission rate can be increased and transmission resources can be saved; at the same time, before compressing and packing each processing unit, the pixels within each processing unit are segmented within a row, where the pixel positions of the pixels within each segment are continuous and the pixel values are the same, and when compressing and packing the processing unit, specifically based on the obtained multiple segments, the pixel information of the pixels included in the processing unit is compressed and packed, which can achieve lossless compression of the video frame data, thereby ensuring the compression quality of each obtained data packet, and further ensuring the quality of the image corresponding to the video frame data displayed by the receiving end based on the received data packets.

[0097] An image data processing method provided by an embodiment of the present application is applicable to an electronic device.

[0098] In some feasible embodiments, the electronic device may be a terminal device having an image display device (such as a display panel). For example, the electronic device may be a smart terminal, a smart mobile terminal, a tablet computer, a notebook computer, a smart handheld device, a personal computer (PC), a computer, a smart screen, a display device, an in-vehicle display device, various wearable devices (such as virtual reality (VR), augmented reality (AR), personal digital assistant (PDA), etc.).

[0099] In some other feasible embodiments, the electronic device may also be a server connected to a device with a display function (such as a display device). For example, the electronic device may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms; among them, the server may be connected to the device with a display function by wired or wireless means, and the connection method is not limited in this application.

[0100] It can be understood that the specific type of the above electronic device is not limited in this application.

[0101] The following combines the accompanying drawings to show a schematic diagram of an application scenario of an optional image data processing method in the embodiments of this application with the electronic device being a display device as an example. As Figure 1A 、 Figure 1B 。 Refer to Figure 1A As shown, the application scenario includes a display device 10. The display device 10 uses an image data processing method provided in the embodiments of this application to perform segmented compression on the video frame data to be processed, so as to improve the transmission rate, save transmission resources, and ensure the image quality.

[0102] Refer to Figure 1B As shown, the display device 10 includes an image processing module 101 and a display 102. The display 102 includes a driving chip 1021 and a display panel 1022. In specific implementation, the image processing module 101 in the display device 10 executes each process of an image data processing method (see the subsequent process content for details) to perform segmented compression on the video frame data to be processed, obtain each data packet of the video frame data, and then the image processing module 101 sends each data packet to the driving chip 1021. The driving chip 1021 generates a driving signal based on the received data packets, and drives the display panel 1022 based on the driving signal, so that the display panel 1022 displays the image corresponding to the video frame data.

[0103] In some feasible embodiments, the image processing module 101 includes, but is not limited to, a Thin Film Transistor Liquid Crystal Display Controller (TCON), etc.; the driving chip 1021 includes, but is not limited to, an Organic Light-Emitting Diode (OLED) driving chip, or an Organic Electroluminescence Display (OLED) driving chip, etc.

[0104] The following takes an electronic device as a server as an example to show a schematic diagram of an application scenario of another optional image data processing method in the embodiments of the present application, as Figure 2 . Refer to Figure 2 As shown, this application scenario includes a display device 10 and a server 20. The display device 10 communicates with the server 20 through the Internet. The server 20 executes each process included in an image data processing method provided in the embodiments of the present application (see the subsequent process content for details), realizes segmenting and compressing the video frame data to be processed, obtains each data packet of the video frame data, and sends each data packet of the video frame data to the display device 10, so that the display device 10 can display the image corresponding to the video frame data based on the received each data packet.

[0105] Of course, the method provided in the embodiments of the present application is not limited to Figure 1A , Figure 2 the application scenario shown, and can also be used in other possible application scenarios, which are not limited in the embodiments of the present application.

[0106] After introducing the application scenario of the embodiments of the present application, the following further details the preferred embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. And without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0107] Refer to Figure 3 As shown, the embodiments of the present application provide an image data processing method, which is applied to the aforementioned electronic device. The specific process of this method is as follows:

[0108] Step 300: Obtain the video frame data to be processed, where the video frame data is the image data corresponding to any video frame in the video stream to be displayed.

[0109] In the embodiments of the present application, refer to Figure 1BAs shown, when step 300 is executed, the image processing module receives the video frame data sent by the previous module. Here, the previous module can be an image acquisition module in the display device, which is used to acquire the video stream to be displayed; it can also be a communication module in the display device, which is used to receive the video stream to be displayed sent by an external device (such as a server, other terminal devices, etc.), or it can be a functional module in other forms, which is not limited in this application.

[0110] Step 310: Perform the following operations for each processing unit of the video frame data:

[0111] Step 3101: Based on the pixel values of each pixel in any processing unit of the video frame data, perform in-line segmentation on each pixel.

[0112] Step 3102: Based on the obtained multiple segments, compress and package the pixel information of each pixel to obtain a data packet corresponding to this processing unit, where the pixel positions of each pixel in each segment are continuous and the pixel values are the same.

[0113] In the embodiment of this application, when step 3101 is executed, the image processing module performs the following operations row by row according to each pixel in each processing unit: For each row of pixels, in the order from left to right, the pixels with the same pixel value are grouped into one segment. When the pixel value changes, the pixel with the changed pixel value is used as the first pixel of the next segment, so that the segments corresponding to each row in this processing unit can be obtained.

[0114] In specific implementation, refer to Figure 4 As shown, when step 3101 is executed, the following steps are specifically executed:

[0115] Step 400: Perform the following steps for each pixel in this processing unit of the video frame data:

[0116] Step 4001: For any pixel among each pixel, if the pixel value of this pixel is the same as the pixel value of the previous pixel, then this pixel is grouped into the segment where the previous pixel is located.

[0117] Step 4002: If the pixel value of this pixel is different from the pixel value of the previous pixel, then this pixel is grouped into a new segment.

[0118] In this way, the image processing module performs in-line segmentation on this processing unit of the video frame data, thus obtaining multiple segments of this processing unit.

[0119] It should be noted that the above processing unit is the compression unit of the selected data compression method. If row compression is used, this processing unit is a row of pixels in the video frame data; if block compression is used, this processing unit is a block in the video frame data.

[0120] In the embodiment of the present application, if block compression is adopted, after obtaining the video frame data to be processed in step 300, the video frame data is first block-processed to obtain each block. Then, in step 310, the specific block-processing method can be implemented by any existing processing method, which will not be elaborated here.

[0121] Then, when step 3102 is executed, refer to Figure 5 As shown, the following steps are specifically executed:

[0122] Step 500: Select N segments with longer segment lengths from multiple segments according to the segment length, where N is a positive integer.

[0123] In the embodiment of the present application, N is set as the maximum number of target segments for data compression. Specifically, when step 500 is executed, the image processing module sorts the multiple segments obtained in step 3101 in descending order of segment length, and selects the first N segments with longer segment lengths. Then, after separately compressing the N segments, the number of bits required for the N segments is calculated, that is, the data volume size of the data corresponding to the N segments, that is, the number of bits required to be occupied.

[0124] Specifically, refer to Figure 6 As shown, the number of bits required for the N segments can be obtained in the following manner:

[0125] Step 600: Determine the number of bits required for a segment by adding the starting position, the ending position of any segment among the N segments, and the sum of the number of bits required for the segment pixel values corresponding to the segment, where the starting position is the pixel position of the first pixel in the segment, and the ending position is the pixel position of the last pixel in the segment.

[0126] In the embodiment of the present application, after the pixels in the processing unit are segmented row by row in step 3101, in order to record the segment information of the segment, that is, which pixels the segment contains, after obtaining each segment, the image processing module determines the pixel position of the first pixel in the segment as the starting position of the segment, and determines the pixel position of the last pixel in the segment as the ending position of the segment, so that after the driving chip receives the compression data packet corresponding to the segment, it can fill the pixel values of each pixel between the starting position and the ending position with the segment pixel values corresponding to the segment based on the starting position and the ending position of the segment. And since the pixel values of the pixels within each segment are the same, the segment pixel value of the segment is the pixel value of each pixel within the segment.

[0127] Under related technologies, according to different image coding formats, the number of bytes occupied by each pixel is different. For example, when using the RGB image coding format, one pixel is equivalent to 3 bytes, and each byte occupies 8 bits. Then, one pixel requires 24 bits.

[0128] For example, taking a 3840*2160 RGB 24-bit map as an example, the number of bits (bit) required for each segment can be expressed by the following formula:

[0129] Per_segment_bits = segmented pixel value Data[24bit] + start position Spos[12bit] + end position Epos[12bit].

[0130] In the above formula, the start position and end position of each segment each require 12 bits.

[0131] Step 610: Determine the number of bits required for N segments as the product value of the number of bits required for this segment and N.

[0132] In the embodiments of the present application, since the segment information corresponding to each segment is composed of the segmented pixel value, start position, and end position corresponding to this segment, then the number of bits required for each segment is the same. Therefore, the number of bits required for N segments in step 610 is the number of bits required for one processing unit multiplied by N, that is, per_segment_bits*N.

[0133] Step 510: Based on the number of bits required for N segments, the number of bits required for other segments within multiple segments, and the number of bits required for the original pixel data within this processing unit, obtain the compression ratio of this processing unit.

[0134] In the embodiments of the present application, after obtaining the number of bits required for N segments, based on the number of bits required for other segments except N segments among multiple segments, the number of bits required for this processing unit after compression can be obtained.

[0135] For example, still taking a 3840*2160 RGB 24-bit map as an example, assuming row compression is used.

[0136] Then, the number of bits required for each processing unit (denoted as Per_line_bits, in this embodiment, one processing unit is the number of bits required for one row) can be expressed by the following formula:

[0137] Per_line_bits = (3840 - (Epos0 - Spos0)… - (EposN - SposN) * 24 + per_segment_bits * N + the number of bits required for the header information.

[0138] The number of bits required for the original pixel data within each processing unit (denoted as Org_line_bits, in this embodiment, a processing unit is the number of bits required for one line) can be expressed by the following formula:

[0139] Org_line_bits = number of pixels * number of bits required for pixel values + number of bits required for packet header information.

[0140] Then, the compression ratio CPR of each processing unit (in this embodiment, each line) can be expressed by the following formula:

[0141] CPR = Per_line_bits / Org_line_bits.

[0142] In the embodiment of the present application, after obtaining the compression ratio of the processing unit, the compression ratio is compared with the compression ratio threshold. If it is determined that the compression ratio is greater than the compression ratio threshold, step 520 is executed. If it is determined that the compression ratio is not greater than the compression ratio threshold, it means that by the above method, data compression of N segments has a too small compression ratio. Considering actual situations such as the time-consuming of specific compression operations, operation difficulty, transmission requirements, and transmission resource occupancy, it is preferably to transmit the original pixel data. Then, after determining that the compression ratio is not greater than the compression ratio threshold, specifically, the original pixel data within the processing unit is packed to obtain the original data packet corresponding to the processing unit. When subsequently transmitting the pixel data corresponding to the processing unit, the original data packet is transmitted to the receiving end.

[0143] Step 520: After determining that the compression ratio is greater than the compression ratio threshold, the following operations are performed for each of the N segments: Compress the segmented pixel values corresponding to any one of the N segments, as well as the start position and end position, to obtain the segmented compressed data corresponding to the segment.

[0144] In the embodiment of the present application, after determining that the compression ratio is greater than the compression ratio threshold, step 520 is executed to separately compress the N segments to obtain the segmented compressed data corresponding to the N segments. Among them, the segmented compressed data includes the segmented pixel values corresponding to the segment (denoted as Data), the start position (denoted as Spos), and the end position (denoted as Epos). The format of the segmented compressed data (denoted as Seg) is as Figure 7 shown.

[0145] Step 530: Pack the segmented compressed data corresponding to the N segments with the pixel values of each pixel in other segments to obtain the compressed data packet corresponding to the processing unit.

[0146] In the embodiment of the present application, after obtaining the segmented compression data corresponding to N segments, the pixel values of each pixel in other segments except the N segments in the processing unit are packed, and packet header information, etc. is added to obtain the compression data packet corresponding to the processing unit. The compression data packet includes indication information 1, indication information 2, compressed data, and original data. Among them, indication information 1 to indication information 2 are optionally added. The indication information can be packet header information indicating the state of the original data; the compressed data includes the segmented compression data corresponding to N segments, and the compressed data part is 8 bits. If the data is insufficient, it is padded with 0s; the original data includes the pixel values of each pixel in other segments except the N segments among the multiple segments; its data packet format is as Figure 8 shown.

[0147] Refer to Figure 8 shown. The compressed data in the compression data packet includes the segmented compression data of 4 segments, which are denoted as Seg0, Seg1, Seg2, and Seg3 in sequence; Seg0 includes segmented pixel values (Data0), start position (Spos0), and end position (Epos0); Seg1 includes segmented pixel values (Data1), start position (Spos1), and end position (Epos1); Seg2 includes segmented pixel values (Data2), start position (Spos2), and end position (Epos2); Seg3 includes segmented pixel values (Data3), start position (Spos3), and end position (Epos3).

[0148] In the embodiment of the present application, after step 520 determines that the compression ratio is greater than the compression ratio threshold, refer to Figure 9A shown, the following steps also need to be executed to determine the transmission control signal for subsequent transmission of the compression data packet:

[0149] Step 900: Generate a transmission control signal based on the compression ratio and the preset transmission protocol.

[0150] Step 910: Replace the preset transmission control signal with the transmission control signal.

[0151] In the embodiment of the present application, the preset transmission protocol is the currently adopted transmission protocol.

[0152] For example, as Figure 9B shown, take the sender and the receiver as an example.

[0153] Assume that there are 4 connection layers (i.e., the aforementioned transmission links) between the sender and the receiver, denoted as connection layer 1, connection layer 2, connection layer 3, and connection layer 4; and the compression ratio of the currently processed processing unit is 50%.

[0154] It is also assumed that based on the currently adopted transmission protocol, 4 connection layers need to work simultaneously.

[0155] Then, referring to Figure 9C as shown, when in-line segmented compression is not performed using an image data processing method in an embodiment of the present application, 4 connection layers work simultaneously, and the time required to transmit the target processing unit to be transmitted currently is t1, as Figure 9C shown in "a";

[0156] When in-line segmented compression is performed using an image data processing method in an embodiment of the present application, the number of compressed data packets of the target processing unit is half of the original. Then, when 4 connection layers work simultaneously, the time required to transmit the target processing unit is only half of t1, as Figure 9C shown in "b", that is, the 4 connection layers only need to work for half of the original time, and the remaining half of the time is in the air interface state, thereby improving the transmission rate and also achieving the purpose of saving transmission resources.

[0157] It should be noted that in the embodiments of the present application, the control state of the connection layer is not limited. Taking the Figure 1B display device shown as an example, Figure 9B the sending end shown is an image processing module, and the receiving end is a driving chip. Taking the Figure 2 application scenario of the server and the display device shown as an example, Figure 9B the sending end shown is the server, and the receiving end is the display device.

[0158] In some other feasible embodiments, when performing step 900 to step 910, the number of transmission links for transmitting the compressed data packet can also be determined based on the compression ratio and the total number of multiple candidate transmission links; select the above-mentioned number of candidate transmission links from the multiple candidate transmission links, and use each selected candidate transmission link as a target transmission link; then, generate a transmission control signal based on the currently adopted transmission protocol, and use the transmission control signal to replace the preset transmission control signal.

[0159] For example, taking the Figure 1B display device shown as an example.

[0160] It is still assumed that there are 4 connection layers between the image processing module and the driving chip, and the compression ratio of the currently processed processing unit is 50%.

[0161] Then, based on the compression ratio (50%), 4 * 50% = 2 connection layers are selected from the 4 connection layers and used as the target connection layers respectively.

[0162] Then, based on the currently adopted transmission protocol, a transmission control signal is generated, which is used to control the operation of each of the above-mentioned target connection layers and the radio interface state to complete the transmission of the compressed data packets. In this way, the other two idle connection layers can be used to transmit other data, so as to achieve the purpose of saving transmission resources.

[0163] In the embodiment of the present application, after determining that the compression ratio is greater than the compression ratio threshold, a transmission control signal is generated based on the compression ratio, which is used to control the operation of the transmission link between the transmitter and the receiver and the radio interface state, achieving the purpose of saving resources.

[0164] Step 320: Send each data packet of the video frame data to the receiver to display the image corresponding to the video frame data.

[0165] In the embodiment of the present application, when transmitting the compressed data packets obtained through step 530 in step 320, refer to Figure 10 as shown, and specifically execute the following steps:

[0166] Step 1000: Based on the transmission control signal, control the activation of each target transmission link.

[0167] In the embodiment of the present application, the target transmission link is denoted as the transmission link used to transmit the compressed data packet.

[0168] Step 1010: Send the compressed data packets to the receiver through each target transmission link.

[0169] Step 1020: After the activation duration reaches the duration included in the transmission control signal, deactivate each target transmission link.

[0170] In the embodiment of the present application, after the receiver receives the above-mentioned compressed data packets, it decompresses the compressed data packets to obtain indication information 1, indication information 2, compressed data, and original data. It decompresses the compressed data to obtain the segmented pixel values, start positions, and end positions of each segment. Then, based on the start positions and end positions of each segment in the compressed data and the original data, it splices the image data of the corresponding processing unit in the video frame data to be processed, and generates a driving signal based on the image data for the display panel to display.

[0171] In some other feasible embodiments, if it is determined that the compression ratio is not greater than the compression ratio threshold, the original pixel data in the processing unit is packed to obtain the original data packet corresponding to the processing unit. Then, when performing step 320 to transmit the original data packet, the following operations are specifically performed:

[0172] Operation 1: Based on a preset transmission control signal, control the activation of multiple candidate transmission links.

[0173] Operation 2: Transmit the original data packet to the receiving end via multiple candidate transmission links.

[0174] Operation 3: After the duration of the enabled preset transmission control signal reaches the duration included therein, close the multiple candidate transmission links.

[0175] It should be noted that the embodiments of the present application do not limit the transmission timing of each data packet of the video frame data. In some preferred embodiments, each time a data packet corresponding to a processing unit is obtained, a data packet is transmitted, and at the same time, the data packets corresponding to the subsequent processing units are calculated, thereby further improving the transmission rate.

[0176] Based on the same inventive concept, refer to Figure 11 As shown, an image data processing apparatus is provided in the embodiments of the present application, including:

[0177] An acquisition module 1110, configured to acquire video frame data to be processed, where the video frame data is image data corresponding to any video frame in a video stream to be displayed;

[0178] A segmentation and packaging module 1120, configured to perform the following operations for each processing unit of the video frame data: based on the pixel values of each pixel in any processing unit of the video frame data, perform in-line segmentation on each pixel, and based on the obtained multiple segments, compress and package the pixel information of each pixel to obtain a data packet corresponding to any processing unit, where the pixel positions of each pixel in each segment are continuous and the pixel values are the same;

[0179] A sending module 1130, configured to send each data packet of the video frame data to a receiving end to display the image corresponding to the video frame data.

[0180] In a possible implementation manner, the segmentation and packaging module 1120 is specifically configured to:

[0181] Perform the following operations for each pixel in any processing unit of the video frame data respectively:

[0182] For any pixel among the pixels, if the pixel value of the any pixel is the same as the pixel value of the previous pixel, then divide the any pixel into the segment where the previous pixel is located;

[0183] If the pixel value of the any pixel is different from the pixel value of the previous pixel, then divide the any pixel into a new segment.

[0184] In a possible implementation manner, the segmentation and packaging module 1120 is specifically configured to:

[0185] Select N segments with longer segment lengths from the multiple segments, where N is a positive integer;

[0186] Based on the number of bits required for the N segments, the number of bits required for other segments within the multiple segments, and the number of bits required for the original pixel data within any one processing unit, obtain the compression ratio of any one processing unit;

[0187] After determining that the compression ratio is greater than the compression ratio threshold, perform the following operations for each of the N segments: compress the segment pixel value, start position, and end position corresponding to any one of the N segments to obtain the segment compression data corresponding to any one of the N segments;

[0188] Pack the segment compression data corresponding to the N segments with the pixel values of each pixel within the other segments to obtain the compression data packet corresponding to any one processing unit.

[0189] In a possible implementation, the number of bits required for the N segments is obtained by the following method:

[0190] Determine the sum of the start position, end position, and the number of bits required for the segment pixel value corresponding to any one of the N segments as the number of bits required for any one of the N segments, where the start position is the pixel position of the first pixel in any one of the N segments, and the end position is the pixel position of the last pixel in any one of the N segments;

[0191] Determine the product value of the number of bits required for any one of the N segments and N as the number of bits required for the N segments.

[0192] In a possible implementation, after determining that the compression ratio is greater than the compression ratio threshold, the segment and packing module 1120 is further configured to:

[0193] Generate a transmission control signal based on the compression ratio and a preset transmission protocol;

[0194] Replace the preset transmission control signal with the transmission control signal;

[0195] The sending module 1130 is specifically configured to:

[0196] When transmitting the compression data packet, control the opening of each target transmission link based on the transmission control signal;

[0197] Send the compression data packet to the receiving end through each target transmission link;

[0198] After the opening duration reaches the duration included in the transmission control signal, close each target transmission link.

[0199] In a possible implementation, after obtaining the compression ratio of any one of the processing units, the segmentation and packaging module 1120 is further configured to:

[0200] After determining that the compression ratio is not greater than the compression ratio threshold, pack the original pixel data in any one of the processing units to obtain an original data packet corresponding to any one of the processing units;

[0201] The sending module 1130 is specifically configured to:

[0202] When transmitting the original data packet, based on a preset transmission control signal, control multiple candidate transmission links to be turned on;

[0203] Transmit the original data packet to the receiving end through the multiple candidate transmission links;

[0204] After the on duration reaches the duration included in the preset transmission control signal, turn off the multiple candidate transmission links.

[0205] Based on the same inventive concept, refer to Figure 12 As shown, an electronic device provided in an embodiment of the present application includes:

[0206] A memory 121 for storing computer programs or instructions;

[0207] A processor 122 for executing the computer programs or instructions in the memory 121, so that any one of the methods in the above various embodiments is executed.

[0208] Among them, the processor 122 may include one or more central processing units (Central Processing Unit, CPU), or digital processing units, etc., for executing the computer programs or instructions in the memory 121, so that any one of the methods in the above various embodiments is executed;

[0209] It should be noted that the specific connection medium between the memory 121 and the processor 122 is not limited in the embodiment of the present application. In the embodiment of the present application Figure 12 In, the memory 121 and the processor 122 are connected through a bus 123. The connection manners between other components are only for illustrative purposes and are not limited thereto. The bus 123 may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 12 In, only a thick line is used to represent it, but it does not mean that there is only one bus or one type of bus.

[0210] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor, the processor can execute any one of the methods in the above embodiments. Since the principle of the above computer-readable storage medium for solving problems is similar to that of an image data processing method, the implementation of the above computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be described again.

[0211] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0212] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0213] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device realizes the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0214] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Therefore, the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0215] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims

1. An image data processing method, characterized in that, Including: Obtain video frame data to be processed, where the video frame data is the image data corresponding to any video frame in the video stream to be displayed; Perform the following operations for each processing unit of the video frame data: Based on the pixel values of each pixel in any processing unit of the video frame data, perform in-line segmentation on each pixel, and based on the obtained multiple segments, compress and pack the pixel information of each pixel to obtain a data packet corresponding to any processing unit, where the pixel positions of each pixel in each segment are continuous and the pixel values are the same; Send the data packets of the video frame data to the receiving end to display the image corresponding to the video frame data.

2. The method according to claim 1, wherein The performing in-line segmentation on each pixel based on the pixel values of each pixel in any processing unit of the video frame data includes: Perform the following operations for each pixel in any processing unit of the video frame data respectively: For any pixel among the pixels, if the pixel value of the any pixel is the same as the pixel value of the previous pixel, then divide the any pixel into the segment where the previous pixel is located; If the pixel value of the any pixel is different from the pixel value of the previous pixel, then divide the any pixel into a new segment.

3. The method according to claim 1 or 2, characterized in that, The compressing and packing the pixel information of each pixel based on the obtained multiple segments to obtain a data packet corresponding to any processing unit includes: Select N segments with longer segment lengths from the multiple segments according to the segment length, where N is a positive integer; Based on the number of bits required for the N segments, the number of bits required for the other segments in the multiple segments, and the number of bits required for the original pixel data in any processing unit, obtain the compression ratio of any processing unit; After determining that the compression ratio is greater than the compression ratio threshold, perform the following operations for each of the N segments: Compress the segment pixel value, start position, and end position corresponding to any one of the N segments to obtain the segment compression data corresponding to any one of the N segments; Pack the segment compression data corresponding to the N segments and the pixel values of each pixel in the other segments to obtain the compressed data packet corresponding to any processing unit.

4. The method according to claim 3, wherein Obtain the number of bits required for the N segments through the following method: Determine the sum of the start position, end position, and the number of bits required for the segment pixel value corresponding to any one of the N segments as the number of bits required for any one of the N segments, where the start position is the pixel position of the first pixel in any one of the N segments, and the end position is the pixel position of the last pixel in any one of the N segments; Determine the product value of the number of bits required for any one of the N segments and N as the number of bits required for the N segments.

5. The method according to claim 3, characterized in that, After determining that the compression ratio is greater than the compression ratio threshold, it further includes: Generate a transmission control signal based on the compression ratio and a preset transmission protocol; Replace the preset transmission control signal with the transmission control signal; The sending the data packets of the video frame data to the receiving end includes: When transmitting the compressed data packets, based on the transmission control signal, control each target transmission link to be turned on; Through each of the target transmission links, send the compressed data packets to the receiving end; After the turn-on duration reaches the duration included in the transmission control signal, turn off each of the target transmission links.

6. The method according to claim 3, characterized in that, After obtaining the compression ratio of any one of the processing units, it further includes: After determining that the compression ratio is not greater than the compression ratio threshold, pack the original pixel data in any one of the processing units to obtain the original data packet corresponding to any one of the processing units; The sending the data packets of the video frame data to the receiving end includes: When transmitting the original data packet, based on a preset transmission control signal, control a plurality of candidate transmission links to be turned on; Through the plurality of candidate transmission links, transmit the original data packet to the receiving end; After the turn-on duration reaches the duration included in the preset transmission control signal, turn off the plurality of candidate transmission links.

7. An image data processing apparatus, characterized in that, It includes: An acquisition module, configured to acquire video frame data to be processed, where the video frame data is image data corresponding to any one video frame in a video stream to be displayed; A segmentation and packing module, configured to perform the following operations for each processing unit of the video frame data: based on the pixel values of each pixel in any one processing unit of the video frame data, perform in-line segmentation on each pixel, and based on the obtained multiple segments, compress and pack the pixel information of each pixel to obtain the data packet corresponding to any one processing unit, where the pixel positions of the pixels in each segment are continuous and the pixel values are the same; A sending module, configured to send the data packets of the video frame data to the receiving end to display the image corresponding to the video frame data.

8. The device according to claim 7, characterized in that, The segmentation and packing module is specifically configured to: Perform the following operations for each pixel in any one processing unit of the video frame data respectively: For any one pixel among the pixels, if the pixel value of the any one pixel is the same as the pixel value of the previous pixel, then divide the any one pixel into the segment where the previous pixel is located; If the pixel value of the any one pixel is different from the pixel value of the previous pixel, then divide the any one pixel into a new segment.

9. An electronic device, characterized in that, It includes: A memory, configured to store computer programs or instructions; A processor, configured to execute the computer programs or instructions in the memory, so that the method according to any one of claims 1-6 is executed.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor, the processor can execute the method according to any one of claims 1-6.

Images