An image frame processing method and device, computer equipment and a storage medium

Abstract

The present disclosure provides an image frame processing method and device, computer equipment and a storage medium, wherein the method comprises: acquiring a to-be-processed image frame set; respectively performing mixing processing on data rows of each to-be-processed image frame in the to-be-processed image frame set to obtain a mixed image frame set, wherein the data row is used to indicate data corresponding to each row of pixel points in the to-be-processed image frame; performing frame filling based on adjacent mixed image frames in the mixed image frame set to obtain a target image frame set, and displaying the target image frame set in a display interface.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more specifically, to an image frame processing method, a device, a computer equipment, and a storage medium. Background Technology

[0002] As television technology continues to develop, users' demands for television refresh rates are increasing. The method of improving the refresh rate of television displays through frequency doubling technology is becoming more and more common. For example, this frequency doubling technology can be HSR (Hardware Super Resolution) technology.

[0003] 0 When performing frequency multiplication using HSR technology, the image can be scanned separately using interlaced scanning.

[0004] The odd-numbered and even-numbered rows of pixels in a frame are scanned to obtain two image frames, namely odd-numbered frames and even-numbered frames, in order to achieve a doubled refresh rate, for example, to achieve a 120Hz display at a 60Hz refresh rate. However, the odd-numbered and even-numbered frames obtained by HSR technology are relative to the original image frames.

[0005] This often results in problems such as loss of outlines and loss of image features, leading to reduced image clarity and affecting the user's viewing experience. Summary of the Invention

[0006] This disclosure provides at least one image frame processing method, apparatus, computer device, and storage medium.

[0007] In a first aspect, embodiments of this disclosure provide an image frame processing method, including: 0. acquiring a set of image frames to be processed;

[0008] The data rows of each image frame in the image frame set to be processed are mixed to obtain a mixed image frame set, wherein the data rows are used to indicate the data corresponding to each row of pixels in the image frame to be processed;

[0009] Based on the adjacent mixed image frames in the mixed image frame set, frame interpolation is performed to obtain the target image frame set, and the target image frame set is displayed on the display interface.

[0010] In one optional implementation, the step of mixing the data rows of each image frame in the image frame set to obtain a mixed image frame set includes:

[0011] Determine adjacent data rows in the image frame to be processed;

[0012] The adjacent data rows are mixed to obtain a mixed image frame, and a set of mixed image frames is determined based on the mixed image frame.

[0013] In one optional implementation, determining adjacent data rows in the image frame to be processed includes:

[0014] Upon receiving the frame-skipping instruction, determine the number of mixed lines of frames indicated by the frame-skipping instruction;

[0015] Determine the adjacent data rows whose number of data rows satisfies the number of frame-mixed rows.

[0016] In one optional implementation, the step of performing mixing processing based on the adjacent data rows to obtain a mixed image frame includes:

[0017] Determine the first scan time, and collect partial data from each data row in the adjacent data rows based on the first scan time;

[0018] Based on the aforementioned partial data, a mixed data row corresponding to the adjacent data rows is obtained;

[0019] The mixed image frame is determined based on the mixed data rows.

[0020] In one optional implementation, the step of performing frame interpolation on adjacent mixed image frames in the mixed image frame set to obtain the target image frame set includes:

[0021] Identify the corresponding moving objects in the adjacent mixed image frames;

[0022] Determine the displacement information of the moving object in the adjacent mixed image frames;

[0023] Based on the displacement information, the estimated position of the moving object between adjacent mixed image frames is calculated;

[0024] Based on the moving object and the estimated position, a compensation image frame is determined and inserted between adjacent image frames to obtain a set of target image frames.

[0025] In one optional implementation, displaying the target image frame set on the display interface includes:

[0026] Determine the target resolution, and based on the target resolution, determine the data rows to be supplemented for the target image frames in the target image frame set;

[0027] Based on the data position of the data row to be supplemented in the target image frame, determine the associated data row corresponding to the data row to be supplemented;

[0028] Based on at least a portion of the data in the associated data row, the data row to be supplemented is supplemented to obtain the scanned image frame corresponding to the target image frame, and the scanned image frame is displayed on the display interface.

[0029] In one optional implementation, the step of supplementing the data row to be supplemented based on at least a portion of the data in the associated data row to obtain the scanned image frame corresponding to the target image frame includes:

[0030] Determine the second scan time corresponding to the data row to be supplemented;

[0031] Based on the second scan time, the associated data row is scanned to obtain at least a portion of the data;

[0032] The at least part of the data is added to the data row to be supplemented to obtain the scanned image frame corresponding to the target image frame.

[0033] Secondly, embodiments of this disclosure also provide an image frame processing apparatus, comprising:

[0034] The acquisition unit is used to acquire a set of image frames to be processed.

[0035] A mixing unit is used to perform mixing processing on the data rows of each image frame in the image frame set to be processed to obtain a mixed image frame set, wherein the data rows are used to indicate the data corresponding to each row of pixels in the image frame to be processed;

[0036] The frame interpolation unit is used to interpolate adjacent mixed image frames in the mixed image frame set to obtain a target image frame set, and to display the target image frame set on the display interface.

[0037] Thirdly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0038] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the first aspect or any possible implementation thereof.

[0039] In this embodiment of the present disclosure, a set of image frames to be processed can first be obtained, and each image frame in the set can be mixed to obtain a set of mixed image frames. Next, adjacent mixed image frames in the set of mixed image frames can be interpolated to obtain a set of target image frames. The set of target image frames is then displayed on the display interface. This achieves frequency multiplication processing of the set of image frames to be processed, while reducing the loss of image features in each image frame to be processed, improving the image clarity, and thus improving the user's viewing experience.

[0040] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0042] Figure 1 A flowchart of an image frame processing method provided by an embodiment of this disclosure is shown;

[0043] Figure 2 A schematic diagram of a data acquisition method for parametric scanning provided in an embodiment of this disclosure is shown;

[0044] Figure 3a A schematic diagram is shown illustrating the insertion of a 4k1k120Hz compensated image frame after a 4k1k120Hz mixed image frame, as provided in an embodiment of this disclosure.

[0045] Figure 3b This illustration shows a schematic diagram of frame extraction of an image frame to be processed based on a frequency multiplication technique related to this disclosure, provided in an embodiment of the present disclosure.

[0046] Figure 4 This illustration shows a schematic diagram of determining the data row to be supplemented in a target image frame according to an embodiment of the present disclosure;

[0047] Figure 5 A schematic diagram of the architecture of the image frame processing system provided in the embodiments of this disclosure is shown;

[0048] Figure 6 A schematic diagram of an image frame processing apparatus provided in an embodiment of the present disclosure is shown;

[0049] Figure 7 A schematic diagram of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0051] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0052] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0053] Research has found that (as television technology continues to develop, users' demands for television refresh rates are also increasing, and the method of improving the refresh rate of television displays through frequency doubling technology is becoming more and more common, for example, this frequency doubling technology can be HSR (Hardware Super Resolution) technology).

[0054] When using HSR (High-Refresh-Rate) technology to double the refresh rate, the odd-numbered rows and even-numbered rows of pixels in an image frame are scanned separately to obtain two image frames, namely odd-numbered frames and even-numbered frames, thus achieving a 120Hz display at a 60Hz refresh rate. However, the odd-numbered and even-numbered frames obtained through HSR technology often suffer from problems such as loss of outlines and image features compared to the original image frames, resulting in reduced image clarity and affecting the user's viewing experience.

[0055] Based on the above research, this disclosure provides an image frame processing method, apparatus, computer device, and storage medium. In the embodiments of this disclosure, a set of image frames to be processed is first obtained, and each image frame in the set is then mixed to obtain a mixed image frame set. Next, adjacent mixed image frames in the mixed image frame set are interpolated to obtain a target image frame set, which is then displayed on the display interface. This achieves frequency multiplication processing of the image frame set to be processed while reducing the loss of image features in each individual image frame, improving image clarity, and thus enhancing the user's viewing experience.

[0056] To facilitate understanding of this embodiment, a detailed description of the image frame processing method disclosed in this disclosure will be provided first. The execution subject of the image frame processing method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device with a display screen, such as a television set or a mobile terminal. In some possible implementations, the image frame processing method can be implemented by a processor calling computer-readable instructions stored in memory.

[0057] See Figure 1 The diagram shows a flowchart of an image frame processing method provided in an embodiment of this disclosure. The method includes steps S101 to S105, wherein:

[0058] S101: Obtain the collection of image frames to be processed.

[0059] In this embodiment of the disclosure, the set of image frames to be processed can first be determined based on the video played on the display interface of the television. Specifically, the preloaded or all content in the video can be obtained. It should be understood that the video is composed of continuous image frames. Therefore, the image frames to be processed included in the obtained content can be determined to obtain the set of image frames to be processed.

[0060] Here, the resolution of the image frame to be processed is adapted to the resolution of the TV's display interface. For example, when the resolution of the TV's display interface is 4k, the resolution of the image frame to be processed can be 4k2k (i.e., 4096×2160 pixels).

[0061] S103: Perform mixing processing on the data rows of each image frame in the image frame set to be processed to obtain a mixed image frame set, wherein the data rows are used to indicate the data corresponding to each row of pixels in the image frame to be processed.

[0062] In this embodiment of the disclosure, when the TCON (Timing Controller) of the television drives the screen to display image frames, the scanning time for each image frame is fixed. The scanning time for each data line in the image frame is also the same. Therefore, in this disclosure, frequency doubling is achieved by halving the scanning time required for each image frame.

[0063] Based on this, a hybrid image frame with half the vertical resolution can be obtained by mixing the data rows in the image frame to be processed. Specifically, adjacent data rows in the image frame to be processed can be mixed pairwise, thereby preserving as many image features as possible in adjacent data rows while halving the vertical resolution. For example, if the resolution of the image frame to be processed is 4096×2160 (i.e., 4k2k), with 2160 data rows and each data row corresponding to 4096 pixels, then after mixing the image frame to be processed, the resolution of the resulting hybrid image frame can be 4096×1080 (i.e., 4k1k). If the refresh rate of the image frame to be processed is 60Hz, then the refresh rate of the hybrid image frame can reach 120Hz. The specific mixing process is described below and will not be repeated here.

[0064] S105: Perform frame interpolation on adjacent mixed image frames in the mixed image frame set to obtain a target image frame set, and display the target image frame set on the display interface.

[0065] In this embodiment of the disclosure, since the refresh rate corresponding to the mixed image frame set is doubled compared to the image frame set to be processed, it is necessary to double the number of mixed image frames contained in the mixed image frame set in order to meet the refresh rate requirements after the frequency doubling.

[0066] As described above, in the background technology related to this disclosure, interlaced scanning is used to scan the odd-numbered rows and even-numbered rows of pixels in an image frame separately, thereby obtaining two new image frames: an odd-numbered frame and an even-numbered frame. This doubles the number of image frames to meet the refresh rate requirements after frequency doubling. However, this frequency doubling method reduces image clarity and affects the user's viewing experience.

[0067] Based on this, this disclosure can double the number of mixed image frames by inserting one image frame into each pair of mixed image frames in the mixed image frame set through frame interpolation, thereby obtaining the target image frame set. Specifically, MEMC (Motion Estimate and Motion Compensation) technology can be used to estimate the image frames between two adjacent image frames to obtain compensated image frames, which are then inserted between the two adjacent image frames. This improves both image clarity and smoothness. The specific method of estimating compensated image frames using MEMC technology is described below and will not be repeated here.

[0068] As described above, in this embodiment of the present disclosure, a set of image frames to be processed can first be obtained, and each image frame in the set can be mixed to obtain a set of mixed image frames. Next, adjacent mixed image frames in the set of mixed image frames can be interpolated to obtain a set of target image frames, and the set of target image frames can be displayed on the display interface. In this way, while realizing the frequency multiplication processing of the set of image frames to be processed, the loss of image features in each image frame to be processed is reduced, the image clarity is improved, and the user's viewing experience is improved.

[0069] In an optional implementation, step S103 above, which involves mixing the data rows of each image frame in the image frame set to be processed to obtain a mixed image frame set, specifically includes the following process:

[0070] S1031: Determine adjacent data rows in the image frame to be processed.

[0071] In this embodiment of the disclosure, the number of adjacent data rows can first be determined, for example, two rows, three rows, etc., and adjacent data rows in the image frame to be processed can be determined based on this number. Specifically, if the determined number is two rows and the resolution of the image frame to be processed is 4k2k, then the first row and the second row can be determined as a group of adjacent data rows, the third row and the fourth row can be determined as a group of adjacent data rows, and so on, until 1080 groups of adjacent data rows are determined in the image frame to be processed.

[0072] S1032: Perform a mixing process based on the adjacent data rows to obtain a mixed image frame, and determine a set of mixed image frames based on the mixed image frame.

[0073] In the process of mixing adjacent data rows, the amount of data to be acquired in each adjacent data row can first be determined. Specifically, this can be based on the number of data rows in the adjacent data row. For example, if there are two data rows in the adjacent data row, the amount of data acquired in each data row can be half of the total data volume in that data row. Next, at least a portion of the data that meets this amount can be acquired from each adjacent data row, and this portion of data can be mixed to obtain the mixed data row corresponding to the adjacent data row. Based on the mixed data row, the mixed image frame corresponding to the image frame to be processed can be determined.

[0074] In this embodiment of the disclosure, adjacent data rows in the image frame to be processed can be mixed to obtain a mixed data row, and a mixed image frame can be determined based on the mixed data row, thereby reducing the image features discarded during the mixing process and improving the clarity of the mixed image frame.

[0075] In an optional implementation, step S1031 above, determining adjacent data rows in the image frame to be processed, specifically includes the following process:

[0076] (1) After receiving the frame-skipping instruction, determine the number of mixed lines of frames indicated by the frame-skipping instruction;

[0077] (2) Determine the adjacent data rows whose number of data rows satisfies the number of frame-sampling mixed rows.

[0078] In this embodiment of the disclosure, considering that in actual use scenarios, the video sources played on the television may differ, and therefore the refresh rates of the videos may also be different. Thus, even when the target refresh rate for the frequency multiplication processing is the same, the frequency multiplication factor corresponding to the set of image frames to be processed for each video may also be different. Simultaneously, the target refresh rate may also change, in which case the frequency multiplication factor corresponding to the set of image frames to be processed will also change.

[0079] Here, during frequency multiplication, a frame extraction command can be used to instruct the image frames in the set to be processed to be mixed. Specifically, the frequency multiplication factor indicated by the frame extraction command can first be determined, and the number of rows to be mixed can be determined based on this factor. For example, when the multiplication factor is two, the number of data rows in the adjacent data rows indicated by the number of rows to be mixed can be two rows; when the multiplication factor is two, the number of data rows in the adjacent data rows indicated by the number of rows to be mixed can be three rows.

[0080] In this embodiment of the disclosure, it is considered that in actual scenarios, the refresh rates corresponding to the collection of image frames to be processed in different videos may be different, and the target refresh rate corresponding to the frequency multiplication processing may change. Therefore, the frequency multiplication processing multiplier corresponding to the collection of image frames to be processed can be indicated by the frame extraction instruction, so as to determine the number of mixed lines of frame extraction based on the multiplier. This makes the frequency multiplication processing scheme in this disclosure more flexible, adaptable to more diverse scenarios, increases the number of video playback platforms that can be adapted, and meets the diverse video viewing needs of users.

[0081] In an optional implementation, step S1032 above, which involves mixing adjacent data rows to obtain a mixed image frame, specifically includes the following process:

[0082] (1) Determine the first scan time, and collect partial data of each data row in the adjacent data rows based on the first scan time;

[0083] (2) Based on the partial data, perform mixing processing to obtain the mixed data row corresponding to the adjacent data row;

[0084] (3) Determine the mixed image frame based on the mixed data row.

[0085] In this embodiment of the disclosure, the first scan time can be used to indicate the time for scanning data in adjacent data rows. The first scan time can be determined based on a timing sequence pre-set for the image frame to be processed. This timing sequence can be used to indicate the scanning order and fixed scan time for each data row in the image frame to be processed.

[0086] In practical implementation, as shown above, during the mixed processing of adjacent data rows, the amount of data to be acquired in each row of adjacent data rows can first be determined, and the first scan time can be determined based on this amount of data. For example, if the amount of data is half of the total amount of data in each adjacent data row, then the first scan time can be half of the aforementioned fixed scan time. Furthermore, the data acquisition methods corresponding to different scan sequences are as follows:

[0087] Method 1: The scanning order is a parameterless scan.

[0088] In the embodiments disclosed herein, such as Figure 2The diagram illustrates this parameter-free scanning data acquisition method, where the first and second data rows are adjacent, and the third and fourth data rows are adjacent. Specifically, since the first scan time is half of a preset fixed scan time, data acquisition for the first data row can begin after half of it has been scanned, and data acquisition for the second data row can be stopped after half of it has been scanned. Furthermore, portions of the data acquired from each data row during this acquisition process are mixed to obtain the desired result. Figure 2 The mixed data row shown.

[0089] Method 2: The scanning order is based on the scanning parameters.

[0090] In this embodiment of the present disclosure, the target object can first be determined in the image frame to be processed. For example, the target object can be a moving object in the image frame to be processed. Next, the scanning parameters can be determined based on the target object, so as to determine the part of the data corresponding to the target object in the adjacent data row based on the scanning parameters, and the part of the data is collected within the first scanning time.

[0091] It should be understood that in the process of determining the partial data corresponding to the target object, the determination can be based on the first scan time mentioned above, so that the acquisition time of the determined partial data can match the first scan time.

[0092] After identifying the mixed data rows, the corresponding mixed image frame for the image frame to be processed can be determined based on these mixed data rows. Specifically, the mixed data rows obtained after mixing processing can be combined based on the acquisition timing of the data in each mixed data row to obtain the mixed image frame, ensuring that the arrangement order of the mixed data rows in the mixed image frame matches the arrangement order of the data rows in the image frame to be processed.

[0093] In this embodiment of the disclosure, partial data of each data row in adjacent data rows can be collected based on the first scanning time, and the partial data can be mixed to obtain the mixed data row corresponding to the adjacent data row, thereby increasing the image features retained in the mixed data row and improving the clarity of the mixed image frame.

[0094] In an optional implementation, step S105 above, which involves interpolating adjacent mixed image frames in the mixed image frame set to obtain the target image frame set, specifically includes the following process:

[0095] S1051: Determine the corresponding moving object in the adjacent mixed image frames.

[0096] In this embodiment of the disclosure, two adjacent mixed image frames may include moving objects and stationary objects. Moving objects are those whose relative positions change between the two mixed image frames, while stationary objects are the opposite of moving objects. It should be understood that because the refresh rate of the mixed image frame set may be insufficient, displaying this set on a screen may result in choppy motion effects of the moving objects, low dynamic clarity, and ghosting.

[0097] Based on this, we can first identify the objects included in adjacent mixed image frames, and determine the same objects in the two mixed image frames based on the identification results. Next, we can determine whether the relative position of the same object has changed, and identify the moving objects whose relative positions have changed.

[0098] S1052: Determine the displacement information of the moving object in the adjacent mixed image frames.

[0099] S1053: Based on the displacement information, calculate the estimated position of the moving object between adjacent mixed image frames.

[0100] S1054: Based on the moving object and the estimated position, determine the compensation image frame and insert the compensation image frame between the adjacent mixed image frames to obtain the target image frame set.

[0101] As can be seen from the above, in the embodiments of this disclosure, the compensation image frame between adjacent mixed image frames can be determined by MEMC technology. Here, the estimated position of the moving object between adjacent mixed image frames can first be determined based on the displacement information of the moving object between adjacent mixed image frames.

[0102] In practice, the displacement information of the moving object between adjacent blended image frames can be determined first. Here, the coordinates of the moving object in each adjacent blended image frame can be determined, and the coordinate difference can be calculated to determine the displacement information based on the coordinate difference.

[0103] Next, a compensation coefficient can be determined based on the number of compensation image frames, and the estimated position of the moving image frame between adjacent blended image frames can be calculated based on this compensation coefficient. Specifically, the estimated position can be determined based on the product of the coordinate difference and the compensation coefficient. For example, if the number of inserted compensation image frames is one frame, the compensation coefficient can be 1 / 2, and the estimated position of the moving object can be determined based on 1 / 2 of the coordinate difference.

[0104] After determining the compensation image frame, it can be inserted between adjacent blended image frames to obtain the target image frame set. Specifically, for example... Figure 3aThe diagram shows a 4k1k120Hz compensation image frame inserted after the 4k1k120Hz mixed image frame 1. The compensation image frame is determined based on the adjacent mixed image frame 1 and mixed image frame 2. The mixed image frame 1 is obtained after mixing the 4k2k60Hz image frame 1 to be processed.

[0105] Other, such as Figure 3b The diagram shows a frame extraction of an image frame to be processed based on a frequency multiplication technique related to this disclosure. In this process, the odd-numbered rows of pixels and the even-numbered rows of pixels in the 4k2k60Hz image frame to be processed can be scanned by interlacing to obtain two new image frames: an odd-numbered frame and an even-numbered frame of 4k1k120Hz.

[0106] In this embodiment of the disclosure, by Figure 3a as well as Figure 3b As can be seen, the mixed image frames obtained based on the frequency doubling method provided in this disclosure contain more image features, which improves the image clarity. At the same time, by adding compensation image frames, the smoothness of the motion effect of moving objects can be improved, dynamic clarity can be enhanced, and image ghosting can be reduced.

[0107] In an optional implementation, step S105, which displays the target image frame set on the display interface, specifically includes the following process:

[0108] S1053: Determine the target resolution, and based on the target resolution, determine the data rows to be supplemented for the target image frames in the target image frame set.

[0109] In this embodiment of the disclosure, the target resolution is the resolution of the display interface. The resolution of the image frames to be processed in the aforementioned set of image frames to be processed is matched with the target resolution. However, as can be seen from the above, after frequency multiplication of the image frames to be processed, the vertical resolution of the target image frames in the resulting set of target image frames is halved. Therefore, it is necessary to supplement the target image frames with data rows to make the resolution of the target image frames reach the target resolution. Specifically, as shown... Figure 4 The diagram illustrates the process of determining the data rows to be supplemented in a target image frame. The data rows to be supplemented can be inserted between adjacent data rows in the target image frame. Here, line1 is data1 in the target image frame, line3 is data2 in the target image frame, line5 is data3 in the target image frame, line7 is data4 in the target image frame, and lines2, 4, and 6 are the data rows to be supplemented.

[0110] S1054: Based on the data position of the data row to be supplemented in the target image frame, determine the associated data row corresponding to the data row to be supplemented.

[0111] S1055: Based on at least a portion of the data in the associated data row, supplement the data row to be supplemented to obtain the scanned image frame corresponding to the target image frame, and display the scanned image frame on the display interface.

[0112] After identifying the data row to be supplemented, the corresponding associated data row can be determined based on its data position in the target image frame. This associated data row can be an adjacent data row, and data can be supplemented based on this adjacent data row. After supplementing the data row to be supplemented in the target image frame, the following can be obtained: Figure 4 The scanned image frame shown has a resolution that matches the target resolution of the display interface. The specific method for supplementing the rows of data to be supplemented is described below and will not be repeated here.

[0113] In this embodiment of the disclosure, the data rows to be supplemented in the target image frame can be supplemented, thereby improving the resolution of the target image frame and thus improving the user's viewing experience.

[0114] In an optional implementation, step S1055 above, which involves supplementing the data row to be supplemented based on at least a portion of the data in the associated data row to obtain the scanned image frame corresponding to the target image frame, specifically includes the following process:

[0115] (1) Determine the second scan time corresponding to the data row to be supplemented;

[0116] (2) Based on the second scan time, the associated data row is scanned to obtain at least a portion of the data;

[0117] (3) The at least part of the data is added to the data row to be added to obtain the scanned image frame corresponding to the target image frame.

[0118] In this embodiment of the disclosure, at least a portion of the data in the associated data row can be collected using HSR scanning technology, and the at least a portion of the data can be added to the data row to be added. Here, the second scanning time corresponding to the data row to be added can first be determined. As can be seen from the above, the scanning time for each data row in the target image frame is fixed. Therefore, the second scanning time can be determined as the scanning time for each data row.

[0119] Next, the corresponding scan order for the data row to be supplemented can be determined, and based on this scan order, the order of at least a portion of the data collected in the associated data row can be determined. For example, this scan order can be the aforementioned parameterless scan, in the data row to be supplemented... Figure 4 In the case of line 2, data collection for data1 can be started when half of the associated data line data1 has been scanned, so as to collect half of the data in data1. Similarly, data collection for data2 can be stopped after half of the associated data line data2 has been scanned, so as to collect half of the data in data2.

[0120] After collecting half of the data from data1 and half of the data from data2, the two can be mixed and added to the data line to be supplemented, so that the data line to be supplemented, line 2, includes the image features from data1 and data2. Similarly, at least some data can be collected from the associated data lines data2 and data3 to supplement the data line to be supplemented, and at least some data can be collected from the associated data lines data3 and data4 to supplement the data line to be supplemented, line 6.

[0121] In this embodiment of the disclosure, the data rows to be supplemented in the target image frame can be supplemented, thereby improving the resolution of the target image frame and thus improving the user's viewing experience.

[0122] In summary, in this embodiment of the present disclosure, a set of image frames to be processed can first be obtained, and each image frame in the set can be mixed to obtain a set of mixed image frames. Next, adjacent mixed image frames in the set of mixed image frames can be interpolated to obtain a set of target image frames, and the set of target image frames can be displayed on the display interface. Thus, while realizing the frequency multiplication processing of the set of image frames to be processed, the loss of image features in each image frame to be processed is reduced, the image clarity is improved, and the user's viewing experience is enhanced.

[0123] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0124] Based on the same inventive concept, this disclosure also provides an image frame processing system corresponding to the image frame processing method, see below. Figure 5 As shown, the image frame processing system includes: a SOC (System on Chip) module, a MEMC module, a TCON module, and a panel module, wherein:

[0125] When the input set of image frames to be processed consists of 4k2k60Hz (4080×2160 resolution, 60Hz refresh rate) image frames, the SOC module can perform the above-mentioned mixing process on each image frame in the set to be processed to obtain a set of mixed image frames, wherein the mixed image frames in the set are 4k1k60Hz (4080×1080 resolution, 60Hz refresh rate) image frames.

[0126] The MEMC module performs frame interpolation on adjacent mixed image frames in the above mixed image frame set to obtain a target image frame set, wherein the target image frames in the target image frame set are 4k1k120Hz (resolution 4080×1080, refresh rate 120Hz) image frames.

[0127] The TCON module determines the scan image frame for the target image frames in the above target image frame set, wherein the scan image frame is a 4k2k120Hz image frame.

[0128] The panel module displays the scanned image frames at a 120Hz refresh rate.

[0129] In this embodiment of the present disclosure, a set of image frames to be processed can first be obtained, and each image frame in the set can be mixed to obtain a set of mixed image frames. Next, adjacent mixed image frames in the set of mixed image frames can be interpolated to obtain a set of target image frames. The set of target image frames is then displayed on the display interface. This achieves frequency multiplication processing of the set of image frames to be processed, while reducing the loss of image features in each image frame to be processed, improving the image clarity, and thus improving the user's viewing experience.

[0130] Based on the same inventive concept, this disclosure also provides an image frame processing apparatus corresponding to the image frame processing method. Since the principle of the apparatus in this disclosure for solving the problem is similar to the image frame processing method described above in this disclosure, the implementation of the apparatus can refer to the implementation of the method, and the repeated parts will not be described again.

[0131] Reference Figure 6 The diagram shown is a schematic representation of an image frame processing apparatus provided in an embodiment of this disclosure. The apparatus includes: an acquisition unit 61, a mixing unit 62, and a frame interpolation unit 63; wherein,

[0132] Acquisition unit 61 is used to acquire a set of image frames to be processed;

[0133] The mixing unit 62 is used to perform mixing processing on the data rows of each image frame to be processed in the image frame to be processed set to obtain a mixed image frame set, wherein the data rows are used to indicate the data corresponding to each row of pixels in the image frame to be processed;

[0134] The frame interpolation unit 63 is used to interpolate adjacent mixed image frames in the mixed image frame set to obtain a target image frame set, and to display the target image frame set on the display interface.

[0135] In this embodiment of the present disclosure, a set of image frames to be processed can first be obtained, and each image frame in the set can be mixed to obtain a set of mixed image frames. Next, adjacent mixed image frames in the set of mixed image frames can be interpolated to obtain a set of target image frames. The set of target image frames is then displayed on the display interface. This achieves frequency multiplication processing of the set of image frames to be processed, while reducing the loss of image features in each image frame to be processed, improving the image clarity, and thus improving the user's viewing experience.

[0136] In one possible implementation, the mixing unit 62 is further configured to:

[0137] Determine adjacent data rows in the image frame to be processed;

[0138] The adjacent data rows are mixed to obtain a mixed image frame, and a set of mixed image frames is determined based on the mixed image frame.

[0139] In one possible implementation, the mixing unit 62 is further configured to:

[0140] Upon receiving the frame-skipping instruction, determine the number of mixed lines of frames indicated by the frame-skipping instruction;

[0141] Determine the adjacent data rows whose number of data rows satisfies the number of frame-mixed rows.

[0142] In one possible implementation, the mixing unit 62 is further configured to:

[0143] Determine the first scan time, and collect partial data from each data row in the adjacent data rows based on the first scan time;

[0144] Based on the aforementioned partial data, a mixed data row corresponding to the adjacent data rows is obtained;

[0145] The mixed image frame is determined based on the mixed data rows.

[0146] In one possible implementation, the frame interpolation unit 63 is further configured to:

[0147] Identify the corresponding moving objects in the adjacent mixed image frames;

[0148] Determine the displacement information of the moving object in the adjacent mixed image frames;

[0149] Based on the displacement information, the estimated position of the moving object between adjacent mixed image frames is calculated;

[0150] Based on the moving object and the estimated position, a compensation image frame is determined and inserted between adjacent image frames to obtain a set of target image frames.

[0151] In one possible implementation, the frame interpolation unit 63 is further configured to:

[0152] Determine the target resolution, and based on the target resolution, determine the data rows to be supplemented for the target image frames in the target image frame set;

[0153] Based on the data position of the data row to be supplemented in the target image frame, determine the associated data row corresponding to the data row to be supplemented;

[0154] Based on at least a portion of the data in the associated data row, the data row to be supplemented is supplemented to obtain the scanned image frame corresponding to the target image frame, and the scanned image frame is displayed on the display interface.

[0155] In one possible implementation, the frame interpolation unit 63 is further configured to:

[0156] Determine the second scan time corresponding to the data row to be supplemented;

[0157] Based on the second scan time, the associated data row is scanned to obtain at least a portion of the data;

[0158] The at least part of the data is added to the data row to be supplemented to obtain the scanned image frame corresponding to the target image frame.

[0159] The description of the processing flow of each unit in the device and the interaction flow between each unit can be found in the relevant descriptions in the above method embodiments, and will not be detailed here.

[0160] Corresponding to Figure 1 In the image frame processing method described in this disclosure, a computer device 700 is also provided, such as... Figure 7 The diagram shown is a structural schematic of a computer device 700 provided in an embodiment of this disclosure, including:

[0161] The computer device 700 comprises a processor 71, a memory 72, and a bus 73. The memory 72 stores execution instructions and includes main memory 721 and external memory 722. The main memory 721, also called internal memory, temporarily stores the processing data in the processor 71, as well as data exchanged with external memory such as a hard disk. The processor 71 exchanges data with the external memory 722 through the main memory 721. When the computer device 700 is running, the processor 71 communicates with the memory 72 through the bus 73, causing the processor 71 to execute the following instructions:

[0162] Obtain the collection of image frames to be processed;

[0163] The data rows of each image frame in the image frame set to be processed are mixed to obtain a mixed image frame set, wherein the data rows are used to indicate the data corresponding to each row of pixels in the image frame to be processed;

[0164] Based on the adjacent mixed image frames in the mixed image frame set, frame interpolation is performed to obtain the target image frame set, and the target image frame set is displayed on the display interface.

[0165] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the image frame processing method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0166] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the image frame processing method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0167] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0168] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0169] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0171] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0172] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. An image frame processing method, characterized in that, include: Obtain the collection of image frames to be processed; The data rows of each image frame in the image frame set to be processed are mixed to obtain a mixed image frame set, wherein the data rows are used to indicate the data corresponding to each row of pixels in the image frame to be processed; The step of mixing the data rows of each image frame in the image frame set to be processed to obtain a mixed image frame set includes: determining adjacent data rows in the image frames to be processed; performing mixing processing based on the adjacent data rows to obtain mixed image frames; and determining a mixed image frame set based on the mixed image frames. The step of performing a mixing process based on the adjacent data rows to obtain a mixed image frame includes: determining a target object in the image frame to be processed; determining scanning parameters based on the target object; determining a first scanning time; determining partial data of the target object in each data row of the adjacent data rows based on the scanning parameters; and collecting partial data of the target object in each data row of the adjacent data rows based on the first scanning time; performing a mixing process based on the collected partial data of the target object in each data row of the adjacent data rows to obtain a mixed data row corresponding to the adjacent data rows; and determining the mixed image frame based on the mixed data row. Based on the adjacent mixed image frames in the mixed image frame set, frame interpolation is performed to obtain the target image frame set, and the target image frame set is displayed on the display interface.

2. The method according to claim 1, characterized in that, Determining adjacent data rows in the image frame to be processed includes: Upon receiving the frame-skipping instruction, determine the number of mixed lines of frames indicated by the frame-skipping instruction; Determine the adjacent data rows whose number of data rows satisfies the number of frame-mixed rows.

3. The method according to claim 1, characterized in that, The step of performing frame interpolation on adjacent mixed image frames in the mixed image frame set to obtain the target image frame set includes: Identify the corresponding moving objects in the adjacent mixed image frames; Determine the displacement information of the moving object in the adjacent mixed image frames; Based on the displacement information, the estimated position of the moving object between adjacent mixed image frames is calculated; Based on the moving object and the estimated position, a compensation image frame is determined and inserted between the adjacent mixed image frames to obtain a set of target image frames.

4. The method according to claim 1, characterized in that, The step of displaying the target image frame set on the display interface includes: Determine the target resolution, and based on the target resolution, determine the data rows to be supplemented for the target image frames in the target image frame set; Based on the data position of the data row to be supplemented in the target image frame, determine the associated data row corresponding to the data row to be supplemented; Based on at least a portion of the data in the associated data row, the data row to be supplemented is supplemented to obtain the scanned image frame corresponding to the target image frame, and the scanned image frame is displayed on the display interface.

5. The method according to claim 4, characterized in that, The step of supplementing the data row to be supplemented based on at least a portion of the data in the associated data row to obtain the scanned image frame corresponding to the target image frame includes: Determine the second scan time corresponding to the data row to be supplemented; Based on the second scan time, the associated data row is scanned to obtain at least a portion of the data; The at least part of the data is added to the data row to be supplemented to obtain the scanned image frame corresponding to the target image frame.

6. An image frame processing apparatus, characterized in that, include: The acquisition unit is used to acquire a set of image frames to be processed. A mixing unit is used to perform mixing processing on the data rows of each image frame in the image frame set to be processed to obtain a mixed image frame set, wherein the data rows are used to indicate the data corresponding to each row of pixels in the image frame to be processed; The step of mixing the data rows of each image frame in the image frame set to be processed to obtain a mixed image frame set includes: determining adjacent data rows in the image frames to be processed; performing mixing processing based on the adjacent data rows to obtain mixed image frames; and determining a mixed image frame set based on the mixed image frames. The step of performing a mixing process based on the adjacent data rows to obtain a mixed image frame includes: determining a target object in the image frame to be processed; determining scanning parameters based on the target object; determining a first scanning time; determining partial data of the target object in each data row of the adjacent data rows based on the scanning parameters; and collecting partial data of the target object in each data row of the adjacent data rows based on the first scanning time; performing a mixing process based on the collected partial data of the target object in each data row of the adjacent data rows to obtain a mixed data row corresponding to the adjacent data rows; and determining the mixed image frame based on the mixed data row. The frame interpolation unit is used to interpolate adjacent mixed image frames in the mixed image frame set to obtain a target image frame set, and to display the target image frame set on the display interface.

7. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the image frame processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image frame processing method as described in any one of claims 1 to 5.

Images