Video preview frame generation method and device, equipment, medium and product

Abstract

The invention discloses a video preview frame generation method and device, equipment, a medium and a product, and relates to the technical field of image processing, and the method comprises the steps: determining original pixel lines contained in a collected original video frame, and selecting a target pixel line from the original pixel lines according to a preset step length; determining the number of original in-line pixels of original in-line pixels contained in the target pixel line, and determining the number of standard in-line pixels required for generating the video preview frame; according to the number of the pixels in the original row and the number of the pixels in the standard row, determining a pixel sampling template in a target row from preset pixel sampling templates in candidate rows, and sampling the pixels in the original row according to the pixel sampling template in the target row to obtain pixels in a sampling row; and generating a video preview frame corresponding to the original video frame according to the pixels in the sampling line. According to the method and the device, the quantity of resources required for generating the video preview frame is reduced, and the performance requirement on equipment hardware is reduced, so that low-performance equipment can also perform video preview normally.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a video preview frame generation method, device, equipment, medium and product. BACKGROUND

[0002] Video preview is a core technology in real-time graphics processing, which refers to a process of pixel sampling and rendering of an original video stream by a heterogeneous computing architecture and outputting a preview picture.

[0003] In the existing video preview frame generation method, a complex pixel sampling algorithm is usually required to be run to sample the pixels of the original video frame. However, this undoubtedly consumes a large amount of resources and requires high performance of the device hardware, so that low-performance devices cannot normally preview the video. SUMMARY

[0004] The present application provides a video preview frame generation method, device, equipment, medium and product to solve the problem of large resource consumption and high performance requirement of the existing video preview frame generation method.

[0005] According to an aspect of the present application, a video preview frame generation method is provided, which comprises:

[0006] determining the original pixel row contained in the collected original video frame, and selecting a target pixel row from the original pixel row according to a preset step size;

[0007] determining the original row-in-pixel quantity of the original row-in-pixel contained in the target pixel row, and determining the standard row-in-pixel quantity required for generating a video preview frame; wherein the standard row-in-pixel quantity is less than the original row-in-pixel quantity;

[0008] determining a target row-in-pixel sampling template from a preset candidate row-in-pixel sampling template according to the original row-in-pixel quantity and the standard row-in-pixel quantity, and sampling the original row-in-pixel according to the target row-in-pixel sampling template to obtain a sampled row-in-pixel;

[0009] generating a video preview frame corresponding to the original video frame according to the sampled row-in-pixel.

[0010] According to another aspect of the present application, a video preview frame generation device is provided, which comprises:

[0011] a pixel row selection module for determining the original pixel row contained in the collected original video frame, and selecting a target pixel row from the original pixel row according to a preset step size;

[0012] a pixel number determination module, configured to determine an original row-pixel number of original row-pixels contained in the target pixel row, and determine a standard row-pixel number required for generating the video preview frame; wherein the standard row-pixel number is less than the original row-pixel number;

[0013] a template determination module, configured to determine a target row-pixel sampling template from preset candidate row-pixel sampling templates according to the original row-pixel number and the standard row-pixel number, and sample the original row-pixels according to the target row-pixel sampling template to obtain sampled row-pixels;

[0014] a video preview frame generation module, configured to generate the video preview frame corresponding to the original video frame according to the sampled row-pixels.

[0015] According to another aspect of the present application, an electronic device is provided, which comprises:

[0016] at least one processor; and

[0017] a memory connected with the at least one processor in communication; wherein,

[0018] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the video preview frame generation method according to any one of the present application.

[0019] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to execute the video preview frame generation method according to any one of the present application when executed by the processor.

[0020] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for enabling a processor to execute the video preview frame generation method according to any one of the present application when executed by the processor.

[0021] The present application determines the original pixel row contained in the original video frame collected, and selects a target pixel row from the original pixel row according to a preset step size; determines an original row-pixel number of original row-pixels contained in the target pixel row, and determines a standard row-pixel number required for generating the video preview frame; wherein the standard row-pixel number is less than the original row-pixel number; determines a target row-pixel sampling template from preset candidate row-pixel sampling templates according to the original row-pixel number and the standard row-pixel number, and samples the original row-pixels according to the target row-pixel sampling template to obtain sampled row-pixels; and generates the video preview frame corresponding to the original video frame according to the sampled row-pixels, which has the beneficial effects that:

[0022] By presetting the candidate in-line pixel sampling template, the original in-line pixel number and the standard in-line pixel number are utilized to determine the adaptive target in-line pixel sampling template from the candidate in-line pixel sampling template, and then the pixel sampling is performed by using the target in-line pixel sampling template, so that the complex pixel sampling algorithm is not required to perform the pixel sampling, the resource amount required for generating the video preview frame is reduced, and the performance requirement of the device hardware is reduced, so that the low-performance device can also normally perform the video preview.

[0023] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 A flow chart of a video preview frame generation method provided for the first embodiment of the present application is shown in the figure.

[0026] Figure 2 A flow chart of a video preview frame generation method provided for the second embodiment of the present application is shown in the figure.

[0027] Figure 3 A structural schematic diagram of a video preview frame generation device provided for the third embodiment of the present application is shown in the figure.

[0028] Figure 4 A structural schematic diagram of an electronic device for implementing the video preview frame generation method of the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0029] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0030] It is to be understood that the terminology "original", "target", "candidate", "first", "second", and the like used in the specification and the claims of the application and the above-described drawings is intended to distinguish similar objects not necessarily described in a specific order or in succession. It should be understood that the data thus used can be interchanged under appropriate circumstances so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment one

[0032] Figure 1 A flowchart of a video preview frame generation method provided for embodiment one of the application, the embodiment can be applicable to the case of performing pixel sampling on the collected original video frame to generate the video preview frame corresponding to the original video frame. The method can be executed by a video preview frame generation device, which can be realized in the form of hardware and / or software. As shown in the figure, the method comprises: Figure 1

[0033] S101, determining the original pixel rows contained in the collected original video frame, and selecting a target pixel row from the original pixel rows according to a preset step length.

[0034] The original video frame refers to the video frame data directly obtained from the video acquisition device or video source without any processing or modification. It is the original image representation of each frame in the video sequence, containing complete pixel information. The original pixel row refers to a row of pixel data in the original video frame without any processing or compression. In the original video frame, the pixel is the smallest unit constituting the original video frame, which is usually arranged in a two-dimensional grid. The "pixel row" refers to the horizontal sequence in this grid, that is, a collection of pixels from left to right in the original video frame. For example, assuming that the original resolution of the original video frame is "1920*1080", it means that the original video frame contains 1920 original pixel columns and 1080 original pixel rows.

[0035] The preset step length refers to a preset integer value used to specify the interval of the number of rows skipped when traversing the original pixel rows. The core purpose is to reduce the vertical resolution of the video frame by skipping part of the original pixel rows, thereby reducing the amount of calculation and data to efficiently generate the video preview frame.

[0036] ​The preset step length can be set according to the quality requirement of the user for the video preview. For example, when the user needs a high-quality preview, a shorter preset step length is set to retain more original pixel rows; when the user needs a low-quality preview, a longer preset step length is set to skip more original pixel rows.

[0037] The preset step length can also be set according to the ratio of the original video resolution of the original video frame to the preview target resolution. For example, if the vertical resolution of the original video resolution is 1080 rows and the required vertical resolution of the preview target resolution is 720 rows, the preset step length can be set to 1.5, etc.

[0038] The target pixel row refers to a subset of pixel rows selected from all original pixel rows of the original video frame according to the preset step length, which is essentially a sparse row set generated by vertical downsampling.

[0039] In one embodiment, the acquired original video frame is subjected to pixel row traversal to determine the original pixel rows contained in the original video frame. Further, starting from the first original pixel row and ending at the last original pixel row, pixel row selection is performed from each original pixel row according to the preset step length to obtain the target pixel row.

[0040] For example, assuming that the original video frame contains 1080 original pixel rows and the preset step length is 2, the first original pixel row (starting pixel row) is selected by jumping pixel row selection at intervals of the preset step length "2" to obtain 540 original pixel rows as the target pixel row.

[0041] S102, determine the number of original in-row pixels contained in the target pixel row, and determine the number of standard in-row pixels required for generating the video preview frame.

[0042] The original in-row pixel refers to the complete pixel set in the target pixel row without any processing, i.e., the row pixel selected from the original video frame according to the preset step length. The number of original in-row pixels refers to the total number of original in-row pixels contained in a single target pixel row, i.e., the horizontal resolution of the target pixel row, which is equal to the width value of the original video frame, i.e., the number of pixel columns. For example, if the original resolution of the original video frame is "1920*1080", the width of the original video frame can be determined as "1920", i.e., the number of pixel columns of the original video frame is "1920", i.e., the number of original in-row pixels of the target pixel row is "1920". The number of original in-row pixels serves as a reference value for in-row pixel sampling, which determines how to compress the original width of the original video frame to the standard width of the video preview frame.

[0043] The standard row-pixel number refers to the pixel number that a single row of pixels must reach when generating a video preview frame, i.e., the horizontal resolution (width value) of the video preview frame, which is pre-set by the video preview requirement and is strongly related to the display device, bandwidth limitation or storage specification. The standard row-pixel number is less than the original row-pixel number, i.e., part of the original row-pixels is discarded in this embodiment, so that the remaining original row-pixel number meets the standard row-pixel number, thereby realizing data volume compression by reducing the horizontal resolution.

[0044] In an embodiment, the file header or stream data corresponding to the original video frame is acquired, and the file header or stream data is parsed to determine the resolution parameter of the original video frame, and the original row-pixel number of the original row-pixel is determined according to the resolution parameter. For example, if the resolution parameter is 3840x2160, the original row-pixel number is determined to be 3840.

[0045] In another embodiment, the pre-set standard row-pixel number is read to acquire the standard row-pixel number required for generating the video preview frame. The standard row-pixel number can be determined comprehensively according to the display device limitation, bandwidth constraint, encoding alignment requirement, system performance target and the like.

[0046] S103, determining a target row-pixel sampling template from the pre-set candidate row-pixel sampling templates according to the original row-pixel number and the standard row-pixel number, and sampling the original row-pixel according to the target row-pixel sampling template to obtain a sampled row-pixel.

[0047] The candidate row-pixel sampling template is a pre-defined pixel position selection rule library, which explicitly specifies which pixels in the original row-pixel are retained and which pixels are discarded through a mathematical mapping relationship, forming a compression sampling scheme from high resolution to low resolution. For example, assuming that the original row-pixel is [P0, P1, P2, P3, P4, P5, P6, P7], the candidate row-pixel sampling template A is [P0, P2, P4, P6], i.e., [P0, P2, P4, P6] in the original row-pixel is sampled; the candidate row-pixel sampling template B is [P0, P3, P6], i.e., [P0, P3, P6] in the original row-pixel is sampled.

[0048] The target row-pixel sampling template is dynamically determined from the candidate row-pixel sampling template according to the original row-pixel number and the standard row-pixel number, and is used to guide how to sample the pixels from the original row-pixel of the original pixel row in the video preview frame generation process to achieve the required preview effect. Its role is to reduce the data volume and improve the processing efficiency while ensuring the video preview quality.

[0049] According to the target in-line pixel sampling template, the original in-line pixels are sampled, which is a specific action of applying the target in-line pixel sampling template, that is, according to the rule of the target in-line pixel sampling template, part of the pixels in the original in-line pixels are extracted to form a new pixel sequence.

[0050] In an embodiment, according to the pixel quantity conversion relationship applicable to each candidate in-line pixel sampling template, a corresponding pixel quantity conversion relationship label is generated for each candidate in-line pixel sampling template. For example, assuming that the pixel quantity conversion relationship applicable to any candidate in-line pixel sampling template is: the in-line pixel quantity "1920" is converted into the in-line pixel quantity "720", then the pixel quantity conversion relationship label "the in-line pixel quantity before conversion 1920; the in-line pixel quantity after conversion 720" is generated for the candidate in-line pixel sampling template.

[0051] Further, the first index information is generated according to the original in-line pixel quantity, and the second index information is generated according to the standard in-line pixel quantity, and the first index information and the second index information are matched with the pixel quantity conversion relationship label respectively, if the first index information and the second index information are matched with any pixel quantity conversion relationship label at the same time, then the candidate in-line pixel sampling template to which the pixel quantity conversion relationship label belongs is taken as the target in-line pixel sampling template.

[0052] For example, assuming that the original in-line pixel quantity is "1920" and the standard in-line pixel quantity is "720", then the original in-line pixel quantity "1920" is taken as the first index information, and the standard in-line pixel quantity "720" is taken as the second index information, the first index information and the second index information are matched with the pixel quantity conversion relationship label respectively, assuming that the matched pixel quantity conversion relationship label is "the in-line pixel quantity before conversion 1920; the in-line pixel quantity after conversion 720", then the candidate in-line pixel sampling template to which the pixel quantity conversion relationship label belongs is taken as the target in-line pixel sampling template.

[0053] Further, according to the pixel position selection rule predefined in the target in-line pixel sampling template, the original in-line pixels are sampled to obtain the sampled in-line pixels. For example, assuming that the original in-line pixels are [P0, P1, P2, P3, P4, P5, P6, P7] and the target in-line pixel sampling template is [P0, P2, P4, P6], then P0, P2, P4 and P6 are sampled as the sampled in-line pixels.

[0054] S104, generating a video preview frame corresponding to the original video frame according to the sampled in-line pixels.

[0055] The video preview frame refers to a temporary image with low resolution and low data volume generated by pixel sampling of the original video frame, which can be used as an intermediate visual carrier for real-time monitoring, quick browsing or system preprocessing.

[0056] In an embodiment, the sample row pixels are written into a temporary register, and the sample row pixels in the temporary register are sequentially spliced and written into the ram register according to the number of single splicing, for example, if the number of single splicing is 16, the sample row pixels in the temporary register are spliced and written into the ram register in groups of 16, and the process is repeated until all the sample row pixels in the temporary register are written into the ram register. Further, the pixel data in the ram register is written into the ram, and the pixel data is cached and read for rendering, thereby generating a video preview frame corresponding to the original video frame.

[0057] In the embodiment, the original pixel row contained in the collected original video frame is determined, and a target pixel row is selected from the original pixel row according to a preset step size. The original row pixel quantity of the original row pixels contained in the target pixel row is determined, and the standard row pixel quantity required for generating a video preview frame is determined. The standard row pixel quantity is less than the original row pixel quantity. The target row pixel sampling template is determined from the preset candidate row pixel sampling template according to the original row pixel quantity and the standard row pixel quantity, and the original row pixels are sampled according to the target row pixel sampling template to obtain sample row pixels. The video preview frame corresponding to the original video frame is generated according to the sample row pixels.

[0058] The preset candidate row pixel sampling template is used to determine the adaptive target row pixel sampling template from the candidate row pixel sampling template according to the original row pixel quantity and the standard row pixel quantity, and then the pixel sampling is performed using the target row pixel sampling template, so that the complex pixel sampling algorithm is not needed for pixel sampling, the resource quantity required for generating the video preview frame is reduced, and the performance requirement of the device hardware is reduced, so that the low-performance device can also normally perform video preview.

[0059] Embodiment Two

[0060] Figure 2 A flowchart of a video preview frame generation method provided by the second embodiment of the present application is shown in FIG. 2. The present embodiment further optimizes and extends the above-mentioned embodiments, and can be combined with the above-mentioned optional embodiments. As shown in FIG. 2, the method comprises the following steps. Figure 2

[0061] S201, determine the original pixel row contained in the collected original video frame, and select a target pixel row from the original pixel row according to a preset step size. ​

[0062] S202, determine the number of original in-line pixels contained in the target pixel row, and determine the number of standard in-line pixels required for generating the video preview frame.

[0063] S203, according to the target in-line pixel group granularity and the number of original in-line pixels, determine the number of candidate in-line pixel groups included in the original in-line pixels; according to the ratio between the number of standard in-line pixels and the number of in-line pixel groups, determine the number of sampling pixels corresponding to the sampling from the candidate in-line pixel groups.

[0064] The target in-line pixel group granularity refers to the basic unit size of dividing the original in-line pixels into multiple candidate in-line pixel groups in the in-line pixel sampling process. The essence is to bundle the original in-line pixels in the original pixel row into a processing unit that cannot be divided, called a candidate in-line pixel group. The number of in-line pixel groups refers to the total number of independent pixel groups obtained after dividing the original in-line pixels according to the target in-line pixel group granularity. The number of sampling pixels refers to the number of pixels that need to be extracted from each candidate in-line pixel group in the grouping sampling process.

[0065] In an embodiment, the number of in-line pixel groups included in the original in-line pixels is determined according to the ratio between the target in-line pixel group granularity and the number of original in-line pixels. For example, the number of original in-line pixels is A1, and the target in-line pixel group granularity is A2, then the number of in-line pixel groups is A1 / A2.

[0066] Further, the number of sampling pixels corresponding to the sampling from the candidate in-line pixel groups is determined according to the ratio between the number of standard in-line pixels and the number of in-line pixel groups.

[0067] Optionally, the number of sampling pixels corresponding to the sampling from the candidate in-line pixel groups is determined according to the ratio between the number of standard in-line pixels and the number of in-line pixel groups, including:

[0068] In the case of an integer ratio, the ratio is taken as the number of sampling pixels corresponding to the sampling from the candidate in-line pixel groups.

[0069] Optionally, the number of sampling pixels corresponding to the sampling from the candidate in-line pixel groups is determined according to the ratio between the number of standard in-line pixels and the number of in-line pixel groups, including:

[0070] S2031, in the case of a non-integer ratio, determine the quotient and the remainder according to the ratio.

[0071] The quotient is the result of the integer part of the ratio. The remainder is the part that cannot be divided by the integer after division, representing the "remaining amount" of the original in-line pixels that cannot be completely allocated to each group.

[0072] S2032, determining the first in-line pixel group from the candidate in-line pixel groups according to the remainder, and taking the candidate in-line pixel groups other than the target in-line pixel group as the second in-line pixel group.

[0073] The first in-line pixel group is a group of sub-pixel blocks selected from the candidate in-line pixel groups according to the remainder, which represents the part that needs additional sampling in the algorithm to compensate for the "uneven sampling deviation" caused by the non-integer ratio. For example, if the remainder is k, then k candidate in-line pixel groups are selected as the first in-line pixel group.

[0074] The second in-line pixel group is the remaining candidate in-line pixel group other than the first in-line pixel group, which represents the part that performs standard sampling in the algorithm, and the number of sampled pixels is directly used as the quotient (integer part) without additional compensation.

[0075] Optionally, determining the first in-line pixel group from the candidate in-line pixel groups according to the remainder includes:

[0076] Determining a pixel group selection step size according to the remainder and the number of in-line pixel groups, and selecting a pixel group from the candidate in-line pixel groups according to the pixel group selection step size to determine the first in-line pixel group.

[0077] The pixel group selection step size is an integer interval value, indicating that every how many candidate in-line pixel groups need to be selected as the first in-line pixel group, i.e., the compensation group that needs additional sampling.

[0078] For example, assuming that there are 100 candidate in-line pixel groups and the remainder is 5, then the step size = 20, i.e., every 20 candidate in-line pixel groups select 1 as the first in-line pixel group.

[0079] By determining the pixel group selection step size according to the remainder and the number of in-line pixel groups, and selecting a pixel group from the candidate in-line pixel groups according to the pixel group selection step size to determine the first in-line pixel group, the beneficial effect is that:

[0080] By dynamically calculating the pixel group selection step size, the abstract mathematical remainder is converted into an operable pixel group selection step size, thereby accurately controlling the spatial distribution density of the first in-line pixel group, ensuring that the compensation sampling points are uniformly distributed in the original pixel row, and avoiding image artifacts caused by local oversampling or undersampling.

[0081] S2033, taking the quotient as the number of sampling pixels corresponding to sampling from the second in-line pixel group, and taking the target value as the number of sampling pixels corresponding to sampling from the first in-line pixel group.

[0082] The target value refers to the number of sampling pixels corresponding to sampling from the first in-line pixel group (i.e., the compensation group that needs additional sampling). The target value is the sum of the quotient and one.

[0083] For example, assuming that the original number of in-line pixels is 12, the standard number of in-line pixels is 7, and the target in-line pixel group granularity is 3 pixels / group, it can be determined that the number of in-line pixel groups is 4 groups. The ratio between the standard number of in-line pixels and the number of in-line pixel groups is 7 / 4, that is, the quotient is "1" and the remainder is "3".

[0084] Further, 3 groups are selected as the first in-line pixel group, the pixel group selection step is = 4 / 3 ≈ 1.33, and the integer step is = 1. Starting from index 0, 3 groups are continuously selected according to the pixel group selection step 1, assuming that the first in-line pixel group is [G0, G1, G2], and the remaining second in-line pixel group is [G3], then 2 pixels are sampled from [G0, G1, G2] and 1 pixel is sampled from [G3], obtaining 7 pixel matching standard in-line pixel number.

[0085] By determining the quotient and the remainder according to the ratio when the ratio is a non-integer, determining the first in-line pixel group from the candidate in-line pixel groups according to the remainder, and taking the candidate in-line pixel groups other than the target in-line pixel group as the second in-line pixel group, and taking the quotient as the corresponding sampling pixel number from the second in-line pixel group and the target value as the corresponding sampling pixel number from the first in-line pixel group, wherein the target value is a sum value between the quotient and 1, the beneficial effects are as follows:

[0086] In the first aspect, by dynamically distinguishing the first in-line pixel group and the second in-line pixel group and differentially allocating the sampling pixel number, the remainder of the non-integer ratio is converted into a physical sampling instruction that can be accurately executed, ensuring that the total number of sampled in-line pixels strictly matches the standard number of in-line pixels, and avoiding the cumulative error caused by forced rounding in traditional uniform sampling.

[0087] In the second aspect, the first in-line pixel group is determined based on the remainder, so that the compensation sampling points maintain uniformity in spatial distribution, effectively suppressing the sawtooth and moire artifacts caused by local oversampling in the image scaling or compression process.

[0088] S204, determining a target in-line pixel sampling template from a preset candidate in-line pixel sampling template according to the sampling pixel number and the target in-line pixel group granularity.

[0089] For example, Table 1 shows the candidate in-line pixel sampling templates corresponding to the sampling pixel number from 1 to 16 when the target in-line pixel group granularity is 16 pixels. Each row in Table 1 corresponds to a candidate in-line pixel sampling template, and there are 16 candidate in-line pixel sampling templates. Assuming that the sampling pixel number is "2", the candidate in-line pixel sampling template corresponding to the sampling pixel number "2" is taken as the target in-line pixel sampling template.

[0090]

[0091] By determining the number of intra-row pixel groups included in the candidate intra-row pixel group according to the target intra-row pixel group granularity and the number of original intra-row pixels; determining the number of sampling pixels corresponding to the sampling from the candidate intra-row pixel group according to the ratio between the number of standard intra-row pixels and the number of intra-row pixel groups; and determining the target intra-row pixel sampling template from the preset candidate intra-row pixel sampling template according to the number of sampling pixels and the target intra-row pixel group granularity, the beneficial effects are as follows:

[0092] In the first aspect, by dynamically determining the number of intra-row pixel groups and calculating the ratio between the number of standard intra-row pixels and the number of intra-row pixel groups, the mathematical modeling of pixel sampling is accurately realized, and the number of sampling pixels is strictly matched with the number of standard intra-row pixels and the spatial distribution is continuous.

[0093] In the second aspect, based on the dual constraints of the number of sampling pixels and the target intra-row pixel group granularity, the target intra-row pixel sampling template is determined from the candidate intra-row pixel sampling template, so that the local sampling accuracy is adaptively matched with the image content features.

[0094] S205, sample the candidate intra-row pixel groups according to the target intra-row pixel sampling template to obtain the group-intra-sampling pixels corresponding to the candidate intra-row pixel groups respectively; and obtain the sampling intra-row pixels according to the group-intra-sampling pixels.

[0095] The group-intra-sampling pixels refer to the intermediate pixel set generated by processing a single candidate intra-row pixel group through the target intra-row pixel sampling template. The sampling intra-row pixels are obtained by combining the group-intra-sampling pixels.

[0096] For example, assuming that the target intra-row pixel sampling template is as shown in Table 2, the first, third, sixth, eighth, tenth, twelfth and fifteenth pixels of the candidate intra-row pixel group are sampled to obtain the group-intra-sampling pixels.

[0097]

[0098] By sampling the candidate intra-row pixel groups according to the target intra-row pixel sampling template to obtain the group-intra-sampling pixels corresponding to the candidate intra-row pixel groups respectively; and obtaining the sampling intra-row pixels according to the group-intra-sampling pixels, the beneficial effects are as follows: by performing localized independent sampling on the candidate intra-row pixel groups through the target intra-row pixel sampling template, the global resampling task is decomposed into controllable parallel sub-operations under the premise of ensuring spatial continuity, which significantly reduces the memory access delay and computational complexity.

[0099] S206, write the in-sampling-row pixels into the candidate data cache area, and perform data reading from the candidate data cache area to obtain to-be-rendered pixels; and render the to-be-rendered pixels to generate a video preview frame.

[0100] The candidate data cache area is a temporary buffer in the video memory or the internal memory, and is used to store the in-sampling-row pixels generated by the sampling. The data reading refers to an operation of extracting the pixels from the candidate data cache area on demand. The to-be-rendered pixels refer to a pixel set that is read out from the candidate data cache area and is subjected to geometric transformation, texture mapping, and is waiting for rasterization and shading. The video preview frame refers to a single frame of picture generated by the to-be-rendered pixels after being processed by the full graphics pipeline, and is used for real-time display.

[0101] By writing the in-sampling-row pixels into the candidate data cache area, and performing data reading from the candidate data cache area to obtain to-be-rendered pixels, and rendering the to-be-rendered pixels to generate a video preview frame, the beneficial effect is that, by using the asynchronous double-buffering architecture of the candidate data cache area, seamless pipeline conversion from the in-sampling-row pixels to the to-be-rendered pixels is realized, the speed difference between the sampling rate fluctuation and the rendering frame rate limitation is effectively isolated, and frame loss or tearing caused by resource competition is avoided.

[0102] Optionally, the writing of the in-sampling-row pixels into the candidate data cache area comprises:

[0103] S2061, taking the in-sampling-row pixels at the current time as to-be-written pixels, and determining a first data cache area in the candidate data cache area that is currently to be written.

[0104] The first data cache area is a candidate data cache area that is currently to be written preferentially according to a preset strategy, such as polling or address mapping.

[0105] S2062, determining whether a data reading operation is being performed in the first data cache area, and if it is determined that the data reading operation is being performed in the first data cache area, determining a previous candidate data cache area adjacent to the first data cache area as a historical writing data cache area according to a data cache area writing sequence.

[0106] The determination of whether the data reading operation is being performed in the first data cache area refers to detecting whether the first data cache area is being accessed by the rendering engine, that is, whether the data is in a "read lock" state. The data cache area writing sequence refers to a physical / logical sequence in which the candidate data cache areas are written in a loop. The historical writing data cache area refers to a candidate data cache area located at a position in front of the first data cache area in the writing sequence.

[0107] S2063, writing the to-be-written pixels into the historical writing data cache area.

[0108] For example, assuming that the first data cache area is data cache area 2, it is determined whether a data read operation is being performed in data cache area 2. If it is determined that no data read operation is being performed in data cache area 2, the pixel to be written is written into data cache area 2. If it is determined that a data read operation is being performed in data cache area 2, the last candidate data cache area adjacent to data cache area 2, such as data cache area 1, is determined as a historical write data cache area according to the data cache area write sequence, and then the pixel to be written is written into data cache area 1.

[0109] By taking the sampling row pixel at the current moment as the pixel to be written, and determining the first data cache area to be written from the candidate data cache areas, it is determined whether a data read operation is being performed in the first data cache area. If it is determined that a data read operation is being performed in the first data cache area, the last candidate data cache area adjacent to the first data cache area is determined as a historical write data cache area according to the data cache area write sequence. The pixel to be written is written into the historical write data cache area. The beneficial effect is that simultaneous data read and write in the same data cache area is avoided, which can cause the problem of picture tearing, and the picture integrity of the video preview frame is ensured.

[0110] Optionally, data is read from the candidate data cache areas, including:

[0111] A. The second data cache area to be read at the current moment is determined from the candidate data cache areas, and it is determined whether a data write operation is being performed in the second data cache area.

[0112] The second data cache area is the candidate data cache area that should be read at the current moment according to a preset reading strategy, such as polling or frame sequence mapping, and is used as the data source of the rendering pipeline. Whether a data write operation is being performed indicates whether the second data cache area is being written with data.

[0113] B. If it is determined that a data write operation is being performed in the second data cache area, the last candidate data cache area adjacent to the second data cache area is determined as a historical read data cache area according to the data cache area read sequence.

[0114] The historical read data cache area refers to the candidate data cache area located at a position in front of the second data cache area in the cache read sequence, that is, the candidate data cache area of the last round of reading operation.

[0115] C. Data is read from the historical read data cache area.

[0116] For example, assuming that the second data cache area is data cache area 2, it is determined whether a data write operation is being performed in data cache area 2, if it is determined that no data write operation is being performed in data cache area 2, data is read from data cache area 2, if it is determined that a data write operation is being performed in data cache area 2, a last candidate data cache area adjacent to data cache area 2 is determined as a historical read data cache area according to a data cache area read sequence, and then data is read from data cache area 1.

[0117] By determining the second data cache area to be read from the candidate data cache areas, and determining whether a data write operation is being performed in the second data cache area, if it is determined that a data write operation is being performed in the second data cache area, a last candidate data cache area adjacent to the second data cache area is determined as a historical read data cache area according to a data cache area read sequence, and data is read from the historical read data cache area, the beneficial effects are that simultaneous data read and write in the same data cache area can be avoided, which can cause the problem of picture tearing, and the picture integrity of the video preview frame is ensured.

[0118] Embodiment three

[0119] Figure 3 A structure diagram of a video preview frame generation device provided by embodiment three of the present application can be applied to the case of performing pixel sampling on the collected original video frame to generate a video preview frame corresponding to the original video frame, as shown in FIG. 3, the device comprises: Figure 3

[0120] A pixel row selection module 31 is configured to determine an original pixel row contained in the collected original video frame, and select a target pixel row from the original pixel row according to a preset step length;

[0121] A pixel number determination module 32 is configured to determine an original row-in-pixel number of original row-in-pixels contained in the target pixel row, and determine a standard row-in-pixel number required for generating a video preview frame; wherein the standard row-in-pixel number is less than the original row-in-pixel number;

[0122] A template determination module 33 is configured to determine a target row-in-pixel sampling template from preset candidate row-in-pixel sampling templates according to the original row-in-pixel number and the standard row-in-pixel number, and sample the original row-in-pixels according to the target row-in-pixel sampling template to obtain sampled row-in-pixels;

[0123] A video preview frame generation module 34 is configured to generate a video preview frame corresponding to the original video frame according to the sampled row-in-pixels.

[0124] ​Optionally, the template determining module 33 is specifically configured to:

[0125] determine, according to the target in-line pixel group granularity and the original in-line pixel quantity, an in-line pixel group quantity of candidate in-line pixel groups included by the original in-line pixels;

[0126] determine, according to a ratio between the standard in-line pixel quantity and the in-line pixel group quantity, a sampling pixel quantity corresponding to sampling from the candidate in-line pixel groups;

[0127] determine, according to the sampling pixel quantity and the target in-line pixel group granularity, a target in-line pixel sampling template from preset candidate in-line pixel sampling templates.

[0128] Optionally, the template determining module 33 is specifically further configured to:

[0129] sample the candidate in-line pixel groups respectively according to the target in-line pixel sampling template, to obtain group-in-sample pixels respectively corresponding to the candidate in-line pixel groups;

[0130] obtain the sample in-line pixels according to the group-in-sample pixels.

[0131] Optionally, the template determining module 33 is specifically further configured to:

[0132] in a case where the ratio is a non-integer, determine a quotient and a remainder according to the ratio;

[0133] determine a first in-line pixel group from the candidate in-line pixel groups according to the remainder, and determine, as a second in-line pixel group, the candidate in-line pixel groups except the target in-line pixel group;

[0134] determine, as a sampling pixel quantity corresponding to sampling from the second in-line pixel group, the quotient, and determine, as a sampling pixel quantity corresponding to sampling from the first in-line pixel group, a target value, wherein the target value is a summation value between the quotient and 1.

[0135] Optionally, the template determining module 33 is specifically further configured to:

[0136] determine, according to the remainder and the in-line pixel group quantity, a pixel group selection step;

[0137] determine the first in-line pixel group according to pixel group selection from the candidate in-line pixel groups according to the pixel group selection step.

[0138] Optionally, the video preview frame generating module 34 is specifically configured to:

[0139] write the in-sampling-line pixel into a candidate data cache area, and perform data reading from the candidate data cache area to obtain a pixel to be rendered;

[0140] render the pixel to be rendered to generate the video preview frame.

[0141] Optionally, the video preview frame generation module 34 is specifically further configured to:

[0142] write the in-sampling-line pixel at the current time as a pixel to be written, and determine a first data cache area to be written currently from the candidate data cache areas;

[0143] determine whether a data reading operation is being performed in the first data cache area, and if it is determined that the data reading operation is being performed in the first data cache area, determine a last candidate data cache area adjacent to the first data cache area as a historical writing data cache area according to a data cache area writing sequence;

[0144] write the pixel to be written into the historical writing data cache area.

[0145] Optionally, the video preview frame generation module 34 is specifically further configured to:

[0146] determine a second data cache area to be read currently from the candidate data cache areas, and determine whether a data writing operation is being performed in the second data cache area;

[0147] if it is determined that the data writing operation is being performed in the second data cache area, determine a last candidate data cache area adjacent to the second data cache area as a historical reading data cache area according to a data cache area reading sequence;

[0148] perform data reading from the historical reading data cache area.

[0149] The video preview frame generation apparatus provided in the embodiments of the present application can perform the video preview frame generation method provided in any of the embodiments of the present application, and has the function modules and beneficial effects corresponding to the execution method.

[0150] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0151] Embodiment Four

[0152] Figure 4A structural diagram of an electronic device 40 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0153] As shown in Figure 4 The electronic device 40 includes at least one processor 41, and a memory, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., connected to the at least one processor 41 in communication, where the memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 42 or loaded into the random access memory (RAM) 43 from the storage unit 48. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0154] Various components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc., an output unit 47, such as various types of displays, speakers, etc., a storage unit 48, such as a magnetic disk, an optical disk, etc., and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0155] The processor 41 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 performs various methods and processes described above, such as the method of generating a video preview frame.

[0156] In some embodiments, the method of generating a video preview frame can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 48. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 40 via, e.g., ROM 42 and / or communication unit 49. When the computer program is loaded onto RAM 43 and executed by processor 41, one or more steps of the method of generating a video preview frame as described above can be performed. Alternatively, in other embodiments, processor 41 can be configured to perform the method of generating a video preview frame by other means, e.g., with the aid of firmware.

[0157] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0158] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0159] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0160] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0161] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0162] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business expansion in traditional physical host and virtual private service.

[0163] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and this is not limited herein.

[0164] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for generating video preview frames, characterized in that, The method includes: Determine the original pixel rows contained in the acquired original video frames, and select the target pixel rows from the original pixel rows according to a preset step size; Determine the original number of inline pixels in the target pixel row, and determine the standard number of inline pixels required to generate a video preview frame; wherein the standard number of inline pixels is less than the original number of inline pixels. Based on the original number of inline pixels and the standard number of inline pixels, a target inline pixel sampling template is determined from the preset candidate inline pixel sampling templates, and the original inline pixels are sampled according to the target inline pixel sampling template to obtain sampled inline pixels; A video preview frame corresponding to the original video frame is generated based on the pixels in the sampled row.

2. The method according to claim 1, characterized in that, The step of determining the target inline pixel sampling template from the preset candidate inline pixel sampling templates based on the original inline pixel count and the standard inline pixel count includes: Based on the target inline pixel group granularity and the original inline pixel count, determine the number of inline pixel groups that include the candidate inline pixel groups in the original inline pixels; The number of sampling pixels corresponding to sampling from the candidate in-row pixel group is determined based on the ratio between the number of standard in-row pixels and the number of in-row pixel groups. Based on the number of sampled pixels and the granularity of the target intra-row pixel group, the target intra-row pixel sampling template is determined from the preset candidate intra-row pixel sampling templates.

3. The method according to claim 2, characterized in that, The step of sampling the original in-row pixels according to the target in-row pixel sampling template to obtain sampled in-row pixels includes: The candidate in-row pixel groups are sampled according to the target in-row pixel sampling template to obtain the in-group sampled pixels corresponding to the candidate in-row pixel groups respectively. The pixels in the sampling row are obtained based on the sampling pixels in each group.

4. The method according to claim 2, characterized in that, Determining the number of sampling pixels corresponding to sampling from the candidate in-row pixel group based on the ratio between the number of standard in-row pixels and the number of in-row pixel groups includes: When the ratio is not an integer, the quotient and remainder are determined based on the ratio. The first row pixel group is determined from the candidate row pixel groups based on the remainder, and the candidate row pixel groups other than the target row pixel group are taken as the second row pixel group. The quotient is used as the number of sampled pixels corresponding to sampling from the second row of pixel groups, and the target value is used as the number of sampled pixels corresponding to sampling from the first row of pixel groups; wherein, the target value is the sum of the quotient and one.

5. The method according to claim 4, characterized in that, Determining the first intra-row pixel group from the candidate intra-row pixel groups based on the remainder includes: The pixel group selection step size is determined based on the remainder and the number of pixel groups in the row; The first row pixel group is determined by selecting a pixel group from the candidate row pixel group according to the pixel group selection step size.

6. The method according to claim 1, characterized in that, The step of generating the video preview frame corresponding to the original video frame based on the pixels in the sampled row includes: The pixels within the sampled row are written into the candidate data buffer area, and data is read from the candidate data buffer area to obtain the pixels to be rendered; The pixels to be rendered are rendered to generate the video preview frame.

7. The method according to claim 6, characterized in that, The step of writing the pixels within the sampled row into the candidate data buffer area includes: The pixels in the sampling row at the current moment are taken as the pixels to be written, and the first data cache area to be written is determined from the candidate data cache area; Determine whether a data read operation is being performed in the first data cache area. If it is determined that a data read operation is being performed in the first data cache area, then determine the previous candidate data cache area adjacent to the first data cache area according to the writing order of the data cache areas, and use it as the historical written data cache area. The pixel to be written is written to the historical write data cache area.

8. The method according to claim 6, characterized in that, The step of reading data from the candidate data cache area includes: From the candidate data cache area, determine the second data cache area to be read, and determine whether a data write operation is being performed in the second data cache area; If it is determined that a data write operation is being performed in the second data cache area, then the previous candidate data cache area adjacent to the second data cache area is determined according to the data cache area reading order, and is used as the historical read data cache area; Data is read from the historical read data cache area.

9. A device for generating video preview frames, characterized in that, The device includes: The pixel row selection module is used to determine the original pixel rows contained in the acquired original video frame, and select the target pixel row from the original pixel rows according to a preset step size. A pixel count determination module is used to determine the original inline pixel count of the original inline pixels contained in the target pixel row, and to determine the standard inline pixel count required to generate a video preview frame; wherein the standard inline pixel count is less than the original inline pixel count. The template determination module is used to determine a target inline pixel sampling template from a preset candidate inline pixel sampling template based on the original inline pixel count and the standard inline pixel count, and to sample the original inline pixels according to the target inline pixel sampling template to obtain sampled inline pixels; The video preview frame generation module is used to generate a video preview frame corresponding to the original video frame based on the pixels in the sampled row.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method for generating video preview frames according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the method for generating a video preview frame according to any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method for generating video preview frames according to any one of claims 1-8.

Images