A method, apparatus, device, medium, and product for processing synthetic video.

Abstract

This invention discloses a method, apparatus, device, medium, and product for processing composite videos, relating to the field of server technology. The method determines a cache address based on the row and column numbers corresponding to the data to be supplemented. It determines the weight of the block pixel size of the read-back data relative to the data to be supplemented based on the size positional relationship between the block pixel size of the read-back data and the block pixel size of the data to be supplemented. Then, based on the relationship between the block pixel size and the read-back data, supplementary data can be predicted and placed within the cache address. This ensures that the values ​​of the supplemented pixels are as close as possible to the original pixels, and that the pixel data is balanced with the pixel data corresponding to the actually missing key information. This avoids the problem of poor display quality caused by black spots in the display image due to traditional zero-padding mechanisms. By predicting the pixel positions of data that cannot be obtained from DDR, the quality of the composite video display is improved, and the risk of missing key information is reduced.

Description

Technical Field

[0001] This invention relates to the field of server technology, and in particular to a method, apparatus, device, medium, and product for processing composite video. Background Technology

[0002] Traditional server management and control chips obtain synthesized video information from Double Data Rate Synchronous Dynamic Random-Access Memory (DDR) and generate control signals at the Digital Visual Interface (DVI) so that it can be displayed on the local monitor via the server motherboard.

[0003] Because the DDR bus is not constantly occupied by video functions, but the timing of the DVI interface is continuous, if the DDR bus is interrupted by other functional buses while being occupied by video functions, the pixel positions corresponding to data that cannot be retrieved from DDR will be padded with zeros before being sent to the local display. The zero-padded display will contain many black dots, resulting in poor display quality. Furthermore, since the unretrieved data may contain critical information, padding the pixels with critical information with zeros could lead to the omission of crucial information, thus posing a risk.

[0004] Therefore, improving display quality and reducing the risk of missing key information are urgent problems that need to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, device, medium, and product for processing composite videos, in order to solve the problems of poor display quality caused by padding pixels with zeros when data cannot be obtained from DDR, and the risks caused by missing key information.

[0006] To address the aforementioned technical problems, this invention provides a method for processing synthesized video, comprising:

[0007] Acquire synthetic video data stored in a double data rate synchronous dynamic random access memory;

[0008] The row and column numbers corresponding to the data to be supplemented are determined based on the synthesized video data; and the cache address corresponding to the data to be supplemented is determined based on the row and column numbers corresponding to the data to be supplemented.

[0009] The block pixel size weight of the read-back data of the synthesized video data is determined based on the size positional relationship between the block pixel size occupied by the read-back data and the block pixel size occupied by the data to be supplemented, so as to obtain the supplementary data through prediction processing;

[0010] The supplementary data is added to the cache address so that it can be transmitted to the local end for display.

[0011] On the one hand, the block pixel size occupied by the read-back data of the synthesized video data and the block pixel size occupied by the cache address corresponding to the data to be supplemented are both based on the block size set by dividing the synthesized video data into multiple regions or pixel blocks in the random access memory array.

[0012] On the other hand, acquiring the synthetic video data stored in the Double Data Rate Synchronous Dynamic Random Access Memory includes:

[0013] Obtain the access priority of each bus of the double data rate synchronous dynamic random access memory;

[0014] Within each bus access priority, the access priority of the video overlay and synthesis controller is adjusted to the top N priorities of each bus transmission priority; where N is greater than or equal to 1.

[0015] The synthesized video data stored in the double data rate synchronous dynamic random access memory is obtained according to the access order of the first N priorities.

[0016] On the other hand, the synthetic video data stored in the double data rate synchronous dynamic random access memory is obtained according to the access order of the first N priorities, including:

[0017] Obtain the data enable signal corresponding to the digital visual interface of the video overlay synthesis controller;

[0018] When the data enable signal is in a high state, the synthesized video data stored in the double data rate synchronous dynamic random access memory is obtained according to the access order of the first N priorities.

[0019] On the other hand, when the data enable signal is in a low state, the method further includes:

[0020] Output an interrupt command to the central processing unit so that the central processing unit can adjust the access priority of the video overlay and compositing controller;

[0021] Adjusting the access priority of the video overlay and compositing controller includes:

[0022] Adjust the access priority of the video overlay and synthesis controller to the last priority of each bus transmission priority.

[0023] On the other hand, determining the row and column numbers corresponding to the data to be supplemented based on the synthesized video data includes:

[0024] Obtain the current pixel of the current frame data of the synthesized video data;

[0025] If the current pixel does not receive the corresponding pixel of the synthesized video data at the pixel position of the first-in-first-out buffer of the video overlay synthesis controller within a preset time, then the current pixel is determined to be data to be supplemented.

[0026] The row and column numbers of the data to be supplemented are determined based on the pixel position of the current pixel in the synthesized video data.

[0027] On the other hand, determining the cache address corresponding to the data to be supplemented based on the row number and column number of the data to be supplemented includes:

[0028] The resolution information of the synthesized video data and the initial address of the double data rate synchronous dynamic random access memory are obtained; wherein, the resolution information includes resolution width and resolution height;

[0029] Subtract 1 from the row number corresponding to the data to be supplemented to obtain the first row number;

[0030] The row resolution information of the read-back data is obtained by multiplying the first row number by the resolution width.

[0031] The row resolution information of the read-back data is added to the column number corresponding to the data to be supplemented to obtain the current position information corresponding to the data to be supplemented;

[0032] The current position information corresponding to the data to be supplemented is multiplied by the bit data corresponding to the unit pixel to obtain the bit data corresponding to the data to be supplemented.

[0033] The bit data corresponding to the data to be supplemented is divided by the bit data in the address space of the synchronous dynamic random access memory at double data rate to obtain the bit data in the address space occupied by the data to be supplemented.

[0034] The cache address corresponding to the data to be supplemented is obtained by adding the bit data of the address space occupied by the data to be supplemented to the initial address.

[0035] On the other hand, before determining the block pixel size weight of the read-back data of the synthesized video data based on the size positional relationship between the block pixel size occupied by the read-back data of the synthesized video data and the block pixel size occupied by the data to be supplemented, and before predicting and processing to obtain the supplementary data, the method further includes:

[0036] Obtain the remaining data volume in the first-in-first-out buffer of the video overlay and synthesis controller;

[0037] The number of pixels read back from the Double Data Rate Synchronous Dynamic Random Access Memory is determined based on the bit width data of the Advanced Extensible Interface and the bit data corresponding to each pixel.

[0038] The readback mechanism of the Double Data Rate Synchronous Dynamic Random Access Memory (DMR) is determined based on the relationship between the rate at which the remaining data is read to the local terminal and the rate at which the number of pixels is read back. When the readback mechanism of the DMR is in normal readback mode, the system waits to read the synthesized video data from the DMR.

[0039] When the readback mechanism of the Double Data Rate Synchronous Dynamic Random Access Memory is abnormal, the process proceeds to the step of determining the block pixel size weight of the readback data of the synthetic video data based on the size positional relationship between the block pixel size occupied by the readback data of the synthetic video data and the block pixel size occupied by the data to be supplemented, in order to predict and process the supplementary data.

[0040] On the other hand, the readback mechanism of the double data rate synchronous dynamic random access memory, which determines the readback mechanism based on the relationship between the rate at which the remaining data is read to the local end and the rate at which the number of pixels is read back, includes:

[0041] Obtain the number of reads from the double data rate synchronous dynamic random access memory within a first preset time period;

[0042] Obtain the frequency period of the pixel clock of the digital vision interface corresponding to the current resolution information;

[0043] The first time parameter for the synthesized video data to be displayed after the first-in-first-out buffer has been read is determined based on the remaining data volume and the frequency period;

[0044] A second time parameter is determined based on the first preset time and the number of readbacks to determine the number of pixels read back from the double data rate synchronous dynamic random access memory corresponding to the number of composite video data to be displayed;

[0045] The corresponding readback mechanism is determined based on the first time parameter and the second time parameter.

[0046] On the other hand, the corresponding readback mechanism is determined based on the first time parameter and the second time parameter, including:

[0047] If the first time parameter is greater than the second time parameter, then the readback mechanism of the double data rate synchronous dynamic random access memory is determined to be a normal readback.

[0048] If the first time parameter is less than or equal to the second time parameter, then the readback mechanism of the double data rate synchronous dynamic random access memory is determined to be an abnormal readback.

[0049] On the other hand, the step of determining the block pixel size weight of the read-back data of the synthetic video data based on the size positional relationship between the block pixel size occupied by the read-back data of the synthetic video data and the block pixel size occupied by the data to be supplemented, in order to obtain supplementary data through prediction processing, includes:

[0050] The synthesized video data is divided into multiple pixel sizes of preset dimensions; wherein, the pixel sizes of preset dimensions include preset rows;

[0051] The read-back data and the data to be supplemented are respectively arranged at the pixel size of the first target preset size and the pixel size of the second target preset size;

[0052] Based on the relationship between the position of the read-back data in the preset row of the first target preset size pixel size and the position of the data to be supplemented in the preset row of the second target preset size pixel size, the weight parameters of the read-back data in the corresponding preset row are set respectively; wherein, the sum of each weight parameter is 1;

[0053] The supplementary data is obtained by weighting and summing the read-back data with the weight parameters of the corresponding preset rows.

[0054] On the other hand, when the data to be supplemented is in the first row of a preset row in the pixel size of the second target preset size, and the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, including:

[0055] Set the weight parameter of the read-back data of the first target preset size to the row that is horizontally adjacent to the preset row of the pixel size of the second target preset size of the data to be supplemented;

[0056] The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the second weight parameter;

[0057] The weight parameter of the read-back data in the last row of the first target preset size, which is adjacent to the pixel size of the data to be supplemented in the pixel size tilt direction of the second target preset size, is set as the third weight parameter; wherein, the third weight parameter includes a first weight sub-parameter and a second weight sub-parameter; within the read-back data to which the third weight parameter belongs, according to the read-back order, the weight parameter of the read-back data in the last row of the first target preset size that is close to the read-back time of the data to be supplemented is set as the first weight sub-parameter; the weight parameter of the read-back data in the last row of the first target preset size that is not close to the read-back time of the data to be supplemented is set as the second weight sub-parameter; wherein, the first weight sub-parameter is greater than the second weight sub-parameter;

[0058] For each preset row of pixel size of the first target preset size, the fourth weight parameter corresponding to the read data is set to gradually decrease from bottom to top according to the row order; wherein, the fourth weight parameter includes a third weight sub-parameter, a fourth weight sub-parameter, and a fifth weight sub-parameter; the third weight sub-parameter is greater than the fourth weight sub-parameter; the fourth weight sub-parameter is greater than the fifth weight sub-parameter; the first weight parameter is greater than the second weight parameter; the second weight parameter is greater than the third weight parameter; the third weight parameter is greater than the fourth weight parameter.

[0059] On the other hand, when the data to be supplemented is in the second row of a preset row in the pixel size of the second target preset size, and the data to be supplemented is in the fifth pixel size, the step of setting the weight parameters of the read-back data in the corresponding preset row according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size includes:

[0060] Set the weight parameter of the read-back data of the first row of the corresponding pixel size of the second target preset size to the fifth weight parameter;

[0061] Set the weight parameter of the last row of the first target preset size, which is horizontally adjacent to the row of the preset row of the second target preset size in the pixel size of the data to be supplemented, as the sixth weight parameter;

[0062] The weight parameter of the read-back data in the second-to-last row of the first target preset size, which is adjacent to the preset row of the pixel size of the data to be supplemented in the second target preset size, is set as the seventh weight parameter; the weight parameter of the pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and is in the same column as the preset row of the pixel size of the data to be supplemented in the second target preset size, and is adjacent to the read-back data in the first target preset size, is set as the eighth weight parameter;

[0063] The weight parameter of the read-back data in the last row of the first target preset size, which is adjacent to the pixel size of the data to be supplemented in the tilt direction of the second target preset size, is set as the ninth weight parameter; wherein, the ninth weight parameter includes a sixth weight sub-parameter and a seventh weight sub-parameter; within the read-back data to which the ninth weight parameter belongs, according to the read-back order, the weight parameter of the read-back data in the last row of the first target preset size that is close to the read-back time of the data to be supplemented is set as the sixth weight sub-parameter; the weight parameter of the read-back data in the last row of the first target preset size that is not close to the read-back time of the data to be supplemented is set as the seventh weight sub-parameter; wherein, the sixth weight sub-parameter is greater than the seventh weight sub-parameter;

[0064] For each preset row of pixel size of the first target preset size, set the tenth weight parameter corresponding to the read data from bottom to top according to the row order, and gradually decrease it; wherein, the tenth weight parameter includes the eighth weight sub-parameter and the ninth weight sub-parameter; the eighth weight sub-parameter is greater than the ninth weight sub-parameter; the fifth weight parameter is greater than the sixth weight parameter; the sixth weight parameter is greater than the seventh weight parameter; the seventh weight parameter is greater than the eighth weight parameter; the eighth weight parameter is greater than the ninth weight parameter; the ninth weight parameter is greater than the tenth weight parameter.

[0065] On the other hand, when the data to be supplemented is in the third row of the preset row of the pixel size of the second target preset size, and the data to be supplemented is in the fifth pixel size, the step of setting the weight parameters of the read-back data in the corresponding preset row according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size includes:

[0066] Set the weight parameters of the read data in the second row and first row of the corresponding pixel size of the second target preset size to the eleventh weight parameter and the thirteenth weight parameter respectively;

[0067] Set the weight parameter of the read-back data of the same row of the first target preset size that is horizontally adjacent to the preset row of the second target preset size pixel size of the data to be supplemented as the twelfth weight parameter;

[0068] The weight parameters of the read data of the first and second rows of the first target preset size, which are horizontally adjacent to the preset row of the pixel size of the second target preset size, are set as the fifteenth weight parameter and the fourteenth weight parameter, respectively.

[0069] The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the sixteenth weight parameter.

[0070] The weight parameter of the read-back data in the last row of the first target preset size, which is adjacent to the pixel size of the data to be supplemented in the pixel size tilt direction of the second target preset size, is set as the seventeenth weight parameter; wherein, the seventeenth weight parameter includes a tenth weight sub-parameter and an eleventh weight sub-parameter; within the read-back data to which the seventeenth weight parameter belongs, according to the read-back order, the weight parameter of the read-back data in the last row of the first target preset size that is close to the read-back time of the data to be supplemented is set as the tenth weight sub-parameter; the weight parameter of the read-back data in the last row of the first target preset size that is not close to the read-back time of the data to be supplemented is set as the eleventh weight sub-parameter; wherein, the tenth weight sub-parameter is greater than the eleventh weight sub-parameter;

[0071] The read-back data of each preset row of pixel size of each first target preset size is set as the eighteenth weight parameter; wherein, the eleventh weight parameter is greater than the twelfth weight parameter; the twelfth weight parameter is greater than the thirteenth weight parameter; the thirteenth weight parameter is greater than the fourteenth weight parameter; the fourteenth weight parameter is greater than the fifteenth weight parameter; the fifteenth weight parameter is greater than the sixteenth weight parameter; the sixteenth weight parameter is greater than the seventeenth weight parameter; and the seventeenth weight parameter is greater than the nineteenth weight parameter.

[0072] On the other hand, when the data to be supplemented is in the last row of the preset row of the pixel size of the second target preset size, and the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, including:

[0073] Set the weight parameters of the read data of the third, second, and first rows of the corresponding pixel size of the second target preset size to the twentieth, twenty-second, and twenty-fourth weight parameters respectively;

[0074] The weight parameter of the read-back data in the same row of the first target preset size that is horizontally adjacent to the preset row of the second target preset size pixel size is set as the twenty-first weight parameter; the weight parameters of the read-back data in the first row, second row, and third row of the first target preset size that are horizontally adjacent to the preset row of the second target preset size pixel size pixel size are set as the twenty-third weight parameter, the twenty-fifth weight parameter, and the twenty-sixth weight parameter, respectively.

[0075] The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the twenty-seventh weight parameter.

[0076] The weight parameter of the pixel size adjacent to the data to be supplemented in the pixel size tilt direction of the second target preset size, and the weight parameter of the read-back data in the last row of the first target preset size, is set as the twenty-eighth weight parameter; wherein, the twenty-eighth weight parameter includes the twelfth weight sub-parameter and the thirteenth weight sub-parameter; the twelfth weight sub-parameter is greater than the thirteenth weight sub-parameter; the weight parameters from largest to smallest are: the twentieth weight parameter, the twenty-first weight parameter, the twenty-second weight parameter, the twenty-third weight parameter, the twenty-fourth weight parameter, the twenty-fifth weight parameter, the twenty-sixth weight parameter, the twenty-seventh weight parameter, and the twenty-eighth weight parameter.

[0077] On the other hand, the triggering process of the readback mechanism of the double data rate synchronous dynamic random access memory includes:

[0078] Obtain the preset remaining amount of the first-in-first-out buffer;

[0079] If the remaining data amount is greater than or equal to the preset remaining amount, the readback mechanism will not be triggered.

[0080] If the remaining data amount is less than the preset remaining amount, the readback mechanism is triggered.

[0081] On the other hand, after adding the supplementary data to the cache address, the method further includes:

[0082] The supplementary data is merged with the read-back data to obtain the supplemented synthetic video data;

[0083] Determine whether the supplemented synthetic video data is the same as the synthetic video data stored in the double data rate synchronous dynamic random access memory;

[0084] If they are the same, then send them to the local end.

[0085] On the other hand, the supplemented composite video data is sent to the local end, including:

[0086] The supplemented synthetic video data is processed in a time-series format using a digital vision interface to obtain digital vision data.

[0087] The digital visual data is sent to the local terminal.

[0088] To address the aforementioned technical problems, the present invention also provides a video processing apparatus, comprising:

[0089] The acquisition module is used to acquire the synthetic video data stored in the Double Data Rate Synchronous Dynamic Random Access Memory;

[0090] The determining module is used to determine the row number and column number corresponding to the data to be supplemented based on the synthesized video data; and to determine the cache address corresponding to the data to be supplemented based on the row number and column number corresponding to the data to be supplemented.

[0091] The prediction processing module is used to determine the block pixel size weight of the read-back data of the synthetic video data based on the size position relationship between the block pixel size occupied by the read-back data of the synthetic video data and the block pixel size occupied by the data to be supplemented, so as to obtain the supplementary data through prediction processing;

[0092] The supplementary module is used to add the supplementary data to the cache address so that it can be transmitted to the local end for display.

[0093] To address the aforementioned technical problems, the present invention also provides a video processing apparatus, comprising:

[0094] Memory, used to store computer programs;

[0095] A processor for executing the computer program to implement the steps of the synthetic video processing method as described above.

[0096] To address the aforementioned technical problems, the present invention also provides a non-volatile storage medium storing a computer program, which, when executed by a processor, implements the steps of the video synthesis processing method described above.

[0097] To address the aforementioned technical problems, the present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the synthetic video processing method.

[0098] This invention provides a method for processing composite video, comprising: acquiring composite video data stored in a double data rate synchronous dynamic random access memory; determining the row and column numbers corresponding to the data to be supplemented based on the composite video data; determining the cache address corresponding to the data to be supplemented based on the row and column numbers; determining the block pixel size weight of the read-back data of the composite video data based on the size positional relationship between the block pixel size occupied by the read-back data of the composite video data and the block pixel size occupied by the data to be supplemented, so as to predict and process to obtain supplementary data; and supplementing the supplementary data into the cache address for transmission to the local end for display.

[0099] The beneficial effect of this invention lies in determining the cache address of the data to be supplemented based on the row and column numbers corresponding to the data to be supplemented. Compared with the traditional zero-padding mechanism, this invention records the cache address of the data to be supplemented, determines the weight of the block pixel size of the read-back data relative to the data to be supplemented based on the size position relationship between the block pixel size of the read-back data and the block pixel size of the data to be supplemented, and then predicts the supplemented data based on the relationship between the block pixel size and the read-back data. The supplemented value is placed within the cache address, so that the value of the supplemented pixel is as close as possible to the original pixel and the pixel data is balanced with the pixel data corresponding to the real missing key information. This avoids the problem of poor display quality caused by black spots in the display screen due to the traditional zero-padding mechanism. By predicting the pixel position data of data that cannot be obtained from DDR, the quality of the displayed composite video is improved and the risk of missing key information is reduced.

[0100] Secondly, the acquisition process of composite video data is achieved through bus access priority adjustment to ensure that the corresponding composite video data is read as much as possible during the enable signal high period, minimizing the need for subsequent data supplementation. The priority access order is adjusted so that as much data as possible is read when the data enable signal is high, and its priority access order is adjusted to last when the data enable signal is low, avoiding impact on other chip functions. The process of determining the row and column numbers corresponding to the data to be supplemented is based on the location of data that the video overlay and synthesis controller's FIFO buffer cannot obtain in time. This facilitates the subsequent determination of buffered data based on the row and column numbers of the data to be supplemented, improving the efficiency of reading effective data, avoiding the reading of invalid data, and improving the utilization of the DDR bus. Before predictive processing, by estimating the current remaining data volume in the prediction module and the frequency of high-level access within a preset time, it is estimated whether the data to be displayed can be read back normally from DDR after all remaining data is read, saving resources corresponding to actual predictive processing and ensuring that predictive processing is performed only when data is truly unavailable.

[0101] In addition, the present invention also provides a processing apparatus, device, medium and product for synthesized video, which has the same beneficial effects as the above-described processing method for synthesized video. Attached Figure Description

[0102] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0103] Figure 1 A schematic diagram of the video function of a traditional server management and control chip;

[0104] Figure 2 This is a schematic diagram of a traditional local video synthesis controller.

[0105] Figure 3 A flowchart illustrating a method for processing synthesized video according to an embodiment of the present invention;

[0106] Figure 4 A schematic diagram illustrating video overlay synthesis for server management and control chip software in traditional solutions;

[0107] Figure 5 This is an architecture diagram of a video overlay and synthesis controller provided in an embodiment of the present invention;

[0108] Figure 6 This is a schematic diagram of a data prediction scenario provided in an embodiment of the present invention;

[0109] Figure 7 This is a schematic diagram of a data prediction scenario two provided by an embodiment of the present invention;

[0110] Figure 8 This is a schematic diagram of a data prediction scenario three provided by an embodiment of the present invention;

[0111] Figure 9 This is a schematic diagram of a data prediction scenario four provided by an embodiment of the present invention;

[0112] Figure 10 A standard DVI timing diagram provided for an embodiment of the present invention;

[0113] Figure 11 A structural diagram of a video processing apparatus provided in an embodiment of the present invention;

[0114] Figure 12 This is a structural diagram of a video synthesis processing device provided in an embodiment of the present invention. Detailed Implementation

[0115] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0116] The core of this invention is to provide a method, apparatus, device, medium, and product for processing composite videos, in order to solve the problems of poor display quality caused by padding pixels with zeros when data cannot be obtained from DDR, and the risks caused by missing key information.

[0117] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0118] The traditional video processing workflow for server management and control chips is as follows:

[0119] Video processing is divided into local and remote processing.

[0120] Figure 1 A schematic diagram of the video function of a traditional server management and control chip, such as... Figure 1As shown, the raw video data (operating system interface) on the host side is transmitted to the board management and control chip via the Peripheral Component Interconnect Express (PCIe) high-speed serial bus. Figure 1 The server management control chip contains a video graphics array (VGA) module. The VGA generates raw video data in optical primary color (Red-Green-Blue, RGB) format. The RGB data is split into two paths: one path goes to an external DAC module for digital-to-analog conversion and then to the local display. The other path undergoes resolution adjustment and then passes through a video compression module (common formats include H.264 / Joint Photographic Experts Group (JPEG)) to obtain compressed video data. This compressed video data is then written to an external double data rate synchronous dynamic random access memory (DDR) via an output control module. The video data sent to the remote end is transmitted to the network interface via the Ethernet Media Access Controller (EMAC) and displayed in the form of a webpage (i.e., the remote user accesses the local server remotely through the network interface). At the same time, the software running on the server management control chip collects information from the host end, motherboard information (in text mode), and motherboard alarm information, which are also displayed on the webpage. Therefore, users can monitor and manage the local server remotely on the webpage.

[0121] Local users have two options: either view the local server's operating system interface directly on the monitor, or view the overlaid video interface. The choice is made via... Figure 1 It is implemented in the SELECT module (pattern selection module).

[0122] The system also allows for viewing server host information (such as the model and quantity of the central processing unit (CPU), the quantity and type of memory and hard drives) and alarm information (such as CPU overheating) locally. In traditional solutions, the software of the basic management control chip composites the host screen (i.e., the host operating system screen) and the text screen (i.e., server host information and alarm information). The composite screen is then cached in DDR, configured with DVI interface timings, and connected to a local monitor, enabling local viewing of server information and alarm information. Figure 1The video data is stored in space A of the Double Data Rate Synchronous Dynamic Random Access Memory (DDR), the composite video data is stored in space C, and the compressed video data output by the output control module is stored in space B. Furthermore, the host, server management control chip, and server motherboard are connected via interfaces; the number of interfaces can be configured according to actual needs. Figure 1 The interfaces are represented by Interface 0, Interface 1, ..., Interface M, which correspond to Module 0', Module 1', ..., Module M' on the server motherboard.

[0123] In traditional solutions, during the process of obtaining composited video information from DDR, there is often a situation where control signals (Hs / Vs / Data_en / R_DATA / G_DATA / B_DATA, etc.) have begun to form a frame or line of data on the DVI interface, but the composited video data cannot be obtained from DDR in time (because the DDR bus controller does not continuously provide this module with its function). Since the timing of the DVI interface is continuous, in traditional solutions, the pixel positions of the unobtainable R_DATA / G_DATA / B_DATA data are padded with 0s before being sent to the local display. This results in many black dots (pixels padded with 0s) in the displayed image, leading to very poor display quality. At the same time, there is a significant risk that critical information cannot be displayed correctly (the pixel positions of critical information are also padded with 0s).

[0124] Figure 2 This is a schematic diagram of a traditional local video synthesis controller, such as... Figure 2 As shown, in this scheme, AXI_RD_MASTER (Advanced eXtensible Interface (AXI) read controller) is responsible for reading the composited video from DDR and then buffering it into the ASYNC_FIFO (Asynchronous First In First Out (FIFO)) module. The DATA_CTRL (data control module) reads the ASYNC_FIFO and sends it to the DVI_CTRL (Digital Vision Interface (DVI) timing generator). When the DVI_CTRL starts generating valid R / G / B data output but the ASYNC_FIFO is empty, the DATA_CTRL pads it with zeros. (Because once standard DVI timing output starts, it cannot be paused, otherwise it will cause display abnormalities). The REG_CTRL (register configuration module) is responsible for configuring the registers of the LOCAL_VIDEO_CTRL module. The composite video processing method provided by this invention can solve the above-mentioned technical problems.

[0125] Figure 3A flowchart of a video synthesis processing method provided in an embodiment of the present invention is shown below. Figure 3 As shown, the method includes:

[0126] S11: Acquire the synthetic video data stored in the Double Data Rate Synchronous Dynamic Random Access Memory;

[0127] S12: Determine the row and column numbers corresponding to the data to be supplemented based on the synthesized video data; and determine the cache address corresponding to the data to be supplemented based on the row and column numbers corresponding to the data to be supplemented.

[0128] S13: Determine the block pixel size weight of the read-back data of the synthetic video data based on the size positional relationship between the block pixel size occupied by the read-back data and the block pixel size occupied by the supplementary data, and obtain the supplementary data through prediction processing;

[0129] S14: Add supplementary data to the cache address so that it can be transmitted to the local end for display.

[0130] Specifically, the composite video data in step S11 can be composite data based on traditional schemes, or composite video data accessed according to priority settings in DDR, without any limitation here. Figure 4 A schematic diagram illustrating video overlay synthesis for server management and control chip software in traditional solutions, as shown below. Figure 4 As shown, within the central processing unit, the host screen is captured, a text screen is generated from this text screen, and then the images are superimposed and composited. Both host information and alarm information are text-based. After searching a digital database, the text information is used to create an image screen. This image screen is then superimposed and composited with the interface information of the host operating system read by the server management controller's CPU software.

[0131] Priority access increases the priority of the controller's access to DDR during the valid data generation period of the DVI timing sequence, so as to ensure that the composite data cached in DDR is read back to the local video overlay composite as soon as possible.

[0132] In step S12, determining the row and column numbers of the data to be supplemented based on the synthesized video data can be understood as referring to data that cannot be traditionally retrieved from DDR, i.e., data that would need to be padded with zeros in traditional solutions. If no data is retrieved during the synthesized video data reading process, it can be set to be considered data to be supplemented if no data is found within a preset time. The row and column numbers of the data to be supplemented are determined based on the row and column numbers of the already retrieved data and the resolution information of the entire synthesized video data. The corresponding cache address is then determined based on these row and column numbers. This cache address is obtained by adding the row and column numbers of the data to be supplemented to the cache address of the already retrieved data.

[0133] In step S13, the block pixel size weight of the read-back data can be determined based on the size positional relationship between the block pixel size occupied by the read-back data and the block pixel size occupied by the supplementary data in the synthetic video data, so as to obtain the supplementary data through predictive processing. It can be understood that the block pixel size is the block size simulated based on the current amount of data written and the entire data set.

[0134] In some embodiments, the block pixel size occupied by the read-back data of the synthesized video data and the block pixel size occupied by the cache address corresponding to the data to be supplemented are both based on the block size set by dividing the synthesized video data into multiple regions or pixel blocks in the random access memory array.

[0135] In video processing, block size typically refers to the basic unit in the video encoding process. The size and division of blocks may differ across different video encoding standards. In video processing, block size not only affects encoding efficiency but can also influence video compression ratio and quality. For example, a larger block size may provide a higher compression ratio but may result in loss of detail; conversely, a smaller block size may retain more detail but may have a lower compression ratio. In practical video processing, it may be necessary to select an appropriate block size and other video parameters based on the specific application scenario and requirements. Therefore, it is necessary to assign weights to the positions of the block pixels occupied by the read-back data and the block pixels occupied by the data to be supplemented, with the weight increasing as the pixel size closer to the block pixels of the data to be supplemented.

[0136] The supplementary data is predicted by multiplying and accumulating the corresponding read data based on the size weights and the corresponding read data. The supplementary data is then added to the cache address and transmitted to the local end for display.

[0137] This invention provides a method for processing composite video, comprising: acquiring composite video data stored in a double data rate synchronous dynamic random access memory; determining the row and column numbers corresponding to the data to be supplemented based on the composite video data; determining the cache address corresponding to the data to be supplemented based on the row and column numbers; determining the block pixel size weight of the read-back data of the composite video data based on the size positional relationship between the block pixel size occupied by the read-back data of the composite video data and the block pixel size occupied by the data to be supplemented, so as to predict and process to obtain supplementary data; and supplementing the supplementary data into the cache address for transmission to the local end for display. The cache address of the data to be supplemented is determined based on the row and column numbers corresponding to the data to be supplemented. Compared with the traditional zero-padding mechanism, this invention records the cache address of the data to be supplemented. Based on the size positional relationship between the block pixel size of the read-back data and the block pixel size of the data to be supplemented, the weight of the block pixel size of the read-back data relative to the data to be supplemented is determined. Then, based on the relationship between the block pixel size and the read-back data, the supplemented data can be predicted and placed within the cache address. This ensures that the value of the supplemented pixel is as close as possible to the original pixel and that the pixel data is balanced with the pixel data corresponding to the key information that was actually missed. This avoids the problem of poor display quality caused by black spots in the display screen due to the traditional zero-padding mechanism. By predicting the pixel position data of data that cannot be obtained from DDR, the quality of the composite video is improved and the risk of missing key information is reduced.

[0138] In some embodiments, acquiring synthetic video data stored in a double data rate synchronous dynamic random access memory includes:

[0139] Obtain the access priority of each bus in the Double Data Rate Synchronous Dynamic Random Access Memory;

[0140] Within each bus access priority, the access priority of the video overlay and synthesis controller is adjusted to the top N priorities of each bus transmission priority; where N is greater than or equal to 1.

[0141] The synthesized video data stored in the synchronous dynamic random access memory at double data rate is retrieved according to the access order of the top N priority levels.

[0142] Specifically, the access priority of each DDR bus is obtained. Within each bus access priority, the access priority of the video overlay and compositing controller (LOCAL_VIDEO_CTRL module) is prioritized. The priority can be the highest level, the second highest level, or any of the top few priorities, which can be set according to the actual situation. The composite video data of DDR is obtained according to the access order of the top N priorities.

[0143] The process of acquiring synthetic video data provided in this embodiment is obtained by adjusting the bus access priority to ensure that the corresponding synthetic video data is read as much as possible during the period when the enable signal is high, thereby minimizing the need for subsequent data supplementation.

[0144] In some embodiments, acquiring synthetic video data stored in a double data rate synchronous dynamic random access memory according to the access order of the top N priorities includes:

[0145] Obtain the data enable signal corresponding to the digital vision interface of the video overlay and synthesis controller;

[0146] When the data enable signal is in the high state, the synthesized video data stored in the double data rate synchronous dynamic random access memory is obtained according to the access order of the first N priorities.

[0147] Specifically, when the data enable signal (DATA_EN) corresponding to the Digital Visual Interface (DVI) of the video overlay compositing controller is high, the composite video data stored in DDR is obtained based on the first N priority access order.

[0148] In some embodiments, when the data enable signal is in a low state, the method further includes:

[0149] Output an interrupt command to the central processing unit (CPU) so that the CPU can adjust the access priority of the video overlay and compositing controller;

[0150] This includes adjusting the access priority of the video overlay and compositing controller, including:

[0151] Adjust the access priority of the video overlay and compositing controller to the last priority of each bus transmission priority.

[0152] Specifically, when the data enable signal is pulled low, an interrupt instruction will be generated to the central processing unit (CPU) so that the CPU can adjust the access priority of the video overlay and compositing controller, making the access priority lower or the last priority, in order to minimize the impact of the video overlay and compositing controller on other functions of the SoC chip and improve the overall performance of the chip.

[0153] Figure 5 This is an architecture diagram of a video overlay and synthesis controller provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the priority adjustment submodule (PRIORITY_ADJUST) implements the function of controlling the DDR_CTRL (DDR controller, in...) Figure 5The first controller is used to modify the priority of the video overlay and compositing controller (LOCAL_VIDEO_CTRL) accessing DDR. RGB data is only needed to be retrieved from DDR during the period when the data enable signal (DATA_EN) is high. Therefore, this submodule receives control signals from the digital vision interface timing generator (DVI_CTRL submodule). When the DATA_EN signal of the DVI_CTRL submodule is about to go high, such as... Figure 5 During the `Ver_addr_time` segment, such as the `H_Back_Porch` and `H_Left_Border` segments, the `PRIORITY_ADJUST` submodule will generate an interrupt to the CPU. Upon receiving the corresponding interrupt, the software notifies the `DDR_CTRL` (DDR controller) to adjust the `LOCAL_VIDEO_CTRL` module's priority for accessing DDR, setting it to the highest or second-highest priority. This ensures that during the period when `DATA_EN` is high, the submodule can read the corresponding RGB data as much as possible, minimizing the need for subsequent data supplementation. Simultaneously, when the `DATA_EN` signal of the `DVI_CTRL` submodule goes low, the `PRIORITY_ADJUST` submodule will generate an interrupt to the CPU (this interrupt differs from the previously described interrupt; it consists of two interrupt signals). Upon receiving this interrupt, the software again notifies the `DDR_CTRL` to adjust the `LOCAL_VIDEO_CTRL` module's priority for accessing DDR, setting it to the lowest priority to minimize the impact of the `LOCAL_VIDEO_CTRL` module on other chip functions and improve overall chip performance.

[0154] by Taking resolution as an example, one frame (the interval between two Vsyns) is The clock cycle per pixel is 432640, while the clock cycle for pixels with DATA_EN high is... =307200, therefore the percentage of valid data is =307200 / 432640=71%. This can greatly increase the probability of reading complete and valid data, and minimize the impact on other functions of the chip.

[0155] The priority access order adjustment provided in this embodiment ensures that as much data as possible is read when the data enable signal is high, and adjusts its priority access order to last when the data enable signal is low, so as to avoid affecting other functions of the chip.

[0156] In some embodiments, determining the row and column numbers corresponding to the data to be supplemented based on the synthesized video data includes:

[0157] Get the current pixel of the current frame data of the synthesized video data;

[0158] If the current pixel does not receive the corresponding composite video data at the pixel position of the first-in-first-out buffer of the video overlay and compositing controller within a preset time, then the current pixel is determined to be data to be supplemented.

[0159] The row and column numbers of the data to be supplemented are determined based on the current pixel position in the synthesized video data.

[0160] Specifically, the current pixel of the current frame data is obtained. If the current pixel does not receive the pixel to be written at the pixel position of the first-in-first-out buffer of the video overlay and synthesis controller within a preset time, the current pixel is regarded as the data to be supplemented, and the row number and column number of the data to be supplemented are determined based on the position of the current pixel.

[0161] The process of determining the row and column numbers corresponding to the data to be supplemented provided in this embodiment is based on the fact that the first-in-first-out buffer of the video overlay and synthesis controller cannot obtain the corresponding data in a timely manner. This facilitates the subsequent determination of cached data based on the row and column numbers of the data to be supplemented, improving the reading efficiency of valid data, avoiding the reading of invalid data, and improving the utilization rate of the DDR bus.

[0162] In some embodiments, determining the cache address corresponding to the data to be supplemented based on the row number and column number of the data to be supplemented includes:

[0163] Obtain the resolution information of the synthesized video data and the initial address in the double data rate synchronous dynamic random access memory; wherein, the resolution information includes the resolution width and resolution height;

[0164] Subtract 1 from the row number corresponding to the data to be supplemented to get the first row number;

[0165] Multiply the first row number by the resolution width to obtain the row resolution information of the read data;

[0166] The current position information of the data to be supplemented is obtained by adding the row resolution information of the data already read back to the column number corresponding to the data to be supplemented.

[0167] The current position information corresponding to the data to be supplemented is multiplied by the bit data corresponding to the unit pixel to obtain the bit data corresponding to the data to be supplemented.

[0168] The bit data corresponding to the data to be supplemented is divided by the bit data in the address space of the synchronous dynamic random access memory at double data rate to obtain the bit data in the address space occupied by the data to be supplemented.

[0169] The cache address corresponding to the data to be supplemented is obtained by adding the bits of the address space occupied by the data to be supplemented to the initial address.

[0170] Specifically, assuming the initial address of the synthesized video data in DDR is INIT_ADDR, and the resolution information includes resolution width (RSL_L) and resolution height (RSL_W), such as... In the figure, 1024 represents the resolution width and 768 represents the resolution height.

[0171] The cache address corresponding to the data to be supplemented is obtained using the following formula:

[0172] ;

[0173] Wherein, LINE_NUM is the row number corresponding to the data to be supplemented, and LINE_NUM-1 is the first row number; This represents the row resolution information of the data already read back; ROW_NUM represents the column number corresponding to the data to be supplemented. 24 represents the current position information corresponding to the data to be supplemented; 8 represents the bit data corresponding to the unit pixel; 8 represents the bit data of the unit double data rate synchronous dynamic random access memory address space; INIT_ADDR is the initial address.

[0174] This embodiment provides a method to calculate the DDR cache address of the data to be supplemented based on the row and column numbers of the data to be supplemented, thereby enabling modification of the generated read address, avoiding the reading of invalid data, and improving the overall performance of the SoC chip.

[0175] In some embodiments, before determining the block pixel size weight of the read-back data of the synthetic video data based on the size positional relationship between the block pixel size occupied by the read-back data of the synthetic video data and the block pixel size occupied by the supplementary data, and before predicting the supplementary data, the method further includes:

[0176] Obtain the amount of remaining data in the first-in-first-out buffer of the video overlay and compositing controller;

[0177] The number of pixels read back from the Double Data Rate Synchronous Dynamic Random Access Memory is determined based on the bit width data of the Advanced Extensible Interface and the bit data corresponding to each pixel.

[0178] The readback mechanism of the double data rate synchronous dynamic random access memory is determined based on the relationship between the rate at which the remaining data is read to the local end and the rate at which the number of pixels is read back.

[0179] When the readback mechanism of the Double Data Rate Synchronous Dynamic Random Access Memory is in normal readback mode, it waits to read the composite video data from the Double Data Rate Synchronous Dynamic Random Access Memory.

[0180] When the readback mechanism of the Double Data Rate Synchronous Dynamic Random Access Memory is abnormal, the process proceeds to the step of determining the block pixel size weight of the readback data of the synthetic video data based on the size positional relationship between the block pixel size occupied by the readback data of the synthetic video data and the block pixel size occupied by the supplementary data, in order to predict and process the supplementary data.

[0181] Specifically, such as Figure 5 As shown, the DDR readback mechanism is determined based on the relationship between the transfer rate of the remaining data in the FIFO buffer of the video overlay and compositing controller to the local end and the rate at which the number of pixels is read back. The number of pixels read back is determined based on the bit width data of the Advanced Extensible Interface (AXI) and the bit data corresponding to each pixel. For example, if the AXI bus data bit width is 128 bits and each pixel occupies 32 bits, then one AXI read operation will read back the number of 4 pixels.

[0182] When DDR readback is normal, it means that data can be read back from DDR, and no prediction is needed; simply wait for the data to be read. When DDR readback is abnormal, prediction in step S13 is required.

[0183] In some embodiments, the readback mechanism of the double data rate synchronous dynamic random access memory is determined based on the relationship between the rate at which the remaining data is read to the local end and the rate at which the number of pixels is read back, including:

[0184] Obtain the number of reads from the double data rate synchronous dynamic random access memory within a first preset time period;

[0185] Obtain the frequency period of the pixel clock of the digital vision interface corresponding to the current resolution information;

[0186] The first timing parameters for the synthesized video data to be displayed after the first-in-first-out buffer has been read are determined based on the remaining data volume and frequency period.

[0187] The second time parameter is determined based on the first preset time and the number of readbacks to determine the number of pixels to be read back from the double data rate synchronous dynamic random access memory, corresponding to the number of composite video data to be displayed.

[0188] The corresponding readback mechanism is determined based on the first and second time parameters.

[0189] Specifically, the first time parameter T1 for reading the composite video data to be displayed from the FIFO buffer is determined based on the remaining data volume and frequency period. N is the remaining data volume, and the frequency period is Period, specifically the frequency period of the DVI interface pixel clock (SYS_CLK) at the current resolution. If the pixel clock is 200MHz, then Period = 5ns, i.e., Period = 1000 / SYS_CLK (MHz). The first preset time is T0, and the number of readbacks is M. The number of times ARREADY and RREADY are high within the T0 segment is counted, M, i.e., M data readbacks. The second time parameter T2 for reading the composite video data to be displayed corresponding to the number of pixels read back from the double data rate synchronous dynamic random access memory is determined based on the first preset time and the number of readbacks. The corresponding readback mechanism is determined based on the first and second time parameters.

[0190] In some embodiments, determining the corresponding readback mechanism based on a first time parameter and a second time parameter includes:

[0191] If the first time parameter is greater than the second time parameter, then the readback mechanism of the double data rate synchronous dynamic random access memory is determined to be a normal readback.

[0192] If the first time parameter is less than or equal to the second time parameter, then the readback mechanism of the double data rate synchronous dynamic random access memory is determined to be an abnormal readback.

[0193] If T1 > T2, the predicted data can be read completely, meaning the data for the next frame can be read back and displayed. If T1 ≤ T2, after the predicted data is read completely, no new data can be read back and displayed, indicating an abnormal readback.

[0194] In some embodiments, the triggering process of the readback mechanism of the Double Data Rate Synchronous Dynamic Random Access Memory includes:

[0195] Get the preset remaining amount in the first-in-first-out buffer;

[0196] If the remaining data amount is greater than or equal to the preset remaining amount, the readback mechanism will not be triggered.

[0197] If the remaining data amount is less than the preset remaining amount, the readback mechanism will be triggered.

[0198] The timing of activating the estimation module can be flexibly configured by the user. For example, the READY_ESTIMATE module function can be activated when the amount of remaining data in ASYNC_FIFO is small (e.g., when 1 / 5 of the capacity remains), and the READY_ESTIMATE module function can be deactivated otherwise.

[0199] This embodiment provides a method to estimate, before prediction processing, whether there is any remaining data that can be fully read out and whether the displayed data can be read back from DDR, by estimating the current remaining data volume in the prediction module and the frequency of increase within a preset time. This saves resources corresponding to the actual prediction processing and ensures that prediction processing is performed when the data is truly unavailable.

[0200] In some embodiments, the block pixel size weight of the read-back data in the synthesized video data is determined based on the positional relationship between the block pixel size occupied by the read-back data and the block pixel size occupied by the supplementary data, in order to obtain supplementary data through prediction processing, including:

[0201] The synthesized video data is divided into multiple preset pixel sizes; wherein, the preset pixel size includes preset rows;

[0202] The read-back data and the data to be supplemented are respectively arranged at the pixel size of the first target preset size and the pixel size of the second target preset size;

[0203] Based on the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, the weight parameters of the read-back data in the corresponding preset row are set respectively; wherein, the sum of each weight parameter is 1;

[0204] Supplementary data is obtained by weighting and summing the read data with the weight parameters of the corresponding preset rows.

[0205] Specifically, the synthesized video data is divided into multiple preset pixel sizes based on the block pixel size, and each preset pixel size includes preset rows. The read-back data and the data to be supplemented are respectively arranged within the pixel sizes of the first target preset size and the second target preset size. Weight parameters for the read-back data in the corresponding preset rows are set according to their positional relationships, wherein the sum of the weight parameters for the read-back data in the preset rows is 1.

[0206] Supplementary data is obtained by weighting and summing the read data with the weight parameters of the corresponding preset rows. In other words, the supplementary data is obtained by weighted summation.

[0207] This embodiment provides a method to adjust the weight parameters so that the values ​​of the supplemented pixels are as close as possible to the original pixel values. At the same time, a READY prediction mechanism is set to achieve a balance between predicted pixel data and real pixel data, which greatly improves the quality of the synthesized video displayed on the local end.

[0208] In some embodiments, with The block size is based on the estimated size, and the number of random access memory arrays (RAM_ARRAY) is 5. The data to be supplemented is placed in the first row of a preset row at the pixel size of the second target preset size. When the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the first target preset size and the position of the data to be supplemented in the preset row of the second target preset size. These parameters include:

[0209] Set the weight parameter of the read-back data of the first target preset size to the row that is horizontally adjacent to the preset row of the pixel size of the data to be supplemented in the second target preset size;

[0210] The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the second weight parameter;

[0211] The weight parameters of the read-back data in the last row of the first target preset size, which are adjacent to the pixel size of the data to be supplemented in the tilt direction of the second target preset size, are set as the third weight parameter; wherein the third weight parameter includes the first weight sub-parameter and the second weight sub-parameter; within the read-back data to which the third weight parameter belongs, according to the read-back order, the weight parameters of the read-back data in the last row of the first target preset size that are close to the read-back time of the data to be supplemented are set as the first weight sub-parameter; the weight parameters of the read-back data in the last row of the first target preset size that are not close to the read-back time of the data to be supplemented are set as the second weight sub-parameter; wherein the first weight sub-parameter is greater than the second weight sub-parameter;

[0212] For each preset row of pixel size of the first target preset size, set the corresponding fourth weight parameter of the read data from bottom to top according to the row order. The fourth weight parameter includes the third weight sub-parameter, the fourth weight sub-parameter and the fifth weight sub-parameter. The third weight sub-parameter is greater than the fourth weight sub-parameter. The fourth weight sub-parameter is greater than the fifth weight sub-parameter. The first weight parameter is greater than the second weight parameter. The second weight parameter is greater than the third weight parameter. The third weight parameter is greater than the fourth weight parameter.

[0213] Figure 6 This is a schematic diagram of a data prediction scenario provided by an embodiment of the present invention, as shown below. Figure 6 As shown, each square represents The block data, in which Figure 5The data prediction module (DATA_PREDICT) stores the data to be predicted, which is also the data to be displayed. It performs predictions based on the A1 / B1 / C1 / D1 data already stored in the RAM array. The dimensions of A1 / B1 / C1 are... D1 is The data of A1 / B1 / C1 / D1 are weighted and calculated.

[0214] The general weighting design uses the following parameters: the first weight parameter is the D1 data; the second weight parameter is the pixel size vertically adjacent to the data to be supplemented, located in the last row of the B1 data in the same column; the third weight parameter is set to the pixel size adjacent to the data to be supplemented in the tilt direction of the pixel size of the second target preset size, and is the read-back data in the last row of the first target preset size. Figure 6 The term "middle" refers to the last row of data A1 and data C1. The third weight parameter includes the first and second weight sub-parameters, with the first weight sub-parameter being the last row of data C1 and the second weight sub-parameter being the last row of data A1. The fourth weight parameter is a general term, based on rows 0 to 2 of data A1, B1, and C1 respectively. The weight sub-parameters corresponding to rows 0 to 2 of data A1 are respectively the fifth, fourth, and third weight sub-parameters.

[0215] Combination Figure 6 Specifically, for the data in row 0 of A1 ( Configure the weight, weight_A1_0;

[0216] For the first row of data in A1 ( Configure the weight, weight_A1_1;

[0217] For the second row of data in A1 ( Configure the weight, weight_A1_2;

[0218] For the data in row 3 of A1 ( Configure the weight, weight_A1_3;

[0219] For the data in B1, row 0 ( Configure the weight, weight_B1_0;

[0220] For the first row of data in B1 ( Configure the weight, weight_B1_1;

[0221] For the second row of data in B1 ( Configure the weight, weight_B1_2;

[0222] For the data in B1, the third row ( Configure the weight, weight_B1_3;

[0223] For the data in C1, row 0 ( Configure the weight, weight_C1_0;

[0224] For the first row of data in C1 ( Configure the weight, weight_C1_1;

[0225] For the second row of data in C1 ( Configure the weight, weight_C1_2;

[0226] For the data in C1, the third row ( Configure the weight, weight_C1_3;

[0227] Configure the weight for D1 (which has only one row), with the weight being weight_D1_0;

[0228] The principle for configuring weights is that the closer the weight is to DATA_PREDICT, the larger the weight should be.

[0229] weight_A1_3>weight_A1_2>weight_A1_1>weight_A1_0;

[0230] weight_B1_3>weight_B1_2>weight_B1_1>weight_B1_0;

[0231] weight_C1_3>weight_C1_2>weight_C1_1>weight_C1_0;

[0232] weight_D1_0>weight_B1_3>weight_A1_3>weight_C1_3;

[0233] And the sum of all weights equals 1;

[0234] Based on the product of the weight and the corresponding data ( The sums of these are accumulated to obtain DATA_PREDICT. .

[0235] In some embodiments, the data to be supplemented is in the second row of a preset row at a pixel size of the second target preset size, and when the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the first target preset size pixel size and the position of the data to be supplemented in the preset row of the second target preset size pixel size, including:

[0236] Set the weight parameter of the read-back data of the first row of the corresponding pixel size of the second target preset size to the fifth weight parameter;

[0237] Set the weight parameter of the last row of the first target preset size to which the data to be supplemented belongs in the preset row of the preset row of the pixel size of the second target preset size, and set the weight parameter of the data read back to the sixth weight parameter.

[0238] Set the weight parameter of the read-back data of the second-to-last row of the first target preset size, which is adjacent to the preset row of the second target preset size pixel size of the data to be supplemented, as the seventh weight parameter;

[0239] The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the eighth weight parameter.

[0240] The weight parameter of the read-back data in the last row of the first target preset size, which is adjacent to the pixel size of the data to be supplemented in the tilt direction of the second target preset size, is set as the ninth weight parameter; wherein the ninth weight parameter includes the sixth weight sub-parameter and the seventh weight sub-parameter; within the read-back data to which the ninth weight parameter belongs, according to the read-back order, the weight parameter of the read-back data in the last row of the first target preset size that is close to the read-back time of the data to be supplemented is set as the sixth weight sub-parameter; the weight parameter of the read-back data in the last row of the first target preset size that is not close to the read-back time of the data to be supplemented is set as the seventh weight sub-parameter; wherein the sixth weight sub-parameter is greater than the seventh weight sub-parameter;

[0241] For each preset row of pixel size of the first target preset size, set the tenth weight parameter corresponding to the read data from bottom to top according to the row order. The tenth weight parameter includes the eighth weight sub-parameter and the ninth weight sub-parameter. The eighth weight sub-parameter is greater than the ninth weight sub-parameter. The fifth weight parameter is greater than the sixth weight parameter. The sixth weight parameter is greater than the seventh weight parameter. The seventh weight parameter is greater than the eighth weight parameter. The eighth weight parameter is greater than the ninth weight parameter. The ninth weight parameter is greater than the tenth weight parameter.

[0242] Specifically, Figure 7 This is a schematic diagram of a data prediction scenario two provided by an embodiment of the present invention, as shown below. Figure 7 As shown, each square represents The block data, in which Figure 5 The data prediction module (DATA_PREDICT) stores the data to be predicted, which is also the data to be displayed. It performs predictions based on the A2 / B2 / C2 / D2 / E data already stored in the RAM array. The dimensions of A2 / B2 / C2 are... D2 is E1 is The data of A2 / B2 / C2 / D2 / E1 are weighted and calculated.

[0243] The general weight design uses the following parameters: the fifth weight parameter is E1 data; the sixth weight parameter is the first row of read-back data D2, which belongs to the last row of the first target preset size, adjacent to the row of the data to be supplemented at the preset row of the second target preset size; the seventh weight parameter is the 0th row of read-back data, which belongs to the second-to-last row of the first target preset size, adjacent to the row of the data to be supplemented at the preset row of the second target preset size; the eighth weight parameter is the last row of B2 data; and the ninth weight parameter is collectively referred to as the last row of A2 and C2 data, where the sixth weight sub-parameter is the last row of C2 data, and the seventh weight sub-parameter is the last row of A2 data. The tenth weight parameter is a general term, based on rows 1 and 2 of A2 data, rows 1 and 2 of B2 data, and rows 1 and 2 of C2 data. The weight sub-parameters corresponding to rows 1 and 2 of A2 data are respectively the eighth and ninth weight sub-parameters.

[0244] For the first row of data in A2 ( Configure the weight, weight_A2_1;

[0245] For the second row of data in A2 ( Configure the weight, weight_A2_2;

[0246] For the data in row 3 of A2 ( Configure the weight, weight_A2_3;

[0247] For the first row of data in B2 ( Configure the weight, weight_B2_1;

[0248] For the second row of data in B2 ( Configure the weight, weight_B2_2;

[0249] For the data in B2, the third row ( Configure the weight, weight_B2_3;

[0250] For the first row of data in C2 ( Configure the weight, weight_C2_1;

[0251] For the second row of data in C2 ( Configure the weight, weight_C2_2;

[0252] For the data in C2, the third row ( Configure the weights, weight_C2_3;

[0253] For the data in row 0 of D2 ( Configure the weight, weight_D2_0;

[0254] For the first row of data in D2 ( Configure the weight, weight_D2_1;

[0255] For the data in E1, row 0 ( Configure the weight, weight_E1_0;

[0256] The principle for configuring weights is that the closer the weight is to DATA_PREDICT, the larger the weight should be.

[0257] weight_A2_3>weight_A2_2>weight_A2_1;

[0258] weight_B2_3>weight_B2_2>weight_B2_1;

[0259] weight_C2_3>weight_C2_2>weight_C2_1;

[0260] weight_E1_0>weight_D2_1>weight_D2_0>weight_B2_3>weight_C2_3>weight_A2_3;

[0261] And the sum of all weights equals 1;

[0262] Based on the product of the weight and the corresponding data ( The sums of these numbers are accumulated to obtain DATA_PREDICT;

[0263] .

[0264] In some embodiments, the data to be supplemented is in the third row of a preset row at the pixel size of the second target preset size, and when the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, including:

[0265] Set the weight parameters of the read data in the second and first rows of the corresponding pixel size of the second target preset size to the eleventh and thirteenth weight parameters respectively;

[0266] Set the weight parameter of the read-back data of the same row of the first target preset size that is horizontally adjacent to the preset row of the second target preset size pixel size of the data to be supplemented as the twelfth weight parameter;

[0267] Set the weight parameters of the read data of the first and second rows of the first target preset size to the corresponding weight parameters of the preset row of the preset row of the second target preset size pixel size that are horizontally adjacent to the data to be supplemented in the second target preset size.

[0268] The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the sixteenth weight parameter.

[0269] The weight parameter of the read-back data in the last row of the first target preset size, which is adjacent to the pixel size of the data to be supplemented in the tilt direction of the second target preset size, is set as the seventeenth weight parameter; wherein, the seventeenth weight parameter includes the tenth weight sub-parameter and the eleventh weight sub-parameter; within the read-back data to which the seventeenth weight parameter belongs, according to the read-back order, the weight parameter of the read-back data in the last row of the first target preset size that is close to the read-back time of the data to be supplemented is set as the tenth weight sub-parameter; the weight parameter of the read-back data in the last row of the first target preset size that is not close to the read-back time of the data to be supplemented is set as the eleventh weight sub-parameter; wherein, the tenth weight sub-parameter is greater than the eleventh weight sub-parameter;

[0270] Set the read-back data of each preset row of pixel size of each first target preset size to the eighteenth weight parameter; wherein, the eleventh weight parameter is greater than the twelfth weight parameter; the twelfth weight parameter is greater than the thirteenth weight parameter; the thirteenth weight parameter is greater than the fourteenth weight parameter; the fourteenth weight parameter is greater than the fifteenth weight parameter; the fifteenth weight parameter is greater than the sixteenth weight parameter; the sixteenth weight parameter is greater than the seventeenth weight parameter; and the seventeenth weight parameter is greater than the nineteenth weight parameter.

[0271] Specifically, Figure 8 This is a schematic diagram of a data prediction scenario three provided by an embodiment of the present invention, as shown below. Figure 8 As shown, each square represents The block data, in which Figure 5 The data prediction module (DATA_PREDICT) stores the data to be predicted, which is also the data to be displayed. It performs predictions based on the A3 / B3 / C3 / D3 / E data already stored in the RAM array. The dimensions of A3 / B3 / C3 are... D3 is E2 is The data in A3 / B3 / C3 / D3 / E2 are weighted and calculated.

[0272] The general weight design uses the following parameters: the eleventh and thirteenth weight parameters are the 0th and 1st rows of E2 data, respectively; the twelfth weight parameter is the 2nd row of D3 data, which is the horizontally adjacent row of the data to be supplemented in the preset row of the second target preset size pixel size, belonging to the same row of the first target preset size; the fifteenth and fourteenth weight parameters are the 0th and 1st rows of D3 data, respectively; the sixteenth weight parameter is the last row of B3 data, which is the vertically adjacent pixel size of the data to be supplemented in the second target preset size pixel size, and is in the same column as the preset row of the data to be supplemented in the second target preset size pixel size; the seventeenth weight parameter is collectively referred to as the last row of A3 and C3 data, where the tenth weight sub-parameter is the last row of C3 data, and the eleventh weight sub-parameter is the last row of A3 data; and the eighteenth weight parameter is a general term, based on the 2nd row of A3 data, the 2nd row of B3 data, and the 2nd row of C3 data.

[0273] For the second row of data in A3 ( Configure the weight, weight_A3_2;

[0274] For the data in row 3 of A3 ( Configure the weight, weight_A3_3;

[0275] For the second row of data in B3 ( Configure the weight, weight_B3_2;

[0276] For the data in B3, the third row ( Configure the weight, weight_B3_3;

[0277] For the second row of data in C3 ( Configure the weight, weight_C3_2;

[0278] For the data in C3, the third row ( Configure the weight, weight_C3_3;

[0279] For the data in D3, row 0 ( Configure the weight, weight_D3_0;

[0280] For the first row of data in D3 ( Configure the weight, weight_D3_1;

[0281] For the second row of data in D3 ( Configure the weight, weight_D3_2;

[0282] For the data in E2, row 0 ( Configure the weight, weight_E2_0;

[0283] For the first row of data in E2 ( Configure the weight, weight_E2_1;

[0284] The principle for configuring weights is that the closer the weight is to DATA_PREDICT, the larger the weight should be.

[0285] weight_A3_3>weight_A3_2;

[0286] weight_B3_3>weight_B3_2;

[0287] weight_C3_3>weight_C3_2;

[0288] weight_E2_1>weight_D3_2>weight_E2_0>weight_D3_1>weight_D3_0>weight_B3_3>weight_C3_3>weight_A3_3;

[0289] And the sum of all weights equals 1;

[0290] Based on the product of the weight and the corresponding data ( The sums of these numbers are accumulated to obtain DATA_PREDICT.

[0291] .

[0292] In other embodiments, the data to be supplemented is placed in the last row of a preset row at the pixel size of the second target preset size, and when the data to be supplemented is in the fifth pixel size, weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, including:

[0293] Set the weight parameters of the read data of the third, second, and first rows of the corresponding pixel size of the second target preset size to the twentieth, twenty-second, and twenty-fourth weight parameters respectively;

[0294] Set the weight parameter of the read-back data of the same row of the first target preset size that is horizontally adjacent to the preset row of the second target preset size pixel size of the data to be supplemented as the twenty-first weight parameter; set the weight parameters of the read-back data of the first row, second row, and third row of the first target preset size that are horizontally adjacent to the preset row of the second target preset size pixel size of the data to be supplemented as the twenty-third weight parameter, the twenty-fifth weight parameter, and the twenty-sixth weight parameter, respectively.

[0295] The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the twenty-seventh weight parameter.

[0296] The weight parameter of the pixel size adjacent to the data to be supplemented in the pixel size tilt direction of the second target preset size, and the weight parameter of the read-back data in the last row of the first target preset size, is set as the twenty-eighth weight parameter; wherein, the twenty-eighth weight parameter includes the twelfth weight sub-parameter and the thirteenth weight sub-parameter; the twelfth weight sub-parameter is greater than the thirteenth weight sub-parameter; the weight parameters from largest to smallest are: the twentieth weight parameter, the twenty-first weight parameter, the twenty-second weight parameter, the twenty-third weight parameter, the twenty-fourth weight parameter, the twenty-fifth weight parameter, the twenty-sixth weight parameter, the twenty-seventh weight parameter, and the twenty-eighth weight parameter.

[0297] Specifically, Figure 9 This is a schematic diagram of a data prediction scenario four provided by an embodiment of the present invention, as shown below. Figure 9 As shown, each square represents The block data, in which Figure 5 The data prediction module (DATA_PREDICT) stores the data to be predicted, which is also the data to be displayed. It performs predictions based on the A4 / B4 / C4 / D4 / E3 data already stored in the RAM array. The dimensions of A4 / B4 / C4 are... D4 is E3 is The data in A4 / B4 / C4 / D4 / E3 are weighted and calculated.

[0298] In the general weight design, the 20th, 22nd, and 24th weight parameters are the 0th, 1st, and 2nd rows of E3 data, respectively; the 21st, 23rd, 25th, and 26th weight parameters are the 0th to 3rd rows of D4 data, respectively; the 27th weight parameter is the last row of B4 data; and the 28th weight parameter is collectively referred to as the last row of A4 and C4 data, where the 12th weight sub-parameter is the last row of C4 data and the 13th weight sub-parameter is the last row of A4 data.

[0299] For the data in row 3 of A4 ( Configure the weight, weight_A4_3;

[0300] For the data in B4, the third row ( Configure the weight, weight_B4_3;

[0301] For the data in C4, the third row ( Configure the weights, weight_C4_3;

[0302] For the data in row 0 of D4 ( Configure the weight, weight_D4_0;

[0303] For the first row of data in D4 ( Configure the weight, weight_D4_1;

[0304] For the second row of data in D4 ( Configure the weight, weight_D4_2;

[0305] For the data in row 3 of D4 ( Configure the weight, weight_D4_3;

[0306] For the data in E3, row 0 ( Configure the weight, weight_E3_0;

[0307] For the first row of data in E3 ( Configure the weight, weight_E3_1;

[0308] For the second row of data in E3 ( Configure the weight, weight_E3_2;

[0309] And the total weight of all items equals 1.

[0310] The principle for configuring weights is that the closer the weight is to DATA_PREDICT, the larger the weight should be.

[0311] weight_E3_2>weight_D4_3>weight_E3_1>weight_D4_2>weight_E3_0>weight_D4_1>weight_D4_0>weight_B4_3>weight_C4_3>weight_A4_3;

[0312] Based on the product of the weight and the corresponding data ( The sums of these numbers are accumulated to obtain DATA_PREDICT.

[0313] .

[0314] The four scenario prediction scenarios provided in this embodiment are based on the principle of configuring weights, with greater weight given to values ​​closer to the data to be supplemented. This ensures that the values ​​of the supplemented pixels are as close as possible to the original pixel values, improving prediction accuracy and significantly enhancing the quality of the synthesized video displayed locally.

[0315] In some embodiments, after supplementing the cache address with supplementary data, the method further includes:

[0316] The supplementary data is merged with the read-back data to obtain the supplemented synthetic video data;

[0317] Determine whether the supplemented synthetic video data is the same as the synthetic video data stored in the Double Data Rate Synchronous Dynamic Random Access Memory;

[0318] If they are the same, then send them to the local end.

[0319] Specifically, the supplementary data and the read-back data are merged to obtain the supplemented composite video data. The supplemented composite video data is then verified to be the same as the composite video data stored in the double data rate synchronous dynamic random access memory. If the verification is successful, the data is sent to the local end for display.

[0320] The verification method provided in this embodiment greatly improves the display instructions for the synthesized video on the local end, ensuring the accuracy of the data display.

[0321] In some embodiments, sending the supplemented composite video data to a local device includes:

[0322] The supplemented synthetic video data is processed in a time-series format using a digital vision interface to obtain digital vision data.

[0323] Send digital visual data to the local device.

[0324] Specifically, the main function of the DVI controller module is to form a standard DVI timing sequence and input relevant control signals to PRIORTY_ADJUST to realize the dynamic adjustment of the DDR access priority of the LOCAL_VIDEO_CTRL module.

[0325] Figure 10 A standard DVI timing diagram is provided for an embodiment of the present invention, as follows: Figure 10 As shown, HSync is the horizontal synchronization signal, which is pulled high for N clock cycles per line; VSync is the vertical synchronization signal, which is pulled high for M clock cycles per frame. For example... At this resolution, VSync is raised by 3 lines in one frame of signal ( The clock cycle is 480 Vsync signals between the two Vsync signals, and each Hsync signal pulls high for 40 pixel clock cycles.

[0326] DATA_EN is a data valid signal, which is only high within the Ver_addr_time and Hor_addr_time segments.

[0327] R_DATA[7:0]: R component data, 8 bits;

[0328] G_DATA[7:0]: G component data, 8 bits;

[0329] B_DATA[7:0]: B component data, 8 bits.

[0330] RGB data is only valid during the period when DATA_EN is high. If ASYNC_FIFO is empty during the period when DATA_EN is high, data cannot be obtained from DDR in time, and data needs to be added to the R_DATA, G_DATA, and B_DATA signals.

[0331] The foregoing has described in detail various embodiments of the synthetic video processing method. Based on this, the present invention also discloses a synthetic video processing apparatus corresponding to the above-described method. Figure 11 This is a structural diagram of a video synthesis processing apparatus provided in an embodiment of the present invention. Figure 11 As shown, the video processing device includes:

[0332] The acquisition module 11 is used to acquire the synthetic video data stored in the double data rate synchronous dynamic random access memory;

[0333] The determination module 12 is used to determine the row number and column number corresponding to the data to be supplemented based on the synthesized video data; and to determine the cache address corresponding to the data to be supplemented based on the row number and column number corresponding to the data to be supplemented.

[0334] The prediction processing module 13 is used to determine the block pixel size weight of the read-back data of the synthetic video data based on the size position relationship between the block pixel size occupied by the read-back data of the synthetic video data and the block pixel size occupied by the data to be supplemented, so as to obtain the supplementary data through prediction processing.

[0335] The supplementary module 14 is used to supplement the cache address with supplementary data so that it can be transmitted to the local end for display.

[0336] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments described in the method part for the embodiments of the device part, and will not be repeated here.

[0337] For a description of the synthetic video processing apparatus provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above synthetic video processing method.

[0338] Figure 12 This is a structural diagram of a video synthesis processing device provided in an embodiment of the present invention, as shown below. Figure 12 As shown, the device includes:

[0339] Memory 21 is used to store computer programs;

[0340] Processor 22 is used to implement the steps of a method for processing synthetic video when executing a computer program.

[0341] The processor 22 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 22 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 22 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 22 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0342] The memory 21 may include one or more non-volatile storage media, which may be non-transitory. The memory 21 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 21 is used to store at least the following computer program 211, which, after being loaded and executed by the processor 22, is capable of implementing the relevant steps of the synthetic video processing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, and the storage method may be temporary storage or permanent storage. The operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include, but is not limited to, the data involved in the synthetic video processing method.

[0343] In some embodiments, the video processing device may further include a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.

[0344] Those skilled in the field can understand, Figure 12 The structures shown do not constitute a limitation on the processing equipment for synthesized video and may include more or fewer components than illustrated.

[0345] The processor 22 implements the synthetic video processing method provided in any of the above embodiments by calling instructions stored in the memory 21.

[0346] For a description of the synthetic video processing device provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above synthetic video processing method.

[0347] Furthermore, the present invention also provides a non-volatile storage medium storing a computer program, which, when executed by the processor 22, implements the steps of the above-described video synthesis processing method.

[0348] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0349] For an introduction to the non-volatile storage medium provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above-described video synthesis processing method.

[0350] Furthermore, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of a method for processing synthetic video.

[0351] For an introduction to the computer program product provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above-described synthetic video processing method.

[0352] The foregoing has provided a detailed description of a method, apparatus, device, medium, and product for processing synthesized video provided by the present invention. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the invention, and these improvements and modifications also fall within the protection scope of the present invention.

[0353] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

Claims

1. A method for processing synthesized video, characterized in that, include: Acquire synthetic video data stored in a double data rate synchronous dynamic random access memory; Determine the row and column numbers corresponding to the data to be supplemented based on the synthesized video data; The cache address corresponding to the data to be supplemented is determined based on the row number and column number of the data to be supplemented; The block pixel size weight of the read-back data of the synthesized video data is determined based on the size positional relationship between the block pixel size occupied by the read-back data and the block pixel size occupied by the data to be supplemented, so as to obtain the supplementary data through prediction processing; The supplementary data is added to the cache address so that it can be transmitted to the local terminal for display.

2. The method for processing synthesized video according to claim 1, characterized in that, The block pixel size occupied by the read-back data of the synthesized video data and the block pixel size occupied by the cache address corresponding to the data to be supplemented are both based on the block size set by dividing the synthesized video data into multiple regions or pixel blocks in the random access memory array.

3. The method for processing synthesized video according to claim 1, characterized in that, Acquire synthetic video data stored in a Double Data Rate Synchronous Dynamic Random Access Memory, including: Obtain the access priority of each bus of the double data rate synchronous dynamic random access memory; Within each bus access priority, the access priority of the video overlay and synthesis controller is adjusted to the top N priorities of each bus transmission priority; where N is greater than or equal to 1. The synthesized video data stored in the double data rate synchronous dynamic random access memory is obtained according to the access order of the first N priorities.

4. The method for processing synthesized video according to claim 3, characterized in that, The synthesized video data stored in the double data rate synchronous dynamic random access memory is obtained according to the access order of the first N priorities, including: Obtain the data enable signal corresponding to the digital visual interface of the video overlay synthesis controller; When the data enable signal is in a high state, the synthesized video data stored in the double data rate synchronous dynamic random access memory is obtained according to the access order of the first N priorities.

5. The method for processing synthesized video according to claim 4, characterized in that, When the data enable signal is in a low state, the method further includes: Output an interrupt command to the central processing unit so that the central processing unit can adjust the access priority of the video overlay and compositing controller; Adjusting the access priority of the video overlay and compositing controller includes: Adjust the access priority of the video overlay and synthesis controller to the last priority of each bus transmission priority.

6. The method for processing synthesized video according to any one of claims 1 to 5, characterized in that, The row and column numbers corresponding to the data to be supplemented are determined based on the synthesized video data, including: Obtain the current pixel of the current frame data of the synthesized video data; If the current pixel does not receive the corresponding pixel of the synthesized video data at the pixel position of the first-in-first-out buffer of the video overlay synthesis controller within a preset time, then the current pixel is determined to be data to be supplemented. The row and column numbers of the data to be supplemented are determined based on the pixel position of the current pixel in the synthesized video data.

7. The method for processing synthesized video according to claim 6, characterized in that, The cache address corresponding to the data to be supplemented is determined based on the row number and column number of the data to be supplemented, including: The resolution information of the synthesized video data and the initial address of the double data rate synchronous dynamic random access memory are obtained; wherein, the resolution information includes resolution width and resolution height; Subtract 1 from the row number corresponding to the data to be supplemented to obtain the first row number; The row resolution information of the read-back data is obtained by multiplying the first row number by the resolution width. The row resolution information of the read-back data is added to the column number corresponding to the data to be supplemented to obtain the current position information corresponding to the data to be supplemented; The current position information corresponding to the data to be supplemented is multiplied by the bit data corresponding to the unit pixel to obtain the bit data corresponding to the data to be supplemented. The bit data corresponding to the data to be supplemented is divided by the bit data in the address space of the synchronous dynamic random access memory at double data rate to obtain the bit data in the address space occupied by the data to be supplemented. The cache address corresponding to the data to be supplemented is obtained by adding the bit data of the address space occupied by the data to be supplemented to the initial address.

8. The method for processing synthesized video according to claim 7, characterized in that, Before determining the block pixel size weight of the read-back data of the synthesized video data based on the positional relationship between the block pixel size occupied by the read-back data of the synthesized video data and the block pixel size occupied by the data to be supplemented, and before predicting and processing to obtain the supplementary data, the method further includes: Obtain the remaining data volume in the first-in-first-out buffer of the video overlay and synthesis controller; The number of pixels read back from the Double Data Rate Synchronous Dynamic Random Access Memory is determined based on the bit width data of the Advanced Extensible Interface and the bit data corresponding to each pixel. The readback mechanism of the Double Data Rate Synchronous Dynamic Random Access Memory (DMR) is determined based on the relationship between the rate at which the remaining data is read to the local terminal and the rate at which the number of pixels is read back. When the readback mechanism of the DMR is in normal readback mode, the system waits to read the synthesized video data from the DMR. When the readback mechanism of the Double Data Rate Synchronous Dynamic Random Access Memory is abnormal, the process proceeds to the step of determining the block pixel size weight of the readback data of the synthetic video data based on the size positional relationship between the block pixel size occupied by the readback data of the synthetic video data and the block pixel size occupied by the data to be supplemented, in order to predict and process the supplementary data.

9. The method for processing synthesized video according to claim 8, characterized in that, The readback mechanism of the double data rate synchronous dynamic random access memory, determined based on the relationship between the rate at which the remaining data is read to the local end and the rate at which the number of pixels is read back, includes: Obtain the number of reads from the double data rate synchronous dynamic random access memory within a first preset time period; Obtain the frequency period of the pixel clock of the digital vision interface corresponding to the current resolution information; The first time parameter for the synthesized video data to be displayed after the first-in-first-out buffer has been read is determined based on the remaining data volume and the frequency period; A second time parameter is determined based on the first preset time and the number of readbacks to determine the number of pixels read back from the double data rate synchronous dynamic random access memory corresponding to the number of composite video data to be displayed; The corresponding readback mechanism is determined based on the first time parameter and the second time parameter.

10. The method for processing synthesized video according to claim 9, characterized in that, The corresponding readback mechanism is determined based on the first time parameter and the second time parameter, including: If the first time parameter is greater than the second time parameter, then the readback mechanism of the double data rate synchronous dynamic random access memory is determined to be a normal readback. If the first time parameter is less than or equal to the second time parameter, then the readback mechanism of the double data rate synchronous dynamic random access memory is determined to be an abnormal readback.

11. The method for processing synthesized video according to claim 10, characterized in that, The step of determining the block pixel size weight of the read-back data of the synthesized video data based on the positional relationship between the block pixel size occupied by the read-back data and the block pixel size occupied by the supplementary data, and then performing prediction processing to obtain supplementary data, includes: The synthesized video data is divided into multiple pixel sizes of preset dimensions; wherein, the pixel sizes of preset dimensions include preset rows; The read-back data and the data to be supplemented are respectively arranged at the pixel size of the first target preset size and the pixel size of the second target preset size; Based on the relationship between the position of the read-back data in the preset row of the first target preset size pixel size and the position of the data to be supplemented in the preset row of the second target preset size pixel size, the weight parameters of the read-back data in the corresponding preset row are set respectively; wherein, the sum of each weight parameter is 1; The supplementary data is obtained by weighting and summing the read-back data with the weight parameters of the corresponding preset rows.

12. The method for processing synthesized video according to claim 11, characterized in that, The data to be supplemented is in the first row of a preset row in the pixel size of the second target preset size, and when the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, including: Set the weight parameter of the read-back data of the first target preset size to the row that is horizontally adjacent to the preset row of the pixel size of the second target preset size of the data to be supplemented; The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the second weight parameter; The weight parameter of the read-back data in the last row of the first target preset size, which is adjacent to the pixel size of the data to be supplemented in the pixel size tilt direction of the second target preset size, is set as the third weight parameter; wherein, the third weight parameter includes a first weight sub-parameter and a second weight sub-parameter; within the read-back data to which the third weight parameter belongs, according to the read-back order, the weight parameter of the read-back data in the last row of the first target preset size that is close to the read-back time of the data to be supplemented is set as the first weight sub-parameter; the weight parameter of the read-back data in the last row of the first target preset size that is not close to the read-back time of the data to be supplemented is set as the second weight sub-parameter; wherein, the first weight sub-parameter is greater than the second weight sub-parameter; For each preset row of pixel size of the first target preset size, the fourth weight parameter corresponding to the read data is set to gradually decrease from bottom to top according to the row order; wherein, the fourth weight parameter includes a third weight sub-parameter, a fourth weight sub-parameter, and a fifth weight sub-parameter; the third weight sub-parameter is greater than the fourth weight sub-parameter; the fourth weight sub-parameter is greater than the fifth weight sub-parameter; the first weight parameter is greater than the second weight parameter; the second weight parameter is greater than the third weight parameter; the third weight parameter is greater than the fourth weight parameter.

13. The method for processing synthesized video according to claim 11, characterized in that, The data to be supplemented is in the second row of a preset row in the pixel size of the second target preset size, and when the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, including: Set the weight parameter of the read-back data of the first row of the corresponding pixel size of the second target preset size to the fifth weight parameter; Set the weight parameter of the last row of the first target preset size, which is horizontally adjacent to the row of the preset row of the second target preset size in the pixel size of the data to be supplemented, as the sixth weight parameter; The weight parameter of the read-back data in the second-to-last row of the first target preset size, which is adjacent to the preset row of the pixel size of the data to be supplemented in the second target preset size, is set as the seventh weight parameter; the weight parameter of the pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and is in the same column as the preset row of the pixel size of the data to be supplemented in the second target preset size, and is adjacent to the read-back data in the first target preset size, is set as the eighth weight parameter; The weight parameter of the read-back data in the last row of the first target preset size, which is adjacent to the pixel size of the data to be supplemented in the tilt direction of the second target preset size, is set as the ninth weight parameter; wherein, the ninth weight parameter includes a sixth weight sub-parameter and a seventh weight sub-parameter; within the read-back data to which the ninth weight parameter belongs, according to the read-back order, the weight parameter of the read-back data in the last row of the first target preset size that is close to the read-back time of the data to be supplemented is set as the sixth weight sub-parameter; the weight parameter of the read-back data in the last row of the first target preset size that is not close to the read-back time of the data to be supplemented is set as the seventh weight sub-parameter; wherein, the sixth weight sub-parameter is greater than the seventh weight sub-parameter; For each preset row of pixel size of the first target preset size, set the tenth weight parameter corresponding to the read data from bottom to top according to the row order, and gradually decrease it; wherein, the tenth weight parameter includes the eighth weight sub-parameter and the ninth weight sub-parameter; the eighth weight sub-parameter is greater than the ninth weight sub-parameter; the fifth weight parameter is greater than the sixth weight parameter; the sixth weight parameter is greater than the seventh weight parameter; the seventh weight parameter is greater than the eighth weight parameter; the eighth weight parameter is greater than the ninth weight parameter; the ninth weight parameter is greater than the tenth weight parameter.

14. The method for processing synthesized video according to claim 11, characterized in that, When the data to be supplemented is in the third row of a preset row in the pixel size of the second target preset size, and the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, including: Set the weight parameters of the read data in the second row and first row of the corresponding pixel size of the second target preset size to the eleventh weight parameter and the thirteenth weight parameter respectively; Set the weight parameter of the read-back data of the same row of the first target preset size that is horizontally adjacent to the preset row of the second target preset size pixel size of the data to be supplemented as the twelfth weight parameter; The weight parameters of the read data of the first and second rows of the first target preset size, which are horizontally adjacent to the preset row of the pixel size of the second target preset size, are set as the fifteenth weight parameter and the fourteenth weight parameter, respectively. The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the sixteenth weight parameter. The weight parameter of the read-back data in the last row of the first target preset size, which is adjacent to the pixel size of the data to be supplemented in the pixel size tilt direction of the second target preset size, is set as the seventeenth weight parameter; wherein, the seventeenth weight parameter includes a tenth weight sub-parameter and an eleventh weight sub-parameter; within the read-back data to which the seventeenth weight parameter belongs, according to the read-back order, the weight parameter of the read-back data in the last row of the first target preset size that is close to the read-back time of the data to be supplemented is set as the tenth weight sub-parameter; the weight parameter of the read-back data in the last row of the first target preset size that is not close to the read-back time of the data to be supplemented is set as the eleventh weight sub-parameter; wherein, the tenth weight sub-parameter is greater than the eleventh weight sub-parameter; The read-back data of each preset row of pixel size of each first target preset size is set as the eighteenth weight parameter; wherein, the eleventh weight parameter is greater than the twelfth weight parameter; the twelfth weight parameter is greater than the thirteenth weight parameter; the thirteenth weight parameter is greater than the fourteenth weight parameter; the fourteenth weight parameter is greater than the fifteenth weight parameter; the fifteenth weight parameter is greater than the sixteenth weight parameter; the sixteenth weight parameter is greater than the seventeenth weight parameter; and the seventeenth weight parameter is greater than the nineteenth weight parameter.

15. The method for processing synthesized video according to claim 11, characterized in that, The data to be supplemented is the last row in a preset row of the pixel size of the second target preset size, and when the data to be supplemented is in the fifth pixel size, the weight parameters of the read-back data in the corresponding preset row are set according to the relationship between the position of the read-back data in the preset row of the pixel size of the first target preset size and the position of the data to be supplemented in the preset row of the pixel size of the second target preset size, including: Set the weight parameters of the read data of the third, second, and first rows of the corresponding pixel size of the second target preset size to the twentieth, twenty-second, and twenty-fourth weight parameters respectively; The weight parameter of the read-back data in the same row of the first target preset size that is horizontally adjacent to the preset row of the second target preset size pixel size is set as the twenty-first weight parameter; the weight parameters of the read-back data in the first row, second row, and third row of the first target preset size that are horizontally adjacent to the preset row of the second target preset size pixel size pixel size are set as the twenty-third weight parameter, the twenty-fifth weight parameter, and the twenty-sixth weight parameter, respectively. The pixel size that is vertically adjacent to the pixel size of the data to be supplemented in the second target preset size, and whose preset row is in the same column as the pixel size of the data to be supplemented in the second target preset size, and whose weight parameter of the read-back data of the adjacent first target preset size is set as the twenty-seventh weight parameter. The weight parameter of the pixel size adjacent to the data to be supplemented in the pixel size tilt direction of the second target preset size, and the weight parameter of the read-back data in the last row of the first target preset size, is set as the twenty-eighth weight parameter; wherein, the twenty-eighth weight parameter includes the twelfth weight sub-parameter and the thirteenth weight sub-parameter; the twelfth weight sub-parameter is greater than the thirteenth weight sub-parameter; the weight parameters from largest to smallest are: the twentieth weight parameter, the twenty-first weight parameter, the twenty-second weight parameter, the twenty-third weight parameter, the twenty-fourth weight parameter, the twenty-fifth weight parameter, the twenty-sixth weight parameter, the twenty-seventh weight parameter, and the twenty-eighth weight parameter.

16. The method for processing synthesized video according to claim 8, characterized in that, The triggering process of the readback mechanism of the double data rate synchronous dynamic random access memory includes: Obtain the preset remaining amount of the first-in-first-out buffer; If the remaining data amount is greater than or equal to the preset remaining amount, the readback mechanism will not be triggered. If the remaining data amount is less than the preset remaining amount, the readback mechanism is triggered.

17. The method for processing synthesized video according to claim 1, characterized in that, After adding the supplementary data to the cache address, the process also includes: The supplementary data is merged with the read-back data to obtain the supplemented synthetic video data; Determine whether the supplemented synthetic video data is the same as the synthetic video data stored in the double data rate synchronous dynamic random access memory; If they are the same, then send them to the local end.

18. The method for processing synthesized video according to claim 17, characterized in that, The supplemented composite video data is sent to the local end, including: The supplemented synthetic video data is processed in a time-series format using a digital vision interface to obtain digital vision data. The digital visual data is sent to the local terminal.

19. A processing apparatus for synthesized video, characterized in that, include: The acquisition module is used to acquire the synthetic video data stored in the Double Data Rate Synchronous Dynamic Random Access Memory; The determining module is used to determine the row number and column number corresponding to the data to be supplemented based on the synthesized video data; and to determine the cache address corresponding to the data to be supplemented based on the row number and column number corresponding to the data to be supplemented. The prediction processing module is used to determine the block pixel size weight of the read-back data of the synthetic video data based on the size position relationship between the block pixel size occupied by the read-back data of the synthetic video data and the block pixel size occupied by the data to be supplemented, so as to obtain the supplementary data through prediction processing; The supplementary module is used to add the supplementary data to the cache address so that it can be transmitted to the local end for display.

20. A video processing apparatus, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the synthetic video processing method as described in any one of claims 1 to 18.

21. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a computer program, which, when executed by a processor, implements the steps of the video synthesis processing method as described in any one of claims 1 to 18.

22. A computer program product comprising a computer program / instructions, characterized in that, When executed by a processor, the computer program / instructions implement the steps of the synthetic video processing method according to any one of claims 1 to 18.

Images