Video processing method and system with real-time variable refresh rate for atmosphere light rendering

Abstract

The present invention discloses a video processing method and system with real-time variable refresh rate for atmosphere light rendering. The method includes: collecting an HDMI video signal and converting it into an RGB signal; obtaining the frame rates X1, X2, X3 of the current video source by detecting the field frequency signal V-SYNC; when the user selects and sets a constant frame rate and sets it as N, obtaining the constant frame rate N set by the user, and through data processing, outputting color rendering data with a fixed frequency in real time; when the user selects and sets a variable frame rate and sets it as M, obtaining the change rate M set by the user, and through data processing, outputting color rendering data with a variable frequency in real time. The present invention mainly collects the RGB signal output by the HDMI conversion chip, and according to the V-SYNC parsed from the signal, through different algorithms, matches the customer's requirements for the rendering effect, ensures that the rendering effect is consistent with the customer's requirements, and enriches the customer's subjective experience in different scenarios.

Description

Technical Field:

[0001] The present invention relates to the technical field of video processing, and in particular to a video processing method and system with a real-time variable refresh rate for atmosphere light rendering. Background Art:

[0002] A synchronous video acquisition device can collect multi-block color RGB signals of a video source in real time and synchronously send each block of RGB signals to corresponding intelligent atmosphere lights to achieve the effect of enhancing the user experience.

[0003] Currently, traditional video synchronous acquisition lighting rendering systems mainly target video sources with a fixed refresh rate and send rendering data to atmosphere lights at a fixed acquisition frequency. With the development of technology, technologies in the e-sports field - variable refresh rate (VRR) and high refresh rate (120Hz / 240Hz) are gradually applied to home display terminals TV. For the same video device, the refresh rate will vary dynamically according to the video content itself. The current system or system setting a fixed rendering frequency cannot match the refresh rate of the video source itself, and in some scenarios, the rendering lights may not keep up with the video, reducing the user's sensory enjoyment. Summary of the Invention:

[0004] The purpose of the present invention is to provide a video processing method and system with a real-time variable refresh rate for atmosphere light rendering to solve the deficiencies of the prior art.

[0005] The present invention is implemented by the following technical solutions: A video processing method with a real-time variable refresh rate for atmosphere light rendering, including the following steps:

[0006] Step 1, collect HDMI video signals and convert them into RGB signals;

[0007] Step 2, by detecting the field frequency signal V-SYNC, obtain the frame rates X1, X2, X3 of the current video source, where X1, X2, X3 are fixed frame rates or changing frame rates;

[0008] Step 3, the user selects to set a constant frame rate or a variable frame rate. When the user sets a constant frame rate, go to Step 4; when the user sets a variable frame rate, go to Step 5;

[0009] Step 4, the user selects to set a constant frame rate and sets it to N. By obtaining the constant frame rate N set by the user, output fixed-frequency color rendering data in real time through data processing;

[0010] Step 5, the user selects to set a variable frame rate and sets it to M. By obtaining the change rate M set by the user, output variable-frequency color rendering data in real time through data processing.

[0011] Further, the specific process of the step 4 for real-time output of color rendering data with a fixed frequency through data processing is as follows: The signals of one field collected are synthesized every Xn / N frames to obtain one frame of data. Within a fixed one-field time, N frames of signals are output to the rendering lamp system; the rendering lamp system renders at a fixed refresh rate of NHz, regardless of the refresh rate of the front-end signal source.

[0012] Further, the specific process of the step 5 for real-time output of color rendering data with a variable frequency through data processing is as follows: The signals of one field collected are synthesized every Xn / M frames to obtain one frame of data. Within a fixed one-field time, M frames of signals are output to the rendering lamp system; the rendering lamp system renders at a refresh rate of Xn / MHz and changes in real time following the change of the front-end signal source refresh rate.

[0013] Further, in the step 5, the user selects and sets a variable frame rate and sets it to M, where M includes three modes: fast, medium, and slow.

[0014] The present invention also provides a video processing system with a real-time variable refresh rate for atmosphere lamp rendering, including:

[0015] An HDMI signal output module for outputting two-way HDMI signal data;

[0016] An HDMI signal conversion module for converting and outputting the HDMI signal data into RGB signal data, and obtaining the frame rates X1, X2, X3 of the current video source by detecting the field frequency signal V-SYNC, where X1, X2, X3 are fixed frame rates or variable frame rates;

[0017] A data processing module for, when the user sets a constant frame rate, outputting color rendering data with a fixed frequency through data processing in real time, and when the user sets a variable frame rate, outputting color rendering data with a variable frequency through data processing in real time.

[0018] Further, the specific process of the data processing module for real-time output of color rendering data with a fixed frequency through data processing is as follows: The signals of one field collected are synthesized every Xn / N frames to obtain one frame of data. Within a fixed one-field time, N frames of signals are output to the rendering lamp system; the rendering lamp system renders at a fixed refresh rate of NHz, regardless of the refresh rate of the front-end signal source.

[0019] Further, the specific process of the data processing module for real-time output of color rendering data with a variable frequency through data processing is as follows: The signals of one field collected are synthesized every Xn / M frames to obtain one frame of data. Within a fixed one-field time, M frames of signals are output to the rendering lamp system; the rendering lamp system renders at a refresh rate of Xn / MHz and changes in real time following the change of the front-end signal source refresh rate.

[0020] Further, the HDMI signal conversion module is an HDMI conversion chip.

[0021] Further, the data processing module is an MCU or an FPGA.

[0022] Advantages of the present invention:

[0023] The present invention mainly collects the RGB signals output by the HDMI conversion chip, and according to the V-SYNC parsed from the signals, through different algorithms, matches the customer's requirements for the rendering effect, including constant refresh rate and variable refresh rate, to ensure that the rendering effect is consistent with the customer's requirements and enriches the customer's subjective experience in different scenarios. For example, when watching a movie, a constant refresh rate is selected to smooth the effect of the atmosphere light; when playing a game, a variable refresh rate is selected, and the atmosphere light changes more dynamically, and a better sense of immersion can be obtained. Description of the drawings:

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0025] Figure 1 It is a flowchart of the video processing method with real-time variable refresh rate for atmosphere light rendering according to an embodiment of the present invention;

[0026] Figure 2 It is a schematic block diagram of the video processing system with real-time variable refresh rate for atmosphere light rendering according to an embodiment of the present invention;

[0027] Figure 3 It is a flowchart of variable refresh rate signal processing of the video processing method with real-time variable refresh rate for atmosphere light rendering according to an embodiment of the present invention;

[0028] Figure 4 It is a flowchart of constant refresh rate signal processing of the video processing method with real-time variable refresh rate for atmosphere light rendering according to an embodiment of the present invention. Specific embodiments:

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0030] Such as Figure 1As shown, a video processing method with a real-time variable refresh rate for ambient light rendering includes the following steps:

[0031] Step 1: Collect the HDMI video signal and convert it into an RGB signal;

[0032] Step 2: By detecting the vertical sync signal V-SYNC, obtain the frame rates X1, X2, X3 of the current video source, where X1, X2, X3 are fixed frame rates or variable frame rates;

[0033] Step 3: The user selects to set a constant frame rate or a variable frame rate. When the user sets a constant frame rate, go to Step 4; when the user sets a variable frame rate, go to Step 5;

[0034] Step 4: The user selects to set a constant frame rate and sets it to N. By obtaining the constant frame rate N set by the user, after data processing, output color rendering data with a fixed frequency in real time;

[0035] Step 5: The user selects to set a variable frame rate and sets it to M. By obtaining the change rate M set by the user, after data processing, output color rendering data with a variable frequency in real time.

[0036] In this embodiment, the specific process of outputting color rendering data with a fixed frequency in real time in Step 4 is as follows: For a captured field signal, synthesize it every Xn / N frames to obtain one frame of data. Within a fixed field time, output N frames of signals to the rendering lamp system; the rendering lamp system renders at a fixed refresh rate of NHz, regardless of the refresh rate of the front-end signal source. For example, when the refresh frame rate of the front-end video is fixed at XHZ, the system can set the refresh frame rate of the rendering lamp to N (1 ≤ N ≤ 5), and the actual rendering frame rate is NHz. The system renders by extracting one frame of data every (X / N) frames; when the front-end video source supports functions such as Freesync or G-sync, the output signal source will have a real-time change in frame rate, such as the frame rate changing in the range of X1Hz ---- X2Hz. At this time, the system can set the refresh rate of the rendering lamp to N (1 ≤ N ≤ 5), and the actual rendering frame rate is NHz. The system renders by extracting one frame of data every (X1 / N) ---- (X2 / N) frames. As Figure 4 As shown, it is a signal processing flow chart with a constant refresh rate, and the output rendering lamp frame rate is NHz.

[0037] In this embodiment, the specific process of step 5 for real-time output of variable-frequency color rendering data through data processing is as follows: The captured signal for one field is synthesized every M frames to obtain one frame of data. Within a fixed one-field time, Xn / M frames of signals are output to the rendering lamp system; the rendering lamp system renders at a refresh rate of Xn / M Hz and changes in real time following the change of the front-end source refresh rate. For example, when the frame rate of the front-end video source is fixed at XHZ, the system can set the frame rate reduction of the rendering lamp to M (1 ≤ M ≤ 5), and the actual rendering frame rate is (X / M) Hz. The system renders by extracting one frame of data every M frames; when the front-end video source supports functions such as Freesync or G-sync, the output signal source will have a situation where the frame rate changes in real time, such as the frame rate changing within the range of X1 Hz ---- X2 Hz. At this time, the system can set the frame rate reduction of the rendering lamp to M (1 ≤ M ≤ 5), and the actual rendering frame rate is (X1 / M) Hz---(X2 / M) Hz. The system renders by extracting one frame of data every M frames. As Figure 3 shown, it is a flowchart of variable refresh rate signal processing, and the output rendering lamp frame rates are 90HZ / M, 48HZ / M, and 240HZ / M respectively.

[0038] In this embodiment, in step 5, the user selects and sets the variable frame rate to M, where M includes three modes: fast, medium, and slow.

[0039] As Figure 2 shown, the present invention also provides a video processing system with real-time variable refresh rate for atmosphere lamp rendering, including:

[0040] An HDMI signal output module for outputting two-way HDMI signal data;

[0041] An HDMI signal conversion module for converting and outputting the HDMI signal data into RGB signal data. By detecting the field frequency signal V-SYNC, the frame rates X1, X2, X3 of the current video source are obtained, and X1, X2, X3 are fixed frame rates or changing frame rates; among them, the HDMI signal conversion module is an HDMI conversion chip;

[0042] A data processing module for, when the user sets a constant frame rate, outputting fixed-frequency color rendering data through data processing in real time, and when the user sets a variable frame rate, outputting variable-frequency color rendering data through data processing in real time. Among them, the data processing module is an MCU or an FPGA.

[0043] As Figure 2 shown, the clock is used to identify the resolution of the current video. Only by detecting the corresponding resolution can the video be processed. The specific process is as follows:

[0044] 1. Detect the current clock DCLK to confirm the resolution of the video signal;

[0045] 2. Start collecting RGB signals and organize the data according to the corresponding resolution. For example, if the resolution is 1080P, the signal format of each frame collected is 1920*1080.

[0046] 3. H-sync is the line frequency, representing the data acquisition period of each frame. V-sync is the field frequency, representing the frame rate per second, that is, the refresh rate we mentioned here.

[0047] 4. The FPGA or MCU can collect each frame of data at the front end according to the clock signal, H-sync, and V-sync, and store it in the internal storage area.

[0048] 5. According to the needs of the back end (such as the constant refresh rate or variable refresh rate set by the customer), perform some operations on this data, and then use it for atmosphere light rendering.

[0049] The fixed frequency and variable frequency can be determined by different algorithms, which can be decided by the end customer according to their own needs. The difference between them is that the constant refresh rate uses a constant refresh rate for the rendering lights regardless of how the front-end signal refresh rate changes; the variable refresh rate is that the rendering lights completely follow the refresh rate of the front-end signal source (at a fixed downsampling factor N).

[0050] In this embodiment, the data processing module outputs color rendering data with a fixed frequency in real time through data processing. Specifically: for a field of signals collected, synthesize them every Xn / N frames to obtain one frame of data. Within a fixed field time, output N frames of signals to the rendering light system; the rendering light system renders at a fixed NHz refresh rate, unaffected by the front-end signal source refresh rate. For example, when the video refresh frame rate at the front end is fixed at XHZ, the system can set the refresh frame rate of the rendering lights to N (1≤N≤5), and the actual rendering frame rate is NHz. The system renders by extracting one frame of data every (X / N) frames; when the front-end video source supports functions such as Freesync or G-sync, the output signal source will have a real-time change in the frame rate, such as the frame rate change range is X1Hz----X2Hz. At this time, the system can set the refresh rate of the rendering lights to N (1≤N≤5), and the actual rendering frame rate is NHz. The system renders by extracting one frame of data every (X1 / N)----(X2 / N) frames. For example Figure 4As shown, it is a flowchart for processing a constant refresh rate signal, and the rendered light frame rates output are all N Hz. After the user sets it to NHZ, through algorithm processing, one frame is output every (X1 / N). For example, when the customer sets it to 10HZ and the refresh rate of the source is 60HZ (which can be known by detecting V-sync), the MCU will output one frame of data every (60 / 10) frames. If the customer sets N = 20, the system will output one frame of rendered data every (60 / 20) frames. If the refresh rate of the front end changes from 60HZ to 120HZ and the refresh rate of the back end is still 20HZ, the system will just output one frame of data every (120 / 20) frames. That is to say, no matter how the frame rate of the front-end signal source changes, the refresh rate of the back-end rendered light remains N = 20.

[0051] In this embodiment, the data processing module outputs color rendering data with a variable frequency in real time through data processing, specifically: synthesizing the collected signal of one field every M frames to obtain one frame of data, and outputting Xn / M frames of signals to the rendered light system within a fixed one-field time; the rendered light system renders at a refresh rate of Xn / MHz and changes in real time following the change of the front-end signal source refresh rate. For example, when the frame rate of the front-end video source is fixed at XHZ, the system can set the frame rate reduction of the rendered light to M (1 ≤ M ≤ 5), and the actual rendering frame rate is (X / M) Hz. The system renders by extracting one frame of data every M frames; when the front-end video source supports functions such as Freesync or G-sync, the output signal source will have a situation where the frame rate changes in real time, such as the frame rate change range is X1Hz ---- X2Hz. At this time, the system can set the frame rate reduction of the rendered light to M (1 ≤ M ≤ 5), and the actual rendering frame rate is (X1 / M) Hz---(X2 / M) Hz. The system renders by extracting one frame of data every M frames. As Figure 3 As shown, it is a flowchart for processing a variable refresh rate signal, and the rendered light frame rates output are 90HZ / M, 48HZ / M, and 240HZ / M respectively.

[0052] The frame rate reduction of variable refresh rate is a configurable parameter, which actually compresses the collected data at a fixed ratio. For example, if a 60HZ signal is collected and compressed by 1 / 6, only 10HZ of rendering data is output. The data of 6 frames is added for each pixel point and then divided by 6 to obtain an average value. The mathematical expression is P = (p1 + p2 + p3 + p4 + p5 + p6) / 6. Since the function of the rendering light is to set off an atmosphere and give the video of the TV or monitor an enlarged effect, the rendering light does not need to change too sharply. Therefore, after the collected video signal is downsampled and then used for the rendering light, a slow-changing setting-off effect can be achieved. However, the refresh rate requirements of each customer for the rendering light are different. For example, some game players hope that the change of the atmosphere light is faster to keep up with the rhythm of game scene switching. Therefore, the present invention provides a variable refresh rate setting, allowing users to determine the refresh rate of the rendering light according to their own needs.

[0053] The practical significance of the present invention lies in that: with the increasing number of home audio-visual devices, the requirements for immersive experience in home theaters and e-sports are getting higher and higher, and the products of audio-visual rendering lights are also becoming more and more abundant. With the increasing number of video terminals supporting VRR (variable refresh rate), the requirements for synchronous rendering lights are also getting higher and higher. At present, for products on the market, in the face of scenarios with fixed and variable refresh rates, generally a fixed refresh rate is set through a mobile APP to meet the customer's requirements for rendering refresh rate.

[0054] 1. If it is connected to a device with a fixed refresh rate, such as an ordinary video box, the refresh rate is generally 50 / 60HZ. At this time, the customer adjusts the rendering speed according to the change of the scene, and the speed can be set to "medium".

[0055] 2. If it is connected to a device with variable refresh rate, such as professional game consoles like PS5 and XBOX, since the refresh rate has a dynamic change from 48 to 120HZ, the customer generally sets the rendering speed to "fast".

[0056] However, due to the same video device, the refresh rate will have a dynamic change according to the video content itself. Therefore, setting a fixed rendering frequency through the APP cannot correspond to the refresh rate of the video source itself, and in some scenarios, the rendering light may not be able to keep up with the video, reducing the user's sensory enjoyment.

[0057] The present invention mainly collects the RGB signals output by the HDMI conversion chip, and according to the V-SYNC parsed from the signals, through different algorithms, it matches the customer's requirements for the rendering effect, including constant refresh rate and variable refresh rate, to ensure that the rendering effect is consistent with the customer's requirements and enriches the customer's subjective experience in different scenarios. For example, when watching a movie, a constant refresh rate is selected to smooth the effect of the ambient light; when playing games, a variable refresh rate is selected, and the ambient light changes more dynamically, and a better sense of immersion can be obtained.

[0058] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A video processing method with real-time variable refresh rate for ambient light rendering, characterized in that, It includes the following steps: Step 1, collect the HDMI video signal and convert it into an RGB signal; Step 2, obtain the frame rate of the current video source by detecting the field frequency signal, where the frame rate of the video source is a fixed frame rate or a variable frame rate; Step 3, the user selects to set a constant frame rate or a variable frame rate. When the user sets a constant frame rate, go to Step 4; when the user sets a variable frame rate, go to Step 5; Step 4, the user selects to set a constant frame rate and set it to N. By obtaining the constant frame rate set by the user, after data processing, output color rendering data with a fixed frequency in real time. Specifically: for a field of signals collected, synthesize them every Xn / N frames to obtain one frame of data, and within a fixed field time, output N frames of signals to the rendering lamp system; The rendering lamp system renders at a fixed refresh rate of N Hz, regardless of the refresh rate of the front-end signal source; where Xn is the frame rate of the video source; Step 5, the user selects to set a variable frame rate. By obtaining the set change rate of the user, after data processing, output color rendering data with a variable frequency in real time. Specifically: for a field of signals collected, synthesize them every M frames to obtain one frame of data, and within a fixed field time, output Xn / M frames of signals to the rendering lamp system; the rendering lamp system renders at a refresh rate of Xn / M Hz and changes in real time following the change of the front-end signal source refresh rate; where Xn is the frame rate of the video source.

2. The video processing method with real-time variable refresh rate for ambient light rendering according to claim 1, wherein The variable frame rate set by the user in Step 5 includes three modes: fast, medium, and slow.

3. A video processing system with real-time variable refresh rate for ambient light rendering, characterized in that, It includes: An HDMI signal output module for outputting two-way HDMI signal data; An HDMI signal conversion module for converting and outputting the HDMI signal data into RGB signal data, and obtaining the frame rate of the current video source by detecting the field frequency signal, where the frame rate of the video source is a fixed frame rate or a variable frame rate; A data processing module for, when the user sets a constant frame rate N, outputting color rendering data with a fixed frequency in real time after data processing. Specifically: for a field of signals collected, synthesize them every Xn / N frames to obtain one frame of data, and within a fixed field time, output N frames of signals to the rendering lamp system; The rendering lamp system renders at a fixed refresh rate of N Hz, regardless of the refresh rate of the front-end signal source; where Xn is the frame rate of the video source; When the user sets a variable frame rate, output color rendering data with a variable frequency in real time after data processing. Specifically: for a field of signals collected, synthesize them every M frames to obtain one frame of data, and within a fixed field time, output Xn / M frames of signals to the rendering lamp system; The rendering lamp system renders at a refresh rate of Xn / M Hz and changes in real time following the change of the front-end signal source refresh rate.

4. The video processing system with real-time variable refresh rate for atmosphere light rendering according to claim 3, characterized in that, The HDMI signal conversion module is an HDMI conversion chip.

5. The video processing system with a real-time variable refresh rate for ambient light rendering according to claim 3, wherein The data processing module is an MCU or an FPGA.

Images