Video data acquisition method and device based on frame synchronization control system

Abstract

The invention relates to a video data acquisition method and device based on a frame synchronization control system, and belongs to the field of optical imaging and image processing. The method is used for collecting the paired video training data of the image restoration neural network based on the deep learning technology. On one hand, by synchronizing with the video played on the display and controlling the camera to shoot frame by frame, the data of each frame is ensured to be accurately matched under the setting of each camera parameter, and the high synchronization between the high-quality video and the low-quality video is realized, which is extremely beneficial to the training of a deep learning model. And on the other hand, different types of video contents are played through the display, so that the diversity of the data set, including different weather conditions, time periods, indoor and outdoor scenes and the like, can be easily expanded, and the generalization ability of the model is improved. Through the innovative frame synchronization control system, the video data acquisition efficiency and quality are remarkably improved, and powerful data support is provided for the development of a deep learning video recovery technology.

Description

Technical Field

[0001] The present invention relates to a video data acquisition method and device, and specifically to a video data acquisition method and device based on a camera and screen frame synchronization control system, which is used to acquire paired video training data of an image restoration neural network based on deep learning technology, and belongs to the field of optical imaging and image processing. Background Art

[0002] In the field of video restoration, deep learning technology has become a key means to improve video quality, especially in denoising, super-resolution, and compression artifact removal. By using deep learning models such as convolutional neural networks and generative adversarial networks, researchers are able to restore high-quality detail information from low-quality videos. In order to train these deep learning models, a large number of paired data sets are required, that is, low-quality videos from the same scene and their corresponding high-quality versions. These paired video data are crucial for supervised learning because they directly affect the loss function calculation and network parameter update during training.

[0003] However, traditional video data collection methods face two major challenges. First, the matching degree between low-quality and high-quality video data is low. The shooting method using a dual-camera system essentially uses two different cameras. There are slight differences between the two cameras, and the positions are difficult to strictly align. Using the same camera to shoot twice is difficult to ensure that the trajectory is completely consistent, and it is impossible to capture the image of moving objects. This makes it difficult to achieve frame-by-frame matching between low-quality videos and high-quality videos, which will bring great difficulties in post-processing and training. Second, the scenes are not rich. Traditional video acquisition solutions require a lot of manpower for on-site shooting, resulting in a limited amount of video data and a lack of scene diversity, which limits the generalization ability of deep learning models.

[0004] In order to solve the above technical problems, there is an urgent need for a paired video data acquisition method and device that can ensure the matching degree of low-quality and high-quality videos and the richness of scenes. Summary of the invention

[0005] The purpose of the present invention is to address the defects and shortcomings of the prior art, and to solve the technical problems of the existing paired video data acquisition technology, such as low matching degree of collected low-quality and high-quality video data, and insufficient scenes, and to creatively propose a video data acquisition method and device based on a frame synchronization control system.

[0006] This method is used to collect paired video training data for image restoration neural networks based on deep learning technology. On the one hand, by synchronizing with the video played on the display, the camera is controlled to shoot frame by frame, ensuring that the data of each frame is accurately matched under the settings of various camera parameters, thereby achieving a high degree of synchronization between high-quality video and low-quality video, which is extremely beneficial for the training of deep learning models. On the other hand, by playing different types of video content on the display, the diversity of the data set can be easily expanded, including different weather conditions, time periods, indoor and outdoor scenes, etc., thereby increasing the generalization ability of the model. The present invention significantly improves the efficiency and quality of video data acquisition through an innovative frame synchronization control system, providing strong data support for the development of deep learning video restoration technology.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions.

[0008] A video data acquisition method based on a frame synchronization control system comprises the following steps:

[0009] Step S1: Initialization.

[0010] First, write various parameters required for this video acquisition into the frame synchronization control system, including the video data to be shot, the start and end frame positions of the shooting, and the parameter group list of the camera. Then, turn on the camera and screen in the darkroom to prepare for shooting.

[0011] Step S2: Switch frames.

[0012] The screen is controlled by the frame synchronization control system to switch to the frame that needs to be shot in the video.

[0013] Step S3: Adjust camera parameters.

[0014] The frame synchronization control system controls the camera to switch to a set of parameters in a pre-set camera parameter group list.

[0015] Step S4: Capture a frame.

[0016] The frame synchronization control system controls the camera to shoot the video frame currently displayed on the screen and saves the data to the frame synchronization control system.

[0017] Step S5: Camera parameter check.

[0018] It is determined whether all parameter groups in the preset camera parameter group list have been traversed. If the traversal is completed, continue; if the traversal is not completed, return to step S3 to continue adjusting the camera parameters and shoot the video frame under the next set of camera parameters.

[0019] Step S6: Video frame inspection.

[0020] It is determined whether all the pre-set video frames have been traversed. If the traversal is completed, the entire video acquisition process is terminated; if the traversal is not completed, it returns to step S2 to continue switching frames and capture the next video frame.

[0021] On the other hand, to achieve the purpose of the present invention, according to the above method, the present invention proposes a video data acquisition device based on a frame synchronization control system, including a camera module, a display module, a frame synchronization control system module and a darkroom module.

[0022] The camera module is used to shoot different types of video data and is fixed by a bracket.

[0023] The display module is used to display the video data to be shot. It is fixed by a bracket, and the center of the screen faces the lens of the camera module. The resolution and refresh rate of the display should be high enough to ensure that the content captured by the camera module has no moiré, pixel grid, blur, etc.

[0024] The frame synchronization control system module is used to control the camera module to switch camera parameters, control the display module to switch video frames, and receive and save the data collected by the camera module.

[0025] The darkroom module is used to wrap the camera module and the display module to isolate them from external light and reduce the reflection of the internal environment to ensure the quality of video shooting. The darkroom module is made of black opaque material and does not reflect light inside.

[0026] The connection relationship of the above modules is:

[0027] The lens of the camera module faces the center of the screen of the display module. When making the device, check the content captured by the camera and adjust the focus to ensure that the display content is clear and correct.

[0028] The control output end of the frame synchronization control system module is connected to the control input ends of the camera module and the display module respectively, and the data output end of the camera module is connected to the data input end of the frame synchronization control system module.

[0029] The camera module and the display module are both placed inside the darkroom module. The frame synchronization control system module is placed outside the darkroom module to facilitate parameter setting and export of captured data.

[0030] Beneficial Effects

[0031] The present invention specifically solves the technical problems of low-quality and low-quality video data matching and lack of scene richness collected by existing paired video data collection schemes. On the one hand, by synchronizing with the video played on the display, the camera is controlled to shoot frame by frame to ensure that the data of each frame is accurately matched under the setting of each camera parameter, thereby achieving a high degree of synchronization between high-quality video and low-quality video, which is extremely beneficial for the training of deep learning models. On the other hand, by playing different types of video content on the display, the diversity of the data set can be easily expanded, including different weather conditions, time periods, indoor and outdoor scenes, etc., thereby increasing the generalization ability of the model.

[0032] Through an innovative frame synchronization control system, the present invention significantly improves the efficiency and quality of video data acquisition, and provides strong data support for the development of deep learning video restoration technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is an operation flow chart of the method of the present invention;

[0034] Figure 2 It is a module composition diagram of the device of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0036] like Figure 1 As shown, the video data acquisition method based on the frame synchronization control system includes the following steps:

[0037] Step S1: Initialization.

[0038] Furthermore, firstly, various parameters required for this video acquisition are written into the frame synchronization control system, including the video data to be shot, the start and end frame positions of the shooting, and the parameter group list of the camera. Secondly, the camera and screen in the darkroom are turned on to prepare for shooting.

[0039] Specifically, after the video data to be shot is stored in the frame synchronization control system, each frame is extracted using the video parsing software. The video frame to be shot is located according to the input start frame and end frame positions. The subsequent shooting will start from the start frame until the program stops after the end frame is shot. The start frame and end frame positions are integer type parameters. In order to ensure the generalization performance of the trained deep learning model, it is necessary to use this method multiple times to shoot paired video data of different scenes.

[0040] Specifically, the camera parameter group list is a list of camera parameter groups, and each item in the list is a camera parameter group. Each camera parameter group contains various parameters that determine the camera shooting effect, including but not limited to exposure time, sensitivity, exposure compensation, focal length, white balance, etc. For example, paired video data shot with inaccurate focus and accurate focus can be used to train a deep learning model for focus recovery, paired video data with low exposure time and medium exposure time can be used to train a deep learning model for dark light video enhancement, and paired video data with high exposure time and medium exposure time can be used to train a deep learning model for overexposure correction.

[0041] Step S2: Switch frames.

[0042] Furthermore, the screen is controlled to switch to a frame required for shooting the video through a frame synchronization control system.

[0043] Specifically, when using a program to control the screen to switch frames, it is necessary to ensure that the video on the screen switches to a desired frame and remains stable before subsequent shooting.

[0044] Step S3: Adjust camera parameters.

[0045] Furthermore, the frame synchronization control system controls the camera to switch to a set of parameters in a preset camera parameter group list.

[0046] Specifically, as mentioned above, the camera parameter group list is a list of camera parameter groups, and each item in the list is a camera parameter group. Each camera parameter group contains various parameters that determine the camera shooting effect, including but not limited to exposure time, sensitivity, exposure compensation, focal length, white balance, etc. For example, paired video data shot with inaccurate focus and accurate focus can be used to train a deep learning model for focus recovery, paired video data with low exposure time and medium exposure time can be used to train a deep learning model for dark light video enhancement, and paired video data with high exposure time and medium exposure time can be used to train a deep learning model for overexposure correction.

[0047] Step S4: Capture a frame.

[0048] Furthermore, the camera is controlled by the frame synchronization control system to capture the video frame currently displayed on the screen, and the data is saved to the frame synchronization control system.

[0049] Specifically, when using a program to control a camera to shoot a screen, it is necessary to ensure that the video on the screen switches to a desired frame and remains stable before shooting.

[0050] Specifically, the frame synchronization control system can use a video encoding program to combine the captured separated video frames into a complete video and then save it to an internal memory, or directly store the separated video frames. Each video frame can be cropped before saving, retaining only the content displayed in the screen area and cropping out redundant information outside the screen.

[0051] Step S5: Camera parameter check.

[0052] Furthermore, it is determined whether all parameter groups in the preset camera parameter group list have been traversed. If the traversal is completed, continue; if the traversal is not completed, return to step 3 to continue adjusting the camera parameters and shoot the video frame under the next set of camera parameters.

[0053] Specifically, if the sequence number of the currently traversed camera parameter group exceeds the length of the camera parameter group list, the traversal is completed. During the traversal process, the sequence number of the camera parameter group increases by 1 each time.

[0054] Step S6: Video frame inspection.

[0055] Furthermore, it is determined whether all pre-set video frames have been traversed. If the traversal is completed, the entire video acquisition process is terminated. If the traversal is not completed, it returns to step 2 to continue switching frames and capture the next video frame.

[0056] Specifically, if the sequence number of the currently traversed video frame is greater than the sequence number of the ending frame, the traversal is completed. The sequence number of the video frame increases by 1 each time during the traversal process.

[0057] Figure 2 It is a module composition diagram of the device of the present invention.

[0058] like Figure 2 As shown, the video data acquisition device based on the frame synchronization control system includes a camera module M1, a display module M2, a frame synchronization control system module M3 and a darkroom module M4.

[0059] The camera module M1 is used to shoot different types of video data and is fixed by a bracket.

[0060] The display module M2 is used to display the video data to be shot. It is fixed by a bracket, and the center of the screen faces the lens of the camera module M1. The display resolution and refresh rate must be high enough to ensure that the content shot by the camera module M1 has no moiré, pixel grid, blur, etc.

[0061] The frame synchronization control system module M3 is used to control the camera module M1 to switch camera parameters, control the display module M2 to switch video frames, and receive and save the data collected by the camera module M1.

[0062] The darkroom module M4 is used to wrap the camera module M1 and the display module M2 to isolate them from external light and reduce the reflection of the internal environment to ensure the quality of video shooting. The darkroom module M4 is made of black opaque material and has no internal reflection.

[0063] The connection relationship between the above modules is:

[0064] The lens of the camera module M1 is facing the center of the screen of the display module M2. When making the device, check the content captured by the camera and adjust the focus to ensure that the display content is clear and correct.

[0065] The control output end of the frame synchronization control system module M3 is connected to the control input ends of the camera module M1 and the display module M2 respectively, and the data output end of the camera module M1 is connected to the data input end of the frame synchronization control system module M3.

[0066] The camera module M1 and the display module M2 are both placed inside the darkroom module M4. The frame synchronization control system module M3 is placed outside the darkroom module M4 to facilitate parameter setting and export of captured data.

[0067] Therefore, the invented video data acquisition method and device based on the frame synchronization control system can solve the technical problems such as low quality, low matching degree of high-quality video data, and insufficient scenes collected by the existing paired video data acquisition scheme, and is used to collect paired video training data of the image restoration neural network based on deep learning technology. On the one hand, by synchronizing with the video played on the display, the camera is controlled to shoot frame by frame to ensure that the data of each frame is accurately matched under the setting of each camera parameter, thereby achieving a high degree of synchronization between high-quality video and low-quality video, which is extremely beneficial for the training of deep learning models. On the other hand, by playing different types of video content on the display, the diversity of the data set can be easily expanded, including different weather conditions, time periods, indoor and outdoor scenes, etc., thereby increasing the generalization ability of the model. The present invention significantly improves the efficiency and quality of video data acquisition through an innovative frame synchronization control system, and provides strong data support for the development of deep learning video restoration technology.

Claims

1. A video data acquisition method based on a frame synchronization control system, characterized in that: The following steps are involved: Step S1: Initialization; first, write various parameters required for this video acquisition into the frame synchronization control system, then turn on the camera and screen in the darkroom to prepare for shooting; Step S2: Switching frames; controlling the screen to switch to a frame required for video capture through a frame synchronization control system; Step S3: adjusting the camera parameters; controlling the camera to switch to a set of parameters in a pre-set camera parameter group list through a frame synchronization control system; Step S4: shooting frames; controlling the camera through the frame synchronization control system, shooting the video frame currently displayed on the screen, and saving the data to the frame synchronization control system; Step S5: Camera parameter check; Determine whether all parameter groups in the preset camera parameter group list have been traversed; if the traversal is completed, continue; if the traversal is not completed, return to step S3 to continue adjusting the camera parameters and shoot the video frame under the next set of camera parameters; Step S6: video frame inspection; Determine whether all pre-set video frames have been traversed; if the traversal is completed, the entire video acquisition process ends; If the traversal is not completed, return to step S2 to continue switching frames and capture the next video frame.

2. A video data acquisition method based on a frame synchronization control system as claimed in claim 1, characterized in that: In step S1, the parameters include the video data to be shot, the start frame and end frame positions of the shooting, and the parameter group list of the camera.

3. A video data acquisition method based on a frame synchronization control system as claimed in claim 2, characterized in that: After the video data to be shot is stored in the frame synchronization control system, each frame is extracted using the video analysis software; the video frame to be shot is located according to the input start frame and end frame positions; the shooting will start from the start frame until the end frame is shot and the program stops; the start frame and end frame positions are integer type parameters; shoot paired video data of different scenes; The camera parameter group list is a list that records camera parameter groups. Each item in the list is a camera parameter group. Each camera parameter group contains various parameters that determine the camera's shooting effect, including exposure time, sensitivity, exposure compensation, focal length, and white balance.

4. The video data acquisition method based on a frame synchronization control system according to claim 1, characterized in that: In step S2, the screen is switched frame by using a program to ensure that the video on the screen switches to the desired frame and remains stable before subsequent shooting.

5. The video data acquisition method based on the frame synchronization control system according to claim 1, characterized in that: In step S4, the frame synchronization control system uses a video encoding program to assemble the captured separated video frames into a complete video and then save it to the internal memory, or directly stores the separated video frames; each video frame is cropped before saving, retaining only the content displayed in the screen area and cropping out redundant information outside the screen.

6. The video data acquisition method based on a frame synchronization control system according to claim 1, characterized in that: In step S5, if the sequence number of the currently traversed camera parameter group exceeds the length of the camera parameter group list, the traversal is completed, and the sequence number of the camera parameter group increases by 1 each time during the traversal process.

7. The video data acquisition method based on a frame synchronization control system according to claim 1, characterized in that: In step S6, if the sequence number of the currently traversed video frame is greater than the sequence number of the ending frame, the traversal is completed, and the sequence number of the video frame increases by 1 each time during the traversal process.

8. A video data acquisition device based on a frame synchronization control system according to claim 1, characterized in that: It includes a camera module, a display module, a frame synchronization control system module and a darkroom module, wherein: A camera module for capturing different types of video data; A display module, used to display the video data to be shot, with the center of the screen facing the lens of the camera module; The frame synchronization control system module is used to control the camera module to switch camera parameters, control the display module to switch video frames, and receive and save the data collected by the camera module; The darkroom module is used to wrap the camera module and the display module to isolate them from external light and reduce the reflection of the internal environment to ensure the quality of video shooting; The connection relationship of the above modules is: The lens of the camera module faces the center of the screen of the display module; when making the device, check the content captured by the camera and adjust the focus to ensure that the display content is clear and correct; The control output end of the frame synchronization control system module is connected to the control input end of the camera module and the display module respectively, and the data output end of the camera module is connected to the data input end of the frame synchronization control system module; The camera module and the display module are both placed inside the darkroom module; the frame synchronization control system module is placed outside the darkroom module to facilitate setting parameters and exporting captured data.

9. A video data acquisition device based on a frame synchronization control system as claimed in claim 8, characterized in that: The resolution and refresh rate of the display must ensure that the content captured by the camera module is free of moiré, pixel grid, and blur.

10. A video data acquisition device based on a frame synchronization control system as claimed in claim 8, characterized in that: The darkroom module is made of black opaque material and does not reflect light inside.

Images