Video processing method, device and equipment and storage medium thereof

Abstract

The embodiment of the invention belongs to the technical field of research and development design and image processing, is applied to a video synthesis scene under a multi-view camera, and relates to a video processing method, device and equipment and a storage medium thereof.The video processing method comprises the steps that video data collected by each camera in the multi-view camera is obtained; performing picture processing on the video data to obtain a video picture set corresponding to each camera; coding and decoding the video picture set corresponding to each camera in an alternate compression mode to obtain a decoded picture set; and according to the decoded picture set, performing collected video synthesis output. As the corresponding decoded pictures at the same acquisition time are formed by coding and decoding the pictures shot by the multi-view camera in an alternate compression mode, the overall compression efficiency and the compression ratio are improved, the decoded pictures have more detailed picture information, and it is guaranteed that the synthesized video is more detailed and clearer.

Description

Technical Field

[0001] The present application relates to the field of R&D design and image processing technology, and is applied to video synthesis scenarios under multi-cameras, and in particular to a video processing method, device, equipment and storage medium thereof. Background Art

[0002] Depth cameras based on multi-eye stereo vision can calculate parallax and depth, and can detect obstacles. Because of the additional depth information, the detection effect is better than that of monocular. With the development of multi-eye stereo cameras, such as binocular cameras and trinocular cameras, the processing of multi-eye stereo video data will also show an increasing trend.

[0003] For stereo video compression of multi-cameras, the existing method is to compress multiple cameras separately. Taking binocular cameras as an example, the left camera and the right camera are compressed separately, and then compressed and merged. This compression method is very inefficient, and the final merged video quality is relatively poor. How to improve the compression efficiency of multi-camera stereo video data and generate clearer and more detailed output videos has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a video processing method, device, equipment and storage medium thereof to improve the compression efficiency of multi-view stereoscopic video data and generate a clearer and more detailed output video.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a video processing method, which adopts the following technical solutions:

[0006] A video processing method comprises the following steps:

[0007] Acquire video data collected by a multi-camera, wherein the video data includes video data corresponding to each camera, and the multi-camera includes two or more cameras;

[0008] Processing the video data into images to obtain a video image set corresponding to each camera;

[0009] The video picture set corresponding to each camera is encoded and decoded by using an alternating compression method to obtain a decoded picture set;

[0010] According to the decoded picture set, the captured video is synthesized and outputted.

[0011] Furthermore, after executing the step of obtaining the video data collected by the multi-camera, the method further includes:

[0012] According to the corresponding relationship between the acquisition output and the distinguishing identification information of each camera, the video data corresponding to each camera is distinguishably marked;

[0013] Cache the video data corresponding to each camera after being distinguished and marked.

[0014] Furthermore, the step of processing the video data into images to obtain a video image set corresponding to each camera specifically includes:

[0015] According to the preset frame processing parameters, the video data corresponding to each camera is subjected to image segmentation processing, wherein the frame processing parameters include the frame processing start time point, the frame processing time interval and the frame processing end time point;

[0016] According to the distinguishing mark of the video data and the acquisition time of the video data, distinguishing mark processing is performed on all the images obtained by segmentation to obtain distinguishing identification information corresponding to all the images respectively;

[0017] Based on the distinguishing identification information corresponding to all the images, the segmented images of the same camera are sorted and organized according to the acquisition time, and the sorted and organized video image set corresponding to each camera is obtained.

[0018] Furthermore, the step of performing image segmentation processing on the video data corresponding to each camera head according to the preset frame processing parameters specifically includes:

[0019] Identify the frame processing start time point, frame processing time interval and frame processing end time point in the frame processing parameters;

[0020] The same frame splitting time interval is used for the video data corresponding to each camera, and the image segmentation processing is performed from the same frame splitting processing starting time point;

[0021] Until the frame processing end time point is reached, the frame processing of the video data corresponding to each camera is stopped to obtain the segmented pictures corresponding to each camera.

[0022] Furthermore, the step of performing distinguishing mark processing on all the images obtained by segmentation according to the distinguishing mark of the video data and the acquisition time of the video data to obtain distinguishing identification information corresponding to all the images specifically includes:

[0023] According to the distinguishing mark of the video data, obtain the distinguishing identification information of each camera;

[0024] Based on the distinguishing identification information of each camera, all segmented images are marked with the first distinguishing field;

[0025] Based on the collection time points corresponding to all the segmented images, and on the basis of the first distinguishing field marking process, a second distinguishing field marking process is performed on all the segmented images by using a field splicing method;

[0026] Obtain the distinguishing identification information corresponding to all the images formed by splicing the distinguishing identification information of each camera at the acquisition time point.

[0027] Furthermore, the step of encoding and decoding the video picture sets corresponding to each camera respectively by using the alternating compression method to obtain the decoded picture sets specifically includes:

[0028] According to the acquisition time of the pictures in the video picture set, the motion vector and motion residual between two consecutive pictures under the same camera are calculated;

[0029] According to the acquisition time of the pictures in the video picture set, the disparity vector and disparity residual of different cameras at two consecutive acquisition time points are calculated;

[0030] The motion vector, motion residual, disparity vector and disparity residual are used as coding supplementary parameters, and the distinguishing identification information corresponding to all pictures is used as coding control parameters, and the pictures in the video picture set are encoded by adopting an entropy coding method;

[0031] The encoding processing result is decoded, and all decoded pictures obtained by decoding are serialized and sorted according to the acquisition time point to obtain the decoded picture set.

[0032] Furthermore, the step of using the motion vector, motion residual, disparity vector and disparity residual as coding supplementary parameters, using the distinguishing identification information corresponding to all pictures as coding control parameters, and using entropy coding to encode pictures in the video picture set specifically includes:

[0033] Identify the encoding supplementary parameters corresponding to each picture;

[0034] According to the encoding control parameters, identifying the acquisition time point corresponding to each picture;

[0035] According to the encoding supplementary parameters and the acquisition time point, the images corresponding to different cameras at the same acquisition time point are encoded and synthesized to obtain the encoding synthesis results of the corresponding images at each acquisition time point;

[0036] The step of decoding the encoding processing result and serializing all decoded pictures obtained by decoding according to the acquisition time point to obtain the decoded picture set specifically includes:

[0037] According to the encoding synthesis result of the corresponding picture at each acquisition time point, decoding processing is performed to obtain a unique decoded picture at each acquisition time point;

[0038] For each unique decoded picture at each acquisition time point, a picture set is constructed in the order of the acquisition time points to generate the decoded picture set.

[0039] In order to solve the above technical problems, the embodiment of the present application also provides a video processing device, which adopts the following technical solution:

[0040] A video processing device, comprising:

[0041] A video data acquisition module is used to acquire video data collected by a multi-eye camera, wherein the video data includes video data corresponding to each eye camera, and the multi-eye camera includes two or more eyes;

[0042] A picture processing module, used to process the video data into pictures to obtain a video picture set corresponding to each camera;

[0043] The encoding and decoding processing module is used to perform encoding and decoding processing on the video picture set corresponding to each camera respectively by using an alternating compression method to obtain a decoded picture set;

[0044] The video synthesis output module is used to synthesize and output the captured video according to the decoded picture set.

[0045] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0046] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned video processing method when executing the computer-readable instructions.

[0047] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0048] A computer-readable storage medium stores computer-readable instructions, which implement the steps of the video processing method described above when executed by a processor.

[0049] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0050] The video processing method described in the embodiment of the present application obtains the video data collected by each camera in the multi-camera; performs image processing on the video data to obtain the video picture set corresponding to each camera; performs encoding and decoding processing on the video picture set corresponding to each camera in an alternating compression manner to obtain a decoded picture set; and performs acquisition video synthesis output according to the decoded picture set. Since the corresponding decoded pictures at the same acquisition time are obtained by encoding and decoding pictures taken by the multi-camera in an alternating compression manner, each decoded picture is synthesized by encoding, decoding and compression of multiple camera pictures, so that a decoded picture has more detailed picture information, ensuring that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0053] Figure 2 is a flow chart of an embodiment of a video processing method according to the present application;

[0054] Figure 3 is a flowchart of a specific embodiment of distinguishing and marking video data in the video processing method described in the present application;

[0055] Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 202 is shown;

[0056] Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 401 is shown;

[0057] Figure 6 yes Figure 4 A flowchart of a specific embodiment of step 402 is shown;

[0058] Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 203 is shown;

[0059] Figure 8 yes Figure 7 A flowchart of a specific embodiment of step 703 is shown;

[0060] Fig. 9 yes Figure 7 A flowchart of a specific embodiment of step 704 is shown;

[0061] Fig.10 is a structural schematic diagram of an embodiment of a video processing device according to the present application;

[0062] Fig.11 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0064] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0065] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0066] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0067] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0068] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0069] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0070] It should be noted that the video processing method provided in the embodiment of the present application is generally executed by a server, and accordingly, the video processing device is generally arranged in the server.

[0071] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0072] Continue to refer Figure 2 , shows a flow chart of an embodiment of a video processing method according to the present application. The video processing method comprises the following steps:

[0073] Step 201, obtaining video data collected by a multi-eye camera, wherein the video data includes video data corresponding to each eye camera, and the multi-eye camera includes two or more cameras.

[0074] Specifically, the multi-eye camera, for example, a binocular camera, a trio of cameras, or cameras with more than three eyes.

[0075] In this embodiment, the binocular camera generally includes a left eye and a right eye, and the video captured by the left eye and the video captured by the right eye are output in separate channels, that is, the left eye video data and the right eye video data can be obtained;

[0076] Correspondingly, the three-eye camera generally includes a left eye, a middle eye and a right eye, or an upper eye, a middle eye and a lower eye according to the position of the eyepiece, and each eye corresponds to different video data, that is, when the three-eye camera is a three-eye camera including a left eye, a middle eye and a right eye, the left eye video data, the middle eye video data and the right eye video data can be obtained; when the three-eye camera is a three-eye camera including an upper eye, a middle eye and a lower eye, the upper eye video data, the middle eye video data and the lower eye video data can be obtained;

[0077] Similarly, cameras with more than three eyes can also obtain video data corresponding to different eyes according to the different eyepiece numbers.

[0078] By acquiring the video data corresponding to each camera, the video data can be post-processed on a per-camera basis.

[0079] Step 202: Process the video data into images to obtain a video image set corresponding to each camera.

[0080] Specifically, the video data corresponding to different cameras are processed separately by frame. During the frame processing, frame by frame processing can be performed according to the actual shooting situation, or key frames can be extracted to finally obtain a set of video pictures corresponding to each camera.

[0081] By performing image processing on the video data, a video picture set corresponding to each camera head is obtained, so that the output video synthesis is performed according to the video picture set corresponding to each camera head later.

[0082] Step 203: Use an alternating compression method to encode and decode the video picture set corresponding to each camera to obtain a decoded picture set.

[0083] In this embodiment, an alternate compression mode is used for encoding and decoding processing, that is, different purpose video pictures at the same acquisition time point are firstly compressed alternately, and then encoded and decoded according to the acquisition time point.

[0084] Specifically, compared with the existing method of compressing the video picture set corresponding to each item separately and then encoding and decoding it, it can pay more attention to the detailed picture information at each acquisition time point, ensuring that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio.

[0085] Step 204: synthesize and output the captured video according to the decoded picture set.

[0086] Specifically, according to the sequence of all decoded pictures in the decoded picture set, the picture-to-video technology is adopted to perform video streaming processing on all the decoded pictures to generate the captured video, and the captured video is output through a unique output interface.

[0087] In this embodiment, since the corresponding decoded pictures at the same acquisition time are obtained by encoding and decoding pictures taken by multiple cameras in an alternating compression manner, each decoded picture is synthesized by encoding, decoding and compression of multiple camera pictures, so that a decoded picture has more detailed picture information, ensuring that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio.

[0088] In this embodiment, the video data collected by each camera in the multi-camera is obtained; the video data is processed into images to obtain the video picture set corresponding to each camera; the video picture set corresponding to each camera is encoded and decoded using an alternating compression method to obtain a decoded picture set; and the collected video is synthesized and output according to the decoded picture set. Since the corresponding decoded pictures at the same acquisition time are encoded and decoded by the pictures taken by the multi-cameras in an alternating compression method, each decoded picture is synthesized by encoding, decoding and compression of multiple camera pictures, so that a decoded picture has more detailed picture information, ensuring that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio.

[0089] Continue to refer Figure 3 In some optional implementations, after step 201, a step of distinguishing and marking the video data is also included. Figure 3 This is a flowchart of a specific embodiment of distinguishing and marking video data in the video processing method described in the present application, comprising the following steps:

[0090] Step 301, according to the acquisition output correspondence relationship and the distinguishing identification information of each camera, distinguishingly marking the video data corresponding to each camera;

[0091] Specifically, the acquisition and output correspondence relationship refers to that within the multi-camera, the video data acquired by each camera corresponds to a different video data output interface.

[0092] Step 302, cache the video data corresponding to each camera after being distinguished and marked.

[0093] By collecting the output correspondence and the distinguishing identification information of each camera, the video data corresponding to each camera is distinguished and marked, so as to facilitate the subsequent identification and processing of the video data corresponding to each camera.

[0094] Continue to refer Figure 4 , Figure 4 yes Figure 2 The flowchart of a specific embodiment of step 202 shown includes the following steps:

[0095] Step 401, performing image segmentation processing on the video data corresponding to each camera head according to preset frame processing parameters, wherein the frame processing parameters include a frame processing start time point, a frame processing time interval, and a frame processing end time point;

[0096] Step 402, performing a distinguishing mark process on all the images obtained by segmentation according to the distinguishing mark of the video data and the acquisition time of the video data, and obtaining distinguishing identification information corresponding to all the images;

[0097] Step 403, based on the distinguishing identification information corresponding to all the pictures, the segmented pictures of the same camera are sorted and organized according to the acquisition time, so as to obtain the sorted and organized video picture set corresponding to each camera.

[0098] Through the preset frame processing parameters, the video data corresponding to each camera is segmented and processed, and the segmented pictures of the same camera are sorted and organized according to the acquisition time to obtain the video picture set corresponding to each camera, so as to facilitate the subsequent encoding, decoding and compression processing of the video picture set corresponding to each camera.

[0099] Continue to refer Figure 5 , Figure 5 yes Figure 4 The flowchart of a specific embodiment of step 401 shown includes the following steps:

[0100] Step 501, identifying the frame processing start time point, frame processing time interval and frame processing end time point in the frame processing parameters;

[0101] Step 502, using the same frame division time interval for the video data corresponding to each camera, and performing image segmentation processing from the same frame division processing start time point;

[0102] Step 503, until the frame processing end time point is reached, the frame processing of the video data corresponding to each eye camera is stopped to obtain the segmented pictures corresponding to each eye camera.

[0103] Specifically, by using the same frame time interval for the video data corresponding to each camera, performing image segmentation processing from the same frame processing start time point, and stopping the frame processing at the same frame processing end time point, the segmented images corresponding to different cameras obtained have the same acquisition time point, which is convenient for subsequent alternating compression processing according to the acquisition time point.

[0104] Continue to refer Figure 6 , Figure 6 yes Figure 4 The flowchart of a specific embodiment of step 402 shown includes the following steps:

[0105] Step 601, obtaining the distinguishing identification information of each camera according to the distinguishing mark of the video data;

[0106] Step 602: Based on the distinguishing identification information of each camera, all the segmented images are marked with a first distinguishing field;

[0107] Continuing with the binocular camera as an example, assuming that the distinguishing identification information of the left camera and the right camera are left and right respectively, all the segmented pictures include the segmented pictures corresponding to the left camera and the right camera respectively. For all segmented pictures corresponding to the left camera, left is set as the first distinguishing field, and for all segmented pictures corresponding to the right camera, right is set as the first distinguishing field.

[0108] Step 603, based on the collection time points corresponding to all the segmented images, on the basis of the first distinguishing field marking process, a second distinguishing field marking process is performed on all the segmented images by using a field splicing method;

[0109] Continuing with the example of the embodiment in step 602, at this time, for all the segmented pictures corresponding to the left-eye camera, left is set as the first distinguishing field, and for all the segmented pictures corresponding to the right-eye camera, right is set as the first distinguishing field. Assuming that there are 10 segmented pictures corresponding to the left-eye camera and 10 segmented pictures corresponding to the right-eye camera, since the segmentation time control was performed during the previous segmentation process, the time of each of the 20 segmented pictures is consistent, and there are a total of 10 different acquisition time points. Assuming that the 10 different acquisition time points are time1, time2, time3, time4, time5, time6, time7, time8, time9 and time10, based on the acquisition time points corresponding to all the segmented pictures, on the basis of the first distinguishing field marking process, the field splicing method is adopted to mark all the segmented pictures. The split pictures are processed by the second distinguishing field marking, that is, the split pictures whose distinguishing identification information corresponding to the left camera is left_time1, left_time2, left_time3, left_time4, left_time5, left_time6, left_time7, left_time8, left_time9 and left_time10, and the split pictures whose distinguishing identification information corresponding to the right camera is right_time1, right_time2, right_time3, right_time4, right_time5, right_time6, right_time7, right_time8, right_time9 and right_time10.

[0110] Step 604, obtaining the distinguishing identification information corresponding to all the pictures formed by splicing the distinguishing identification information of each camera at the acquisition time point.

[0111] By combining the distinguishing identification information of each camera in the multi-camera and the acquisition time points corresponding to all the segmented images, the distinguishing identification information of the segmented images is generated, so that the segmented images at the same acquisition time point can be quickly processed according to the subsequent processing requirements, and the segmented images with the same purpose can be quickly processed.

[0112] Continue to refer Figure 7 , Figure 7 yes Figure 2 The flowchart of a specific embodiment of step 203 shown includes the following steps:

[0113] Step 701, calculating the motion vector and motion residual between two consecutive pictures taken by the same camera according to the acquisition time of the pictures in the video picture set;

[0114] In step 403, the segmented pictures of the same camera have been sorted and organized according to the acquisition time based on the distinguishing identification information corresponding to all the pictures, and the sorted video picture set corresponding to each camera has been obtained. Therefore, the segmented pictures in the same video picture set correspond to the same camera. Therefore, according to the acquisition time of the pictures in the video picture set, the motion vector and motion residual between two consecutive pictures under the same camera are calculated.

[0115] Specifically, continuing to take the above-mentioned binocular camera as an example, at this time, based on the binocular camera, two video picture sets are generated accordingly, namely, the video picture set corresponding to the left camera and the video picture set corresponding to the right camera. Assume that there are 10 segmented pictures in the video picture set corresponding to the left camera, that is, the segmented pictures whose distinguishing identification information is left_time1, left_time2, left_time3, left_time4, left_time5, left_time6, left_time7, left_time8, left_time9 and left_time10. The motion vector and motion residual between the pictures left_time1 and left_time2, the motion vector and motion residual between the pictures left_time2 and left_time3, the motion vector and motion residual between the pictures left_time3 and left_time4, and so on are calculated successively. Finally, the motion vector and motion residual between the pictures left_time9 and left_time10 are calculated, wherein the motion vector and the motion residual can be understood as the shooting error caused by the movement of the target object in the video during shooting.

[0116] Step 702, calculating the disparity vectors and disparity residuals of different cameras at two consecutive acquisition time points according to the acquisition time of the pictures in the video picture set;

[0117] Correspondingly, assuming that there are 10 segmented pictures in the video picture set corresponding to the right-eye camera, that is, the segmented pictures whose distinguishing identification information is right_time1, right_time2, right_time3, right_time4, right_time5, right_time6, right_time7, right_time8, right_time9 and right_time10, the motion vector and motion residual between the pictures right_time1 and right_time2, the motion vector and motion residual between the pictures right_time2 and right_time3, the motion vector and motion residual between the pictures right_time3 and right_time4, and so on, are calculated in sequence, and finally the motion vector and motion residual between the pictures right_time9 and right_time10 are calculated;

[0118] Then, the motion vector and motion residual between the pictures left_time1 and left_time2, as well as the motion vector and motion residual between the pictures right_time1 and right_time2 are used to jointly calculate the disparity vector and disparity residual of different eyepiece cameras at two consecutive acquisition time points, wherein the disparity vector and the disparity residual can be understood as the shooting error between multiple eyes caused by the different eyepiece positions themselves.

[0119] Step 703, using the motion vector, motion residual, disparity vector and disparity residual as coding supplementary parameters, using the distinguishing identification information corresponding to all pictures as coding control parameters, and using entropy coding to encode the pictures in the video picture set;

[0120] The entropy coding method is a coding method that uses the statistical characteristics of the information source to perform code rate compression, aiming to make the average code length after coding approach the entropy limit, thereby eliminating redundancy between data.

[0121] Step 704: decode the encoding processing result, and serialize and sort all decoded pictures obtained by decoding according to the acquisition time point to obtain the decoded picture set.

[0122] Continue to refer Figure 8 , Figure 8 yes Figure 7 The flowchart of a specific embodiment of step 703 shown includes the following steps:

[0123] Step 801, identifying the encoding supplementary parameters corresponding to each picture;

[0124] Step 802, identifying the acquisition time point corresponding to each picture according to the encoding control parameter;

[0125] Step 803, according to the encoding supplementary parameters and the acquisition time point, the images corresponding to different cameras at the same acquisition time point are encoded and synthesized to obtain the encoding synthesis results of the corresponding images at each acquisition time point.

[0126] It should be understood that according to the encoding supplementary parameters and the acquisition time point, the pictures corresponding to different eye cameras at the same acquisition time point are encoded and synthesized. Continuing with the above-mentioned binocular camera as an example, it is assumed that the motion vector and motion residual between the pictures left_time1 and left_time2, the motion vector and motion residual between the pictures right_time1 and right_time2, and the disparity vector and disparity residual between the acquisition time points time1 and time2 are calculated. During encoding, the motion vector and motion residual between the pictures left_time1 and left_time2, the motion vector and motion residual between the pictures right_time1 and right_time2, and the disparity vector and disparity residual between time1 and time2 corresponding to the acquisition time point time2 are used as encoding supplementary parameters to complete the encoding processing of the corresponding decoded picture at the acquisition time point time2, and so on, to obtain the encoding processing results of the decoded pictures corresponding to each acquisition time point.

[0127] By performing encoding synthesis processing on the pictures corresponding to different cameras at the same acquisition time point according to the encoding supplementary parameters and the acquisition time point during encoding, that is, introducing the encoding supplementary parameters for encoding synthesis during encoding, it is ensured that during encoding, redundant features between different segmented pictures at the same acquisition time point are first eliminated by an alternating compression method, and then synthesis is performed, so that the decoded picture finally outputted is generated with fewer redundant features.

[0128] Continue to refer Fig. 9 , Fig. 9 yes Figure 7 The flowchart of a specific embodiment of step 704 shown includes the following steps:

[0129] Step 901, performing decoding processing according to the coding synthesis result of the corresponding picture at each acquisition time point to obtain a unique decoded picture at each acquisition time point;

[0130] Specifically, assuming that the multi-eye camera is a binocular camera, that is, during the encoding synthesis, the two segmented pictures corresponding to the binocular at the same acquisition time point are encoded and synthesized, so that only a unique decoded picture is output; assuming that the multi-eye camera is a trinocular camera, that is, during the encoding synthesis, the three segmented pictures corresponding to the trinocular at the same acquisition time point are encoded and synthesized, so that only a unique decoded picture is output.

[0131] Step 902: for each unique decoded picture at each acquisition time point, a picture set is constructed in the order of the acquisition time points to generate the decoded picture set.

[0132] This application obtains the video data collected by each camera in the multi-camera; processes the video data into images to obtain the video picture set corresponding to each camera; uses an alternating compression method to encode and decode the video picture set corresponding to each camera to obtain a decoded picture set; and performs a collected video synthesis output according to the decoded picture set. Since the corresponding decoded pictures at the same acquisition time are encoded and decoded by the pictures taken by the multi-camera in an alternating compression method, each decoded picture is synthesized by encoding, decoding and compression of multiple camera pictures, which realizes that a decoded picture has more detailed picture information, ensures that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio.

[0133] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0134] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0135] In the embodiment of the present application, the video data collected by each camera in the multi-camera is obtained; the video data is processed into images to obtain the video picture set corresponding to each camera; the video picture set corresponding to each camera is encoded and decoded using an alternating compression method to obtain a decoded picture set; and the collected video is synthesized and output according to the decoded picture set. Since the corresponding decoded pictures at the same acquisition time are encoded and decoded by the pictures taken by the multi-cameras in an alternating compression method, each decoded picture is synthesized by encoding, decoding and compression of multiple camera pictures, so that a decoded picture has more detailed picture information, ensuring that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio.

[0136] Further references Fig.10 , as a response to the above Figure 2 The present application provides an embodiment of a video processing device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0137] like Fig.10 As shown, the video processing device 10 described in this embodiment includes: a video data acquisition module 10a, a picture processing module 10b, a codec processing module 10c and a video synthesis output module 10d. Among them:

[0138] The video data acquisition module 10a is used to acquire video data collected by a multi-camera, wherein the video data includes video data corresponding to each camera, and the multi-camera includes two or more cameras;

[0139] A picture processing module 10b is used to process the video data into pictures to obtain a video picture set corresponding to each camera;

[0140] The encoding and decoding processing module 10c is used to perform encoding and decoding processing on the video picture set corresponding to each camera respectively by using an alternating compression method to obtain a decoded picture set;

[0141] The video synthesis output module 10d is used to synthesize and output the captured video according to the decoded picture set.

[0142] This application obtains the video data collected by each camera in the multi-camera; processes the video data into images to obtain the video picture set corresponding to each camera; uses an alternating compression method to encode and decode the video picture set corresponding to each camera to obtain a decoded picture set; and performs a collected video synthesis output according to the decoded picture set. Since the corresponding decoded pictures at the same acquisition time are encoded and decoded by the pictures taken by the multi-camera in an alternating compression method, each decoded picture is synthesized by encoding, decoding and compression of multiple camera pictures, which realizes that a decoded picture has more detailed picture information, ensures that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0144] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0145] To solve the above technical problems, the present application also provides a computer device. Fig.11 , Fig.11 This is a basic structural block diagram of the computer device in this embodiment.

[0146] The computer device 11 includes a memory 11a, a processor 11b, and a network interface 11c which are interconnected through a system bus. It should be noted that: Fig.11Only a computer device 11 having a component memory 11a, a processor 11b, and a network interface 11c is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0147] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0148] The memory 11a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11a can be an internal storage unit of the computer device 11, such as a hard disk or memory of the computer device 11. In other embodiments, the memory 11a can also be an external storage device of the computer device 11, such as a plug-in hard disk equipped on the computer device 11, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 11a can also include both the internal storage unit of the computer device 11 and its external storage device. In this embodiment, the memory 11a is generally used to store the operating system and various application software installed on the computer device 11, such as a computer-readable instruction of a video processing method, etc. In addition, the memory 11a can also be used to temporarily store various types of data that have been output or are to be output.

[0149] The processor 11b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 11b is generally used to control the overall operation of the computer device 11. In this embodiment, the processor 11b is used to run the computer-readable instructions stored in the memory 11a or process data, such as computer-readable instructions for running the video processing method.

[0150] The network interface 11c may include a wireless network interface or a wired network interface. The network interface 11c is generally used to establish a communication connection between the computer device 11 and other electronic devices.

[0151] The computer device proposed in this embodiment belongs to the field of research and development design and image processing technology, and is applied to video synthesis scenarios under multi-cameras. This application obtains the video data collected by each camera in the multi-camera; processes the video data into images to obtain the video picture set corresponding to each camera; uses an alternating compression method to encode and decode the video picture set corresponding to each camera to obtain a decoded picture set; and performs the collected video synthesis output according to the decoded picture set. Since the decoded pictures corresponding to the same acquisition time are encoded and decoded by the pictures taken by the multi-cameras in an alternating compression method, each decoded picture is synthesized by encoding, decoding and compression of multiple camera pictures, so that a decoded picture has more detailed picture information, ensuring that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio.

[0152] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the video processing method as described above.

[0153] The computer-readable storage medium proposed in this embodiment belongs to the field of research and development design and image processing technology, and is applied to video synthesis scenarios under multi-cameras. This application obtains the video data collected by each camera in the multi-camera; processes the video data into images to obtain the video picture set corresponding to each camera; uses an alternating compression method to encode and decode the video picture set corresponding to each camera to obtain a decoded picture set; and performs a captured video synthesis output according to the decoded picture set. Since the decoded pictures corresponding to the same acquisition time are encoded and decoded by the pictures taken by the multi-cameras in an alternating compression method, each decoded picture is synthesized by encoding, decoding and compression of multiple camera pictures, so that a decoded picture has more detailed picture information, ensuring that the synthesized video is more detailed and clear, and at the same time, it can also improve the overall compression efficiency and compression ratio.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0155] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A video processing method, characterized in that: The steps include: Acquire video data collected by a multi-camera, wherein the video data includes video data corresponding to each camera, and the multi-camera includes two or more cameras; Processing the video data into images to obtain a video image set corresponding to each camera; The video picture set corresponding to each camera is encoded and decoded by using an alternating compression method to obtain a decoded picture set; According to the decoded picture set, the captured video is synthesized and outputted.

2. The video processing method according to claim 1, characterized in that: After executing the step of acquiring the video data collected by the multi-camera, the method further includes: According to the corresponding relationship between the acquisition output and the distinguishing identification information of each camera, the video data corresponding to each camera is distinguishably marked; Cache the video data corresponding to each camera after being distinguished and marked.

3. The video processing method according to claim 1, characterized in that: The step of processing the video data into images to obtain a video image set corresponding to each camera specifically includes: According to the preset frame processing parameters, the video data corresponding to each camera is subjected to image segmentation processing, wherein the frame processing parameters include the frame processing start time point, the frame processing time interval and the frame processing end time point; According to the distinguishing mark of the video data and the acquisition time of the video data, distinguishing mark processing is performed on all the images obtained by segmentation to obtain distinguishing identification information corresponding to all the images respectively; Based on the distinguishing identification information corresponding to all the images, the segmented images of the same camera are sorted and organized according to the acquisition time, and the sorted and organized video image set corresponding to each camera is obtained.

4. The video processing method according to claim 3, characterized in that: The step of performing image segmentation processing on the video data corresponding to each camera head according to the preset frame processing parameters specifically includes: Identify the frame processing start time point, frame processing time interval and frame processing end time point in the frame processing parameters; The same frame splitting time interval is used for the video data corresponding to each camera, and the image segmentation processing is performed from the same frame splitting processing starting time point; Until the frame processing end time point is reached, the frame processing of the video data corresponding to each camera is stopped to obtain the segmented pictures corresponding to each camera.

5. The video processing method according to claim 3, characterized in that: The step of performing distinguishing mark processing on all the images obtained by segmentation according to the distinguishing mark of the video data and the acquisition time of the video data to obtain distinguishing identification information corresponding to all the images specifically includes: According to the distinguishing mark of the video data, obtain the distinguishing identification information of each camera; Based on the distinguishing identification information of each camera, all segmented images are marked with the first distinguishing field; Based on the collection time points corresponding to all the segmented images, and on the basis of the first distinguishing field marking process, a second distinguishing field marking process is performed on all the segmented images by using a field splicing method; Obtain the distinguishing identification information corresponding to all the images formed by splicing the distinguishing identification information of each camera at the acquisition time point.

6. The video processing method according to claim 1, characterized in that: The step of encoding and decoding the video picture sets corresponding to each camera head by using an alternating compression method to obtain a decoded picture set specifically includes: According to the acquisition time of the pictures in the video picture set, the motion vector and motion residual between two consecutive pictures under the same camera are calculated; According to the acquisition time of the pictures in the video picture set, the disparity vector and disparity residual of different cameras at two consecutive acquisition time points are calculated; The motion vector, motion residual, disparity vector and disparity residual are used as coding supplementary parameters, and the distinguishing identification information corresponding to all pictures is used as coding control parameters, and the pictures in the video picture set are encoded by adopting an entropy coding method; The encoding processing result is decoded, and all decoded pictures obtained by decoding are serialized and sorted according to the acquisition time point to obtain the decoded picture set.

7. The video processing method according to claim 6, characterized in that: The step of using the motion vector, motion residual, disparity vector and disparity residual as encoding supplementary parameters, using the distinguishing identification information corresponding to all pictures as encoding control parameters, and using entropy encoding to encode pictures in the video picture set specifically includes: Identify the encoding supplementary parameters corresponding to each picture; According to the encoding control parameters, identifying the acquisition time point corresponding to each picture; According to the encoding supplementary parameters and the acquisition time point, the images corresponding to different cameras at the same acquisition time point are encoded and synthesized to obtain the encoding synthesis results of the corresponding images at each acquisition time point; The step of decoding the encoding processing result and serializing all decoded pictures obtained by decoding according to the acquisition time point to obtain the decoded picture set specifically includes: According to the encoding synthesis result of the corresponding picture at each acquisition time point, decoding processing is performed to obtain a unique decoded picture at each acquisition time point; For each unique decoded picture at each acquisition time point, a picture set is constructed in the order of the acquisition time points to generate the decoded picture set.

8. A video processing device, characterized in that: include: A video data acquisition module is used to acquire video data collected by a multi-eye camera, wherein the video data includes video data corresponding to each eye camera, and the multi-eye camera includes two or more eyes; A picture processing module, used to process the video data into pictures to obtain a video picture set corresponding to each camera; The encoding and decoding processing module is used to perform encoding and decoding processing on the video picture set corresponding to each camera respectively by using an alternating compression method to obtain a decoded picture set; The video synthesis output module is used to synthesize and output the captured video according to the decoded picture set.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the video processing method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the video processing method according to any one of claims 1 to 7 are implemented.

Images