Video processing method, device, server and terminal

Abstract

The present invention provides a video processing method, device, server and terminal, the method comprising: receiving multiple video data uploaded by a terminal and shot from different viewing angles of the same object; grading the quality of the multiple video data to obtain a video code stream of at least one quality level; sending the video code stream of at least one quality level and a trained neural network model corresponding to the video code stream to the terminal. The solution of the present invention not only reduces the transmitted bit stream and increases the utilization rate of the bandwidth, but also improves the quality of low-definition video, thereby improving the overall video quality, meeting the needs of users for different forms of viewing, and improving user experience.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video processing method, device, server and terminal. Background Art

[0002] The multi-view video application scenario describes the use of multiple cameras to shoot the same activity event, thereby providing online users with videos from different perspectives. Users can trigger the terminal device to continuously change the perspective and find the viewing angle that the user most wants, satisfying the user's needs to watch different perspectives at the same time and switch perspectives autonomously and continuously.

[0003] The existing technical solution is to encode the videos shot by multiple cameras independently and upload them to the server. The server distributes the content corresponding to the user's requested perspective through CDN (Content Distribution Network), or distributes the content of all perspectives to the user. The former causes a large switching delay, and the latter brings pressure on the bandwidth due to the large amount of transmitted data.

[0004] In the prior art, when encoding and transmitting the high-definition video of the current viewing angle and the low-definition video of all viewing angles, when the user switches the viewing angle, the low-definition picture of the current viewing angle is first displayed, and then the high-definition video is obtained from the server and switched to. The quality difference between the high-definition video and the low-definition video brings a bad experience to the user, and the user cannot watch the high-definition videos of different viewing angles at the same time, the content quality is poor, which limits the diversified experience of the user, and the network transmission delay is large. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to provide a video processing method, device, server and terminal, which can reduce the bit stream of transmission, increase the utilization rate of bandwidth, improve the quality of low-definition video, thereby improving the overall video quality, meeting the needs of users for different forms of viewing, and improving user experience.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] The present invention provides a video processing method, which is applied to a server, and the method comprises:

[0008] Receive multiple video data uploaded by the terminal, shot from different perspectives of the same object;

[0009] Performing quality classification on multiple channels of video data to obtain a video bitstream of at least one quality level;

[0010] The video bitstream of at least one quality level and the trained neural network model corresponding to the video bitstream are sent to the terminal.

[0011] Optionally, performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes:

[0012] According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and

[0013] Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

[0014] Optionally, the first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

[0015] Optionally, the neural network model is trained by the following process:

[0016] Inputting the picture group data of the first quality level and the picture group data of the second quality level obtained by quality grading at least one video data into the neural network;

[0017] The first quality level group of pictures data and the second quality level group of pictures data are respectively compared with a reference video frame to obtain a comparison result;

[0018] The trained network model is outputted according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level.

[0019] Optionally, outputting a trained network model according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level includes:

[0020] If the group of pictures data of the first quality level and the group of pictures data of the second quality level are cross-transmitted at the same time interval, outputting a first neural network model according to the comparison result;

[0021] If the picture group data of the first quality level is adjacent to the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then the second neural network model is output according to the comparison result.

[0022] The present invention provides a video processing method, which is applied to a terminal, and the method comprises:

[0023] Shoot the same object from different perspectives to obtain multiple video data;

[0024] Performing quality classification on multiple channels of video data to obtain a video bitstream of at least one quality level;

[0025] The video code stream of at least one quality level is input into a trained neural network model corresponding to the video code stream received from a server for processing to obtain a processed video stream.

[0026] Optionally, performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes:

[0027] According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and

[0028] Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

[0029] Optionally, the first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

[0030] Optionally, the video stream of at least one quality level is input into a trained neural network model corresponding to the video stream and received from a server for processing to obtain a processed video stream, including:

[0031] The picture group data of the first quality level and the picture group data of the second quality level are input into the trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream.

[0032] Optionally, the arrangement order of the group of pictures data of the first quality level and the group of pictures data of the second quality level is input into a trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream, including:

[0033] If the picture group data of the first quality level and the picture group data of the second quality level are arranged crosswise at the same time interval, they are input into the first neural network model for processing to obtain a processed video stream;

[0034] If the picture group data of the first quality level are adjacent frames of the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then they are input into the second neural network model for processing to obtain a processed video stream.

[0035] The present invention provides a video processing device, which is applied to a server, and the device comprises:

[0036] A transceiver module, used for receiving multi-channel video data uploaded by a terminal, shot from different perspectives of the same object;

[0037] A processing module, used for performing quality classification on multiple channels of video data to obtain a video bit stream of at least one quality level;

[0038] The transceiver module is also used to send the video code stream of at least one quality level and the trained neural network model corresponding to the video code stream to the terminal.

[0039] The present invention provides a server, comprising:

[0040] A transceiver, used for receiving multi-channel video data uploaded by a terminal, shot from different perspectives of the same object;

[0041] A processor, used for performing quality classification on multiple channels of video data to obtain a video code stream of at least one quality level;

[0042] The transceiver is also used to send the video code stream of at least one quality level and the trained neural network model corresponding to the video code stream to the terminal.

[0043] The present invention provides a video processing device, applied to a terminal, the device comprising:

[0044] An acquisition module is used to shoot the same object from different viewing angles to obtain multiple video data;

[0045] The processing module is used to perform quality classification on multiple video data to obtain a video stream of at least one quality level; the video stream of at least one quality level is input into a trained neural network model corresponding to the video stream received from a server for processing to obtain a processed video stream.

[0046] The present invention also provides a terminal, comprising:

[0047] A collector is used to shoot the same object from different perspectives to obtain multiple video data;

[0048] The processor is used to perform quality classification on multiple channels of video data to obtain a video stream of at least one quality level; the video stream of at least one quality level is input into a trained neural network model corresponding to the video stream received from a server for processing to obtain a processed video stream.

[0049] The present invention provides a processing device, comprising: a processor and a memory storing a computer program, wherein the computer program executes the above method when being executed by the processor.

[0050] The present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above method.

[0051] The above solution of the present invention includes at least the following beneficial effects:

[0052] By receiving multiple video data uploaded by a terminal and shot from different angles of the same object; performing quality classification on the multiple video data to obtain a video code stream of at least one quality level; and sending the video code stream of at least one quality level and a trained neural network model corresponding to the video code stream to the terminal. The solution of the present invention not only reduces the transmitted bit stream and increases the utilization rate of bandwidth, but also improves the quality of low-definition video, thereby improving the overall video quality, meeting the needs of users for different forms of viewing, and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of a video processing method applied to a server according to an embodiment of the present invention;

[0054] Figure 2 is a schematic diagram of the overall flow of the video processing method according to an embodiment of the present invention;

[0055] Figure 3 is a schematic diagram of a terminal performing multi-viewing scene shooting according to an embodiment of the present invention;

[0056] Figure 4 is a schematic diagram of model training preprocessing in a video processing method according to an embodiment of the present invention;

[0057] Figure 5 is a schematic diagram of a training output model in a video processing method according to an embodiment of the present invention;

[0058] Figure 6 is a schematic diagram of a first neural network model in a video processing method according to an embodiment of the present invention;

[0059] Figure 7 is a schematic diagram of a second neural network model in the video processing method of an embodiment of the present invention;

[0060] Figure 8 is a flow chart of a video processing method applied to a terminal according to an embodiment of the present invention;

[0061] Fig. 9 is a module block diagram of a video processing device applied to a server according to an embodiment of the present invention;

[0062] Fig.10 It is a module block diagram of a video processing device applied to a terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0064] like Figure 1 As shown, an embodiment of the present invention provides a video processing method, which is applied to a server, and the method includes:

[0065] Step 11, receiving multiple video data uploaded by a terminal, which are taken from different viewing angles of the same object;

[0066] Step 12, performing quality classification on the multiple channels of video data to obtain a video bit stream of at least one quality level;

[0067] Step 13: Send the video stream of at least one quality level and the trained neural network model corresponding to the video stream to the terminal.

[0068] This embodiment receives multiple video data uploaded by a terminal that are shot from different angles and are of the same object; performs quality grading on the multiple video data to obtain a video stream of at least one quality level; and sends the video stream of at least one quality level and a trained neural network model corresponding to the video stream to the terminal. The solution of the present invention not only reduces the bit stream of transmission and increases the utilization of bandwidth, but also improves the quality of low-definition video, thereby improving the overall video quality, meeting the needs of users for different forms of viewing, and improving user experience.

[0069] like Figure 2 As shown, this embodiment is a method applied to the server side, which transmits multi-channel video data shot from multiple perspectives on-site uploaded by the receiving terminal losslessly and without compression to the server, performs quality graded encoding on the data to obtain a video stream of at least one quality level, and sends the video stream and its trained neural network model to the terminal, thereby reducing the transmitted bit stream and increasing bandwidth utilization.

[0070] like Figure 3 As shown, the multi-channel video data in step 11 is obtained by the terminal setting the camera array into a ring shape, each camera focuses on the same object to shoot, the system collects multi-channel video and audio data and numbers each video, sets the multi-channel video data synchronization and transmits it to the server.

[0071] In an optional embodiment of the present invention, step 12 includes:

[0072] Step 121, dividing each channel of video data into a plurality of picture group data of a first quality level according to a first quality threshold set for each channel of video data, and

[0073] Step 122: divide each channel of video data into a plurality of GOP data of a second quality level according to a second quality threshold set for each channel of video data.

[0074] It should be noted that the first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

[0075] In this embodiment, the received multiple channels of video data are subjected to quality grading to obtain at least one video code stream, wherein the quality grading is performed based on setting quality thresholds of different sizes for each channel of video data, and encoding into a version of the group of pictures data of a first quality level and a version of the group of pictures data of a second quality level. The quality threshold of the group of pictures data of the first quality level is defined as 0 to 30, and the quality threshold of the group of pictures data of the second quality level is defined as 31 to 50.

[0076] In an optional embodiment of the present invention, the neural network model in step 13 is trained by the following process:

[0077] Step 131, inputting the picture group data of the first quality level and the picture group data of the second quality level obtained by quality grading at least one video data into a neural network;

[0078] Step 132, the group of pictures data of the first quality level and the group of pictures data of the second quality level are respectively compared with a reference video frame to obtain a comparison result;

[0079] Step 133: outputting the trained network model according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level.

[0080] like Figure 4As shown, in this embodiment, step 131 is a preprocessing of the neural network training, and video frame data of the same time at different viewing angles in multiple video data are respectively taken out in units of time. Figure 3 The video data of view 1, view 2, and view 3 are divided into picture group data of the first quality level and picture group data of the second quality level according to time units and input into the neural network. The frame at time Tn-1 in the neural network is a picture group data frame of the second quality level, and the frame at time Tn is a picture group data frame of the first quality level and is a neighboring frame.

[0081] like Figure 5 As shown, in this embodiment, step 132 can use the frames of lossless video as reference video frames, the picture group data frames of the first quality level (i.e., neighboring frames) as auxiliary frames, and the picture group data frames of the second quality level as frames to be improved to input into the neural network for comparison, and iterative training and testing to obtain comparison results.

[0082] In an optional embodiment of the present invention, step 133 includes:

[0083] Step 1331, if the picture group data of the first quality level and the picture group data of the second quality level are cross-transmitted at the same time interval, outputting a first neural network model according to the comparison result;

[0084] Step 1332: if the picture group data of the first quality level is adjacent to the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then output the second neural network model according to the comparison result.

[0085] In this embodiment, according to the video number sequence, the video data frames of each view are encoded and output respectively, so as to ensure that each segment of the second quality level group of pictures video data must have an adjacent first quality level group of pictures video data frame (i.e., neighboring frame). Therefore, there will be two network models, which are described below:

[0086] like Figure 6 As shown, the first neural network model is that in step 1331, the picture group data of the first quality level and the picture group data of the second quality level are cross-transmitted at the same interval, and the minimum unit is a GOP (picture group; Group of Pictures);

[0087] The second neural network model is that in step 1332, the picture group data of the first quality level is adjacent to the picture group data of the second quality level, the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information of part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level.

[0088] like Figure 7 As shown, for each viewing angle of the video data, a segment of group of pictures data of the first quality level is first uploaded, and the minimum time unit is a GOP, and then two segments of group of pictures data of the second quality level with the same viewing angle are transmitted.

[0089] The present invention also provides a video processing method, applied to a terminal, the method comprising:

[0090] Step 81, photographing the same object from different viewing angles to obtain multiple channels of video data;

[0091] Step 82, performing quality classification on the multiple channels of video data to obtain a video bit stream of at least one quality level;

[0092] Step 83: Input the video code stream of at least one quality level into the trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream.

[0093] This embodiment is a method applied to the terminal side, which obtains multiple channels of video data by shooting the same object from multiple perspectives, and then performs quality classification to obtain at least one level of video code stream, which is input to the server and receives the corresponding trained neural network model for processing, and finally obtains the processed video stream. The quality of low-definition video is improved, thereby improving the quality of the entire video, meeting the needs of users for different forms of viewing, and improving the user experience.

[0094] In an optional embodiment of the present invention, step 82 includes:

[0095] According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and

[0096] Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

[0097] The first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

[0098] In an optional embodiment of the present invention, step 83 includes:

[0099] Step 831, according to the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level, input them into the trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream.

[0100] In an optional embodiment of the present invention, step 831 includes:

[0101] If the picture group data of the first quality level and the picture group data of the second quality level are arranged crosswise at the same time interval, they are input into the first neural network model for processing to obtain a processed video stream;

[0102] If the picture group data of the first quality level are adjacent frames of the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then they are input into the second neural network model for processing to obtain a processed video stream.

[0103] In this embodiment, the two models trained and output by the server side and the quality cross-encoded video are associated together and sent to the terminal; after receiving the video stream, the terminal side detects the low-quality frames, extracts the first quality level picture group data of the nearest neighbor, and then uses the same neural network structure as the server side, with the second quality level picture group data frame as the first input and the first quality level picture group data frame as the second input, and calculates with the received model to obtain this so-called high-true frame, which corresponds one-to-one with the second quality level picture group data frame, and finally enhances the quality of the entire video stream.

[0104] Ultimately, the terminal should be able to provide multiple viewing angles for simultaneous viewing, any viewing angle can be switched continuously clockwise and counterclockwise with continuous images, and viewpoints can be switched directly at any time. One or more of these can be selected to display video content according to user needs.

[0105] In a specific embodiment, Figure 2 As shown, assuming that there are 48 cameras in a multi-view scene, the GOP value of each video stream is set to 16 for encoding, the encoding quality level of the first quality level group of pictures data is set to 22, and the encoding quality level of the second quality level group of pictures data is set to 36. The first GOP of each video stream data is the group of pictures data of the first quality level, the second and third GOPs are the group of pictures data of the second quality level, the fourth GOP is the group of pictures data of the first quality level, the fifth and sixth GOPs are the group of pictures data of the second quality level, and so on for transmitting video. Low-quality encoding saves a lot of bit streams.

[0106] The server also encodes with the same quality level value, and then uses ARCNN (Artifacts Reduction Convolutional Neural Network) neural network training to output two models and transmit them to the terminal.

[0107] At the terminal, the second GOP video stream uses the last frame in the first GOP as an auxiliary frame and uses a model to restore the quality;

[0108] The third GOP video stream uses the first frame in the fourth GOP as an auxiliary frame and uses another model to restore the quality. It is input into the ARCNN neural network and finally the overall video quality is enhanced to improve the user experience.

[0109] The embodiment of the present invention improves the quality of low-definition video by processing multi-data video data obtained by multi-view shooting, thereby improving the quality of the entire video, meeting the needs of users for different forms of viewing, and improving the technical effect of user experience. It should be noted here that the principle of the method for video processing by the terminal is the same as that of the method for video processing by the server, which will not be repeated here.

[0110] like Fig. 9 As shown, an embodiment of the present invention further provides a video processing device 90, which is applied to a server, and the device includes:

[0111] The transceiver module 91 is used to receive multiple video data uploaded by the terminal, which are taken from different perspectives of the same object;

[0112] A processing module 92, configured to perform quality classification on multiple channels of video data to obtain a video bit stream of at least one quality level;

[0113] The transceiver module 91 is also used to send the video code stream of at least one quality level and the trained neural network model corresponding to the video code stream to the terminal.

[0114] Optionally, performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes:

[0115] According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and

[0116] Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

[0117] Optionally, the first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

[0118] Optionally, the neural network model is trained by the following process:

[0119] Inputting the picture group data of the first quality level and the picture group data of the second quality level obtained by quality grading at least one video data into the neural network;

[0120] The first quality level group of pictures data and the second quality level group of pictures data are respectively compared with a reference video frame to obtain a comparison result;

[0121] The trained network model is outputted according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level.

[0122] Optionally, outputting a trained network model according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level includes:

[0123] If the group of pictures data of the first quality level and the group of pictures data of the second quality level are cross-transmitted at the same time interval, outputting a first neural network model according to the comparison result;

[0124] If the picture group data of the first quality level is adjacent to the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then the second neural network model is output according to the comparison result.

[0125] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0126] An embodiment of the present invention further provides a server, comprising:

[0127] A transceiver, used for receiving multi-channel video data uploaded by a terminal, shot from different perspectives of the same object;

[0128] A processor, used for performing quality classification on multiple channels of video data to obtain a video code stream of at least one quality level;

[0129] The transceiver is also used to send the video code stream of at least one quality level and the trained neural network model corresponding to the video code stream to the terminal.

[0130] Optionally, performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes:

[0131] According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and

[0132] Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

[0133] Optionally, the first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

[0134] Optionally, the neural network model is trained by the following process:

[0135] Inputting the picture group data of the first quality level and the picture group data of the second quality level obtained by quality grading at least one video data into the neural network;

[0136] The first quality level group of pictures data and the second quality level group of pictures data are respectively compared with a reference video frame to obtain a comparison result;

[0137] The trained network model is outputted according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level.

[0138] Optionally, outputting a trained network model according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level includes:

[0139] If the group of pictures data of the first quality level and the group of pictures data of the second quality level are cross-transmitted at the same time interval, outputting a first neural network model according to the comparison result;

[0140] If the picture group data of the first quality level is adjacent to the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then the second neural network model is output according to the comparison result.

[0141] It should be noted that the server is a server corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the server and can achieve the same technical effects.

[0142] like Fig.10 As shown, an embodiment of the present invention further provides a video processing device 100, which is applied to a terminal, and the device includes:

[0143] An acquisition module 101 is used to shoot the same object from different viewing angles to obtain multiple channels of video data;

[0144] The processing module 102 is used to perform quality classification on multiple channels of video data to obtain a video stream of at least one quality level; the video stream of at least one quality level is input into a trained neural network model corresponding to the video stream received from a server for processing to obtain a processed video stream.

[0145] Optionally, performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes:

[0146] According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and

[0147] Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

[0148] Optionally, the first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

[0149] Optionally, the video code stream of at least one quality level is input into a trained neural network model corresponding to the video code stream and received from a server for processing to obtain a processed video stream, including:

[0150] The picture group data of the first quality level and the picture group data of the second quality level are input into the trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream.

[0151] Optionally, the arrangement order of the group of pictures data of the first quality level and the group of pictures data of the second quality level is input into a trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream, including:

[0152] If the picture group data of the first quality level and the picture group data of the second quality level are arranged crosswise at the same time interval, they are input into the first neural network model for processing to obtain a processed video stream;

[0153] If the picture group data of the first quality level are adjacent frames of the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then they are input into the second neural network model for processing to obtain a processed video stream.

[0154] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0155] An embodiment of the present invention further provides a terminal, including:

[0156] A collector is used to shoot the same object from different perspectives to obtain multiple video data;

[0157] The processor is used to perform quality classification on multiple channels of video data to obtain a video stream of at least one quality level; the video stream of at least one quality level is input into a trained neural network model corresponding to the video stream received from a server for processing to obtain a processed video stream.

[0158] Optionally, performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes:

[0159] According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and

[0160] Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

[0161] Optionally, the first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

[0162] Optionally, the video stream of at least one quality level is input into a trained neural network model corresponding to the video stream and received from a server for processing to obtain a processed video stream, including:

[0163] The picture group data of the first quality level and the picture group data of the second quality level are input into the trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream.

[0164] Optionally, the arrangement order of the group of pictures data of the first quality level and the group of pictures data of the second quality level is input into a trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream, including:

[0165] If the picture group data of the first quality level and the picture group data of the second quality level are arranged crosswise at the same time interval, they are input into the first neural network model for processing to obtain a processed video stream;

[0166] If the picture group data of the first quality level are adjacent frames of the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then they are input into the second neural network model for processing to obtain a processed video stream.

[0167] It should be noted that the terminal is a terminal corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the terminal and can achieve the same technical effects.

[0168] The embodiment of the present invention further provides a processing device, comprising: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0169] The embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above method. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0170] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0171] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0172] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0175] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0176] In addition, it should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0177] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0178] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A video processing method, characterized in that: Applied to a server, the method comprises: Receive multiple video data uploaded by the terminal, shot from different perspectives of the same object; Performing quality classification on multiple channels of video data to obtain a video bitstream of at least one quality level; Sending the video bitstream of at least one quality level and the trained neural network model corresponding to the video bitstream to the terminal; Performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes: According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

2. The video processing method according to claim 1, characterized in that: The first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

3. The video processing method according to claim 1, characterized in that: The neural network model is trained through the following process: Inputting the picture group data of the first quality level and the picture group data of the second quality level obtained by quality grading at least one video data into the neural network; The first quality level group of pictures data and the second quality level group of pictures data are respectively compared with a reference video frame to obtain a comparison result; The trained network model is outputted according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level.

4. The video processing method according to claim 3, characterized in that: Outputting a trained network model according to the comparison result and the arrangement order of the picture group data of the first quality level and the picture group data of the second quality level includes: If the group of pictures data of the first quality level and the group of pictures data of the second quality level are cross-transmitted at the same time interval, outputting a first neural network model according to the comparison result; If the picture group data of the first quality level is adjacent to the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then the second neural network model is output according to the comparison result.

5. A video processing method, characterized in that: Applied to a terminal, the method comprises: Shoot the same object from different perspectives to obtain multiple video data; Performing quality classification on multiple channels of video data to obtain a video bitstream of at least one quality level; Inputting the video stream of at least one quality level into a trained neural network model corresponding to the video stream received from the server for processing to obtain a processed video stream; Performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes: According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

6. The video processing method according to claim 5, characterized in that: The first quality threshold and the second quality threshold are quality quantization parameters, the first quality threshold ranges from 0 to 30, and the second quality threshold ranges from 31 to 50.

7. The video processing method according to claim 5, characterized in that: The video code stream of at least one quality level is input into a trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream, including: The picture group data of the first quality level and the picture group data of the second quality level are input into the trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream.

8. The video processing method according to claim 7, characterized in that: Inputting the picture group data of the first quality level and the picture group data of the second quality level into a trained neural network model corresponding to the video code stream received from the server for processing to obtain a processed video stream, including: If the picture group data of the first quality level and the picture group data of the second quality level are arranged alternately at the same time interval, they are input into the first neural network model for processing to obtain a processed video stream; If the picture group data of the first quality level are adjacent frames of the picture group data of the second quality level, and the time information corresponding to part of the picture group data of the first quality level is before the time information corresponding to the picture group data of the second quality level, and the time information corresponding to another part of the picture group data of the first quality level is after the time information corresponding to the picture group data of the second quality level, then they are input into the second neural network model for processing to obtain a processed video stream.

9. A video processing device, characterized in that: Applied to a server, the device comprises: A transceiver module, used for receiving multi-channel video data uploaded by a terminal, shot from different perspectives of the same object; A processing module, used for performing quality classification on multiple channels of video data to obtain a video bit stream of at least one quality level; The transceiver module is also used to send the video code stream of at least one quality level and the trained neural network model corresponding to the video code stream to the terminal; Performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes: According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

10. A server, characterized in that: include: A transceiver, used for receiving multi-channel video data uploaded by a terminal, shot from different perspectives of the same object; A processor, used for performing quality classification on multiple channels of video data to obtain a video code stream of at least one quality level; The transceiver is also used to send the video code stream of at least one quality level and the trained neural network model corresponding to the video code stream to the terminal; Performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes: According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

11. A video processing device, characterized in that: Applied to a terminal, the device comprises: An acquisition module is used to shoot the same object from different viewing angles to obtain multiple video data; A processing module is used to perform quality classification on multiple channels of video data to obtain a video stream of at least one quality level; the video stream of at least one quality level is input into a trained neural network model corresponding to the video stream received from a server for processing to obtain a processed video stream; Performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes: According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

12. A terminal, characterized in that: include: A collector is used to shoot the same object from different perspectives to obtain multiple video data; The processor is used to perform quality classification on multiple channels of video data to obtain a video stream of at least one quality level; the video stream of at least one quality level is input into a trained neural network model corresponding to the video stream received from the server for processing to obtain a processed video stream; Performing quality grading on multiple channels of video data to obtain a video stream of at least one quality level includes: According to a first quality threshold set for each channel of video data, each channel of video data is divided into a plurality of picture group data of a first quality level, and Each channel of video data is divided into a plurality of picture group data of a second quality level according to a second quality threshold set for each channel of video data.

13. A processing device, characterized in that: include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 4 or the method according to any one of claims 5 to 8 is executed.

14. A computer-readable storage medium, characterized in that: The storage device comprises a storage instruction, and when the instruction is executed on a computer, the computer executes the method according to any one of claims 1 to 4 or the method according to any one of claims 5 to 8.

Images