A video telemetry coding system based on scene recognition and corresponding method

Abstract

The present invention discloses a video telemetry encoding system based on scene recognition, which provides a basis for the selection of video compression encoding parameters by identifying the dynamics of the scene contained in the video screen, so as to achieve the lowest bit rate possible while ensuring the video quality, and to make full use of the bit rate bandwidth to maximize the quality of the telemetry video image. It includes: a video acquisition and recognition module, which receives various video input signals for format conversion, automatically recognizes the scene of each video screen, and transmits the recognition result to the video compression encoding module; a video compression encoding module, which automatically sets the compression encoding parameters according to the scene recognition result output by the video acquisition and recognition module, and compresses and encodes the original digital video code stream output by it; and a PCM encoding and signal transmission module, which receives the compressed video code stream output by the video compression encoding module.

Description

Technical Field

[0001] The present invention relates to the field of aerospace telemetry, and in particular to a video telemetry coding system based on scene recognition. The present invention also provides a video telemetry coding method based on scene recognition. Background Art

[0002] Telemetry is an important technical means for aerospace flight tests and remote monitoring. It is a technology that transmits short-range measurement values ​​of target parameters and on-site conditions to a distant measurement station to achieve long-distance measurement. Among all types of telemetry data, video image data is an important part of telemetry data because it can intuitively and accurately describe the status of aerospace vehicles. With the advancement of technology, more and more video data needs to be processed in telemetry systems. For example, more cameras are installed on aircraft to monitor target status. At the same time, the resolution, frame rate, and quality of videos are gradually improving, resulting in an increasing amount of video data to be telemetered.

[0003] The bandwidth of telemetry data communication transmission link is usually limited. For example, when PCM-FM and SOQPSK modulation modes are used to encode the information on the flight equipment into a pulse code modulation (PCM) format for wireless transmission to the ground, each video image is usually required to be compressed and encoded and then combined and encoded in the PCM encoding format for transmission. With the increase in the number of airborne video channels, the resolution and frame rate are getting higher and higher, the limitation of telemetry bandwidth is gradually highlighted. Especially when the telemetry link is interfered and the link bandwidth is reduced, if the multi-channel telemetry video screen is transmitted according to the previously set processing parameters, each video channel will be greatly affected. In addition, in different telemetry application scenarios, the data volume and quality of the video after compression according to the same parameters will be different. In different stages of the same application scenario, there will also be differences in the changes in the video scene, resulting in differences in the data volume and quality of the compressed video data. For example, there is a big difference between monitoring the cabin environment of the aircraft and monitoring the moving parts outside the cabin. If fixed video compression encoding parameters and PCM encoding parameters are used for processing during the telemetry process, the quality of some video images will be reduced when the data volume is large, and the bandwidth of the telemetry communication link will not be fully utilized when the data volume is small. Summary of the invention

[0004] In view of the above problems, the present invention provides a video telemetry encoding system based on scene recognition, which provides a basis for selecting video compression encoding parameters by identifying the dynamics of the scenes contained in the video images, so as to achieve the lowest possible bit rate while ensuring the video quality, and make full use of the bit rate bandwidth to maximize the quality of the telemetry video image.

[0005] A video telemetry encoding system based on scene recognition, characterized in that it includes:

[0006] Video acquisition and recognition module;

[0007] Video compression encoding module;

[0008] And PCM encoding and signal transmission module;

[0009] After receiving the video input signals from various channels and converting their formats, the video acquisition and recognition module automatically recognizes the scenes of the video images from various channels, and transmits the recognition results to the video compression and encoding module;

[0010] The video compression and encoding module automatically sets compression and encoding parameters according to the scene recognition result output by the video acquisition and recognition module, and compresses and encodes the original digital video code stream output by the module;

[0011] The PCM encoding and signal transmission module receives the compressed video code stream output by the video compression encoding module, selects several videos according to the configuration, splits each frame of the compressed video data and packages them into PCM data packets, forms a digital signal according to the set digital encoding type, and modulates the signal and transmits it.

[0012] It is further characterized by:

[0013] The input video of the video acquisition and recognition module supports analog signals and digital signals, and corresponds to videos of various formats. A DSP chip or an FPGA is used for data acquisition and format conversion to obtain a digital video image stream in RGB format; the automatic scene recognition is implemented by a SoC chip or an FPGA chip or a GPU chip containing a deep learning neural network reasoning calculation unit;

[0014] The compression coding in the video compression coding module uses a DSP chip or a SoC chip to hard-code the conventional H.264 / H.265 compression coding format, and the automatically set compression coding parameters include: the selected channel, video resolution and frame rate, GOP length, and bit rate.

[0015] The video telemetry encoding method based on scene recognition adopts a video telemetry encoding system based on scene recognition, and is characterized in that it comprises the following steps:

[0016] Step 1: Collect the video input signals from various channels and convert their formats to obtain a digital video image stream in RGB format; when an external input time signal is received, the parsed time information is superimposed on each frame of the video in real time to achieve time code synchronization;

[0017] Step 2: Automatically identify the scene of each video channel, with the input being a small video segment of each channel, and the output being the dynamic degree score of the input small video segment;

[0018] Step 3: Calculate and set compression coding parameters according to the scene recognition result, taking the PCM coding transmission bit rate as a benchmark, according to the priority order of each channel, according to the scene recognition result of each channel video in step 2, refer to the compression coding parameter setting table, calculate the parameter configuration when the sum of the compression coding bit rates of each channel is less than the PCM coding transmission bit rate and the video quality is the highest, and use it as the current compression coding parameter setting value of each channel;

[0019] Step 4: Perform H.264 or H.265 compression encoding, and perform H.264 or H.265 compression encoding on the video signal collected and converted in step 1 according to the parameters set in step 3. The compression encoding process is hard-coded using a DSP chip or a SoC chip;

[0020] Step 5, PCM encoding and output, split each frame of the video data compressed in step 4 and package it into PCM data packets, and form a digital signal according to the set digital encoding type, and modulate the signal and transmit it.

[0021] It is further characterized by:

[0022] The scene automatic recognition in step 2 is deployed and transplanted to the video telemetry encoding system after offline training in advance. The video telemetry encoding system is implemented using a SoC chip, FPGA chip or GPU chip that includes a deep learning neural network inference computing unit;

[0023] It uses a C3D network model structure to realize automatic scene recognition. The C3D model performs scene classification and outputs the video dynamic intensity value, that is, a score belonging to [0,1] is used to measure the strength of the video dynamics. The number of neurons in the last layer is changed from multiple to one, that is, the classification value is changed to a regression value. The loss function during training uses a regression type function. The specific training method of the scene automatic recognition network is as follows:

[0024] Step 2-1: Collect training data

[0025] Collect the original video data collected by the system in each scene under the application conditions, including various video data in different scenes, targets, weather and light environments, and the same category of video should include both dynamic scenes and static scenes as much as possible;

[0026] Step 2-2: compress and encode the original video data

[0027] Use H.264 or H.265 encoding method to perform compression encoding according to fixed parameters, and the parameters are set to a medium compression level within the settable range in the corresponding application scenario;

[0028] Step 2-3: Evaluate the dynamic quality of the compressed and encoded video and perform data annotation

[0029] Decode the video compressed in step 2-2 using H.264 or H.265, evaluate the decoded video, and score each frame according to the video quality to measure the degree of dynamics. The score value is between [0,1]. A score close to 0 represents high video image quality and low dynamics, and a score close to 1 represents low video image quality and high dynamics. The dynamics score of each frame is annotated as the true value and recorded in the training data set.

[0030] Step 2-4: Use labeled data for network training

[0031] According to the number of video segment frames that matches the input dimension of the established C3D network model, segmented videos are intercepted from the training video data set annotated in step 2-3 as input for model training to obtain a trained scene automatic recognition model.

[0032] Step 2-3 evaluates the video image quality by manual subjective evaluation or by using an image quality evaluation algorithm. When evaluating, the score should be mainly based on the image quality caused by different degrees of dynamics. If the video is long, the video can be scored in segments, and the scores between segments are transitioned by linear interpolation.

[0033] The compression encoding parameter setting table in step 3 needs to be prepared in advance. The method for making the compression encoding parameter setting table is as follows:

[0034] According to the training data of the scene recognition network in step 2, the scene is divided into multiple intervals according to the dynamic degree score. The score of each interval can be evenly distributed, and multiple sets of compression encoding parameters Para are set for each interval. (d，g) =[r (d，g) , f (d，g) ,i (d，g) , b (d，g) ] Wherein the subscript d is the dynamic degree interval, g is the parameter group number of various compression coding effects, and the bit rate is used as the distinction. After setting the d value, the corresponding video original image is selected from the training data of the scene recognition network in step 2. According to the range of g values, different bit rates are fixed according to the application requirements, that is, b (d，g) , and then comprehensively adjust r (d，g) ,f (d，g) ,i (d，g)The parameters are used for H.264 or H.265 compression encoding, and the quality after compression is evaluated by an artificial supervisory judgment method or an image quality evaluation algorithm, so as to achieve the compressed image quality effect corresponding to g, thereby determining the parameters of each gear and finally forming a compression encoding parameter setting table.

[0035] The method for calculating and determining the encoding parameters of each channel according to the compression encoding parameter setting table is as follows:

[0036] Step 3-1, obtain the current system PCM encoding transmission bit rate B pcm ;

[0037] Step 3-2: Obtain the current priority order weights of each channel P = [p1, p2, ..., p i ], where i = 1, 2, ..., c represents the channel number, c is the number of channels, p i Use different values ​​to indicate different priorities; i ∈{1, 2, 3}, the priority order weights of each channel are set in advance in different application scenarios;

[0038] Step 3-3, determine each p i Value, according to the compression encoding parameter setting table, calculate p i =3 channel B i The sum of values ​​B sum3 =∑B i , i∈p i =3; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image. (d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) ;

[0039] Judgment B sum3 Is it less than B? pcm If yes, go to step 3-4. Otherwise, the PCM encoding transmission bandwidth cannot meet the current application requirements, and prompt to modify the priority order weight P of each channel, or end the process;

[0040] Step 3-4: Calculate p according to the compression encoding parameter setting table i =2 channel B i The sum of values ​​B sum2 =∑B i , i∈p i =2; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image.(d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) .

[0041] Step 3-5, judge B sum3 +B sum2 Is it less than B? pcm If yes, go to step 3-8, otherwise go to step 3-6;

[0042] Step 3-6: Recalculate p according to the method in step 3-4 i =2 channel B i The sum of values ​​B sum2 , just choose the g value that represents the average quality of the compressed image, and get a smaller bit rate B i ;

[0043] Step 3-7: Re-judge B sum3 +B sum2 Is it less than B? pcm If yes, go to step 3-8. Otherwise, the PCM encoding transmission bandwidth cannot meet the current application requirements, and prompt to modify the priority order weight P of each channel, or end the process;

[0044] Step 3-8: Calculate p according to the compression encoding parameter setting table i = 1 for channel B i The sum of values ​​B sum1 =∑B i , i∈p i =1; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image. (d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) ;

[0045] Step 3-9, Determine B sum3 +B sum2 +B sum1 Is it less than B? pcm If yes, go to step 3-12, otherwise go to step 3-10;

[0046] Step 3-10: Recalculate p according to step 3-8 i = 1 for channel B i The sum of values ​​B sum1 , just choose the g value that represents the average quality of the compressed image, and get a smaller bit rate B i ;

[0047] Step 3-11, re-judge Bsum3 +B sum2 +B sum1 Is it less than B? pcm If yes, go to step 3-12, otherwise do not select p i =1 channel for compression coding and PCM coding transmission;

[0048] Step 3-12, obtain all compression encoding parameters and the process ends.

[0049] After adopting the present invention, by identifying the dynamics of the scenes contained in the video screen, a basis is provided for the selection of video compression encoding parameters, so as to achieve the lowest possible bit rate while ensuring the video quality, and make full use of the bit rate bandwidth to maximize the quality of the telemetry video image; it can automatically select the number of telemetry channels and adjust the video compression encoding parameters of each channel according to the scene recognition result and the priority of the channel telemetry, so as to realize the dynamic adjustment and full application of system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a block diagram of the system of the present invention;

[0051] Figure 2 The present invention is an operational flow chart of the method. DETAILED DESCRIPTION

[0052] A video telemetry coding system based on scene recognition, see Figure 1 , which includes:

[0053] The video acquisition and recognition module receives the video input signals from various channels and converts the formats of the signals. It then automatically recognizes the scenes of the various video images and transmits the recognition results to the video compression and encoding module.

[0054] A video compression and encoding module, which automatically sets compression and encoding parameters according to the scene recognition results output by the video acquisition and recognition module, and compresses and encodes the original digital video code stream output by it;

[0055] And the PCM encoding and signal transmission module receives the compressed video code stream output by the video compression encoding module, selects several videos according to the configuration, splits each frame of the compressed video data and packages them into PCM data packets, forms digital signals according to the set digital encoding type, and modulates the signal before transmitting it.

[0056] In specific implementation, the input video of the video acquisition and recognition module supports analog signals and digital signals, such as: PAL, VGA, LVDS, DVI, SDI, network video, GigE; corresponding to various formats of video, DSP chip or FPGA is used for data acquisition and format conversion to obtain a digital video image stream in RGB format; automatic scene recognition is implemented by SoC chip or FPGA chip or GPU chip containing deep learning neural network reasoning calculation unit;

[0057] The compression coding in the video compression coding module uses DSP chip or SoC chip to hard-code the conventional H.264 / H.265 compression coding format. The automatically set compression coding parameters include: the selected channel, video resolution and frame rate, GOP (Group of Pictures) length (I frame spacing), and bit rate.

[0058] The video telemetry coding method based on scene recognition adopts a video telemetry coding system based on scene recognition, see Figure 2 , which comprises the following steps:

[0059] Step 1: Collect the video input signals from various channels and convert their formats to obtain a digital video image stream in RGB format; when an external input time signal is received, the parsed time information is superimposed on each frame of the video in real time to achieve time code synchronization;

[0060] The input video supports analog signals and digital signals, such as: PAL, VGA, LVDS, DVI, SDI, network video, GigE; corresponding to various formats of video, DSP chip or FPGA is used for data acquisition and format conversion to obtain digital video image stream in RGB format.

[0061] Step 2: Automatically identify the scene of each video channel, with the input being a small video segment of each channel, and the output being the dynamic degree score of the input small video segment;

[0062] Automatic scene recognition is trained offline in advance and then deployed and transplanted to the video telemetry encoding system. The video telemetry encoding system is implemented using a SoC chip, FPGA chip or GPU chip that includes a deep learning neural network inference computing unit;

[0063] The automatic scene recognition is realized by using the C3D (3-Dimensional Convolution) network model structure. The C3D model performs scene classification and outputs the video dynamic intensity value, that is, a score belonging to [0,1] is used to measure the strength of the video dynamics. The number of neurons in the last layer is changed from multiple to one, that is, the classification value is changed to a regression value. The loss function during training uses a regression type function, such as the mean square error MSE function or the mean absolute error MAE function or the Huber loss function. The specific training method of the scene automatic recognition network is as follows:

[0064] Step 2-1: Collect training data

[0065] Collect the original video data collected by the system in each scene under the application conditions, including various video data in different scenes, targets, weather and light environments, and the same category of video should include both dynamic scenes and static scenes as much as possible;

[0066] Step 2-2: compress and encode the original video data

[0067] Use H.264 or H.265 encoding to perform compression encoding according to fixed parameters. The parameters are set to a medium compression level within the settable range in the corresponding application scenario. For example, when the bit rate range is 0.5Mbps to nMbps, the bit rate is set to (n / 2)Mbps. Similarly, the quantization parameter (QP), GOP length, etc. are also fixed to the middle value of the range.

[0068] Step 2-3: Evaluate the dynamic quality of the compressed and encoded video and perform data annotation

[0069] Decode the video compressed in step 2-2 using H.264 or H.265, evaluate the decoded video, and score each frame according to the video quality to measure the degree of dynamics. The score value is between [0,1]. A score close to 0 represents high video image quality and low dynamics, and a score close to 1 represents low video image quality and high dynamics. The dynamics score of each frame is annotated as the true value and recorded in the training data set.

[0070] The video image quality is evaluated by manual subjective evaluation or by using an image quality evaluation algorithm (such as the existing PSNR, VMAF, DeepVQA, Patch-VQ, and VSFA algorithms). When evaluating, the score should be mainly based on the performance of the image quality caused by different degrees of dynamics, such as mosaics, blur, smearing, or motion artifacts caused by the degradation of video image quality. If the video is long, the video can be scored in segments, and the scores between the segments are transitioned by linear interpolation.

[0071] Step 2-4: Use labeled data for network training

[0072] According to the number of video segment frames (e.g., 16) that matches the input dimension of the established C3D network model, segmented videos are intercepted from the training video data set annotated in step 2-3 as input for model training to obtain a trained scene automatic recognition model. In a preferred embodiment, the intercepted video segments include both scenes with a high degree of dynamics and scenes with a low degree of dynamics.

[0073] Step 3: Calculate and set compression coding parameters according to the scene recognition result, taking the PCM coding transmission bit rate as a benchmark, according to the priority order of each channel, according to the scene recognition result of each channel video in step 2, refer to the compression coding parameter setting table, calculate the parameter configuration when the sum of the compression coding bit rates of each channel is less than the PCM coding transmission bit rate and the video quality is the highest, and use it as the current compression coding parameter setting value of each channel;

[0074] The compression encoding parameter setting table needs to be prepared in advance. The method for making the compression encoding parameter setting table is as follows:

[0075] According to the training data of the scene recognition network in step 2, the scene is divided into multiple intervals according to the dynamic degree score. The score of each interval can be evenly distributed, and multiple sets of compression encoding parameters Para are set for each interval. (d，g) =[r (d，g) , f (d，g) ,i (d，g) , b (d，g) ] Wherein the subscript d is the dynamic degree interval, for example, d = 1 corresponds to the score interval [0, 0.33], d = 2 corresponds to [0.34, 0.66], and g = 3 corresponds to [0.67, 1.0]; g is the parameter group number of various compression coding effects, with the bit rate as the distinction. For example, g = 1 corresponds to a parameter group with general image quality after compression when the bit rate is small, and g = 2 corresponds to a parameter group with good image quality after compression when the bit rate is large. The number of values ​​of g can be determined according to the number of subdivisions required by the application. After setting the d value, select the corresponding original video image from the training data of the scene recognition network in step 2, and fix different bit rates according to the application requirements according to the range of values ​​that g can take, that is, b (d，g) , and then comprehensively adjust r (d，g) ,f (d，g) ,i (d，g) The parameters are used for H.264 or H.265 compression encoding, and the quality after compression is evaluated by an artificial supervisory judgment method or an image quality evaluation algorithm, so as to achieve the compressed image quality effect corresponding to g, thereby determining the parameters of each gear and finally forming a compression encoding parameter setting table;

[0076] The specific method for calculating and determining the encoding parameters of each channel in the compression encoding parameter setting table is as follows:

[0077] Step 3-1, obtain the current system PCM encoding transmission bit rate B pcm ;

[0078] Step 3-2: Obtain the current priority order weights of each channel P = [p1, p2, ..., p i ], where i = 1, 2, ..., c represents the channel number, c is the number of channels, p i Use different values ​​to indicate different priorities; i ∈{1, 2, 3}, 1 represents low weight, and the channel can be unselected; 2 represents medium weight, and the channel must be selected, and the quality priority is low; 3 represents high weight, and the channel must be selected, and the quality priority is high. The priority order and weight of each channel are set in advance in different application scenarios;

[0079] Step 3-3, determine each p i Value, according to the compression encoding parameter setting table, calculate p i =3 channel B i The sum of values ​​B sum3 =∑B i , i∈p i =3; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image. (d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) ;

[0080] Judgment B sum3 Is it less than B? pcm If yes, go to step 3-4. Otherwise, the PCM encoding transmission bandwidth cannot meet the current application requirements, and prompt to modify the priority order weight P of each channel, or end the process;

[0081] Step 3-4: Calculate p according to the compression encoding parameter setting table i =2 channel B i The sum of values ​​B sum2 =∑B i , i∈p i =2; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image. (d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) .

[0082] Step 3-5, judge B sum3+B sum2 Is it less than B? pcm If yes, go to step 3-8, otherwise go to step 3-6;

[0083] Step 3-6: Recalculate p according to the method in step 3-4 i =2 channel B i The sum of values ​​B sum2 , just choose the g value that represents the average quality of the compressed image, and get a smaller bit rate B i ;

[0084] Step 3-7: Re-judge B sum3 +B sum2 Is it less than B? pcm If yes, go to step 3-8. Otherwise, the PCM encoding transmission bandwidth cannot meet the current application requirements, and prompt to modify the priority order weight P of each channel, or end the process;

[0085] Step 3-8: Calculate p according to the compression encoding parameter setting table i = 1 for channel B i The sum of values ​​B sum1 =∑B i , i∈p i =1; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image. (d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) ;

[0086] Step 3-9, Determine B sum3 +B sum2 +B sum1 Is it less than B? pcm If yes, go to step 3-12, otherwise go to step 3-10;

[0087] Step 3-10: Recalculate p according to step 3-8 i = 1 for channel B i The sum of values ​​B sum1 , just choose the g value that represents the average quality of the compressed image, and get a smaller bit rate B i ;

[0088] Step 3-11, re-judge B sum3 +B sum2 +B sum1 Is it less than B? pcm If yes, go to step 3-12, otherwise do not select p i=1 channel for compression coding and PCM coding transmission;

[0089] Step 3-12, obtain all compression encoding parameters and the process ends.

[0090] Step 4: Perform H.264 or H.265 compression encoding. Perform H.264 or H.265 compression encoding on the video signal collected and converted in step 1 according to the parameters set in step 3. The compression encoding process is hard-coded using a DSP chip or a SoC chip.

[0091] Step 5, PCM encoding and output, split each frame of the compressed video data in step 4 and pack it into PCM data packets, and form a digital signal according to the set digital encoding type, modulate the signal and transmit it for transmission. For the specific PCM encoding method and message format, please refer to the implementation method in the document "Airborne Multi-channel Video PCM Telemetry Transmission Technology. Zhang Jie, Zou Qiang, Yan Hui. Computer and Digital Engineering. 2013 Issue 5. p805-807" (no further description here).

[0092] After adopting the present invention, by identifying the dynamics of the scenes contained in the video screen, a basis is provided for the selection of video compression encoding parameters, so as to achieve the lowest possible bit rate while ensuring the video quality, and make full use of the bit rate bandwidth to maximize the quality of the telemetry video image; it can automatically select the number of telemetry channels and adjust the video compression encoding parameters of each channel according to the scene recognition result and the priority of the channel telemetry, so as to realize the dynamic adjustment and full application of system resources.

[0093] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0094] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A video telemetry coding system based on scene recognition, characterized in that: It includes: Video acquisition and recognition module; Video compression encoding module; And PCM encoding and signal transmission module; After receiving the video input signals from various channels and converting their formats, the video acquisition and recognition module automatically recognizes the scenes of the video images from various channels, and transmits the recognition results to the video compression and encoding module; The video compression encoding module automatically sets compression encoding parameters according to the scene recognition result output by the video acquisition and recognition module, and compresses and encodes the original digital video code stream output by it; The PCM encoding and signal transmission module receives the compressed video code stream output by the video compression encoding module, selects several videos according to the configuration, splits each frame of the compressed video data and packages them into PCM data packets, forms a digital signal according to the set digital encoding type, and modulates the signal and transmits it.

2. The video telemetry encoding system based on scene recognition according to claim 1, characterized in that: The input video of the video acquisition and recognition module supports analog signals and digital signals, and corresponds to videos of various formats. A DSP chip or an FPGA is used for data acquisition and format conversion to obtain a digital video image stream in RGB format; the automatic scene recognition is implemented using a SoC chip, an FPGA chip, or a GPU chip that includes a deep learning neural network inference computing unit.

3. The video telemetry encoding system based on scene recognition according to claim 1, characterized in that: The input video of the video acquisition and recognition module supports analog signals and digital signals, and corresponds to videos of various formats. A DSP chip or an FPGA is used for data acquisition and format conversion to obtain a digital video image stream in RGB format; the automatic scene recognition is implemented using a SoC chip, an FPGA chip, or a GPU chip that includes a deep learning neural network inference computing unit.

4. A video telemetry encoding method based on scene recognition, which adopts a video telemetry encoding system based on scene recognition as claimed in any one of claims 1 to 3, characterized in that: It includes the following steps: Step 1: Collect the video input signals from various channels and convert their formats to obtain a digital video image stream in RGB format; when an external input time signal is received, the parsed time information is superimposed on each frame of the video in real time to achieve time code synchronization; Step 2: Automatically identify the scene of each video channel, with the input being a small video segment of each channel, and the output being the dynamic degree score of the input small video segment; Step 3: Calculate and set compression coding parameters according to the scene recognition result, taking the PCM coding transmission bit rate as a benchmark, according to the priority order of each channel, according to the scene recognition result of each channel video in step 2, refer to the compression coding parameter setting table, calculate the parameter configuration when the sum of the compression coding bit rates of each channel is less than the PCM coding transmission bit rate and the video quality is the highest, and use it as the current compression coding parameter setting value of each channel; Step 4: Perform H.264 or H.265 compression encoding, and perform H.264 or H.265 compression encoding on the video signal collected and converted in step 1 according to the parameters set in step 3. The compression encoding process is hard-coded using a DSP chip or a SoC chip; Step 5, PCM encoding and output, split each frame of the video data compressed in step 4 and package it into PCM data packets, and form a digital signal according to the set digital encoding type, and modulate the signal and transmit it.

5. The video telemetry encoding method based on scene recognition according to claim 4 is characterized in that: The scene automatic recognition in step 2 is deployed and transplanted to the video telemetry encoding system after offline training in advance. The video telemetry encoding system is implemented using a SoC chip, FPGA chip or GPU chip that includes a deep learning neural network inference computing unit.

6. The video telemetry encoding method based on scene recognition according to claim 5, characterized in that: It uses a C3D network model structure to realize automatic scene recognition. The C3D model performs scene classification and outputs the video dynamic intensity value, that is, a score belonging to [0,1] is used to measure the strength of the video dynamics. The number of neurons in the last layer is changed from multiple to one, that is, the classification value is changed to a regression value. The loss function during training uses a regression type function. The specific training method of the scene automatic recognition network is as follows: Step 2-1: Collect training data Collect the original video data collected by the system in each scene under the application conditions, including various video data in different scenes, targets, weather and light environments, and the same category of video should include both dynamic scenes and static scenes as much as possible; Step 2-2: compress and encode the original video data Use H.264 or H.265 encoding method to perform compression encoding according to fixed parameters, and the parameters are set to a medium compression level within the settable range in the corresponding application scenario; Step 2-3, evaluate the dynamic degree of the compressed and encoded video quality and perform data annotation. The video compressed in step 2-2 is decoded by H.264 or H.265, and the decoded video is evaluated. Each frame of the image is scored according to the video quality to measure the dynamic degree. The score value is between [0,1]. A score close to 0 represents high video image quality and low dynamic degree, and a score close to 1 represents low video image quality and high dynamic degree. The dynamic degree score of each frame is annotated as the true value and recorded in the training data set; Step 2-4: Use labeled data for network training According to the number of video segment frames that matches the input dimension of the established C3D network model, segmented videos are intercepted from the training video data set annotated in step 2-3 as input for model training to obtain a trained scene automatic recognition model.

7. The video telemetry encoding method based on scene recognition according to claim 6 is characterized in that: Step 2-3 evaluates the video image quality by manual subjective evaluation or by using an image quality evaluation algorithm. When evaluating, the score should be mainly based on the image quality caused by different degrees of dynamics. If the video is long, the video can be scored in segments, and the scores between segments are transitioned by linear interpolation.

8. The video telemetry encoding method based on scene recognition according to claim 4 is characterized in that: The compression encoding parameter setting table in step 3 needs to be prepared in advance. The method for making the compression encoding parameter setting table is as follows: According to the training data of the scene recognition network in step 2, the scene is divided into multiple intervals according to the dynamic degree score. The score of each interval can be evenly distributed, and multiple sets of compression encoding parameters Para are set for each interval. (d，g) =[r (d，g) ,f (d，g) ,i (d，g) ,b (b，g) ], where the subscript d is the dynamic range, g is the parameter group number of various compression coding effects, and the bit rate is used as the distinction. After setting the d value, the corresponding video original image is selected from the training data of the scene recognition network in step 2, and different bit rates are fixed according to the application requirements according to the range of g values, that is, b (d，g) , and then comprehensively adjust r (d，g) ,f (d，g) ,i (d，g) The parameters are used for H.264 or H.265 compression encoding, and the quality after compression is evaluated by an artificial supervisory judgment method or an image quality evaluation algorithm, so as to achieve the compressed image quality effect corresponding to g, thereby determining the parameters of each gear and finally forming a compression encoding parameter setting table.

9. The video telemetry encoding method based on scene recognition according to claim 8, characterized in that: The method for calculating and determining the encoding parameters of each channel according to the compression encoding parameter setting table is as follows: Step 3-1, obtain the current system PCM encoding transmission bit rate B pcm ; Step 3-2: Obtain the current priority order weights of each channel P = [p1, p2, ..., p i ], where i = 1, 2, ..., c represents the channel number, c is the number of channels, p i Use different values ​​to indicate different priorities; i ∈{1, 2, 3}, the priority order weights of each channel are set in advance in different application scenarios; Step 3-3, determine each p i Value, according to the compression encoding parameter setting table, calculate p i =3 channel B i The sum of values ​​B sum3 =ΣB i , i∈p i =3; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image. (d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) ; Judgment B sum3 Is it less than B? pcm If yes, go to step 3-4. Otherwise, the PCM encoding transmission bandwidth cannot meet the current application requirements, and prompt to modify the priority order weight P of each channel, or end the process; Step 3-4: Calculate p according to the compression encoding parameter setting table i =2 channel B i The sum of values ​​B sum2 =∑B i , i∈p i =2; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image. (d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) ; Step 3-5, judge B sum3 +B sum2 Is it less than B? pcm If yes, go to step 3-8, otherwise go to step 3-6; Step 3-6: Recalculate p according to the method in step 3-4 i =2 channel B i The sum of values ​​B sum2 , just choose the g value that represents the average quality of the compressed image, and get a smaller bit rate B i ; Step 3-7: Re-judge B sum3 +B sum2 Is it less than B? pcm If yes, go to step 3-8. Otherwise, the PCM encoding transmission bandwidth cannot meet the current application requirements, and prompt to modify the priority order weight P of each channel, or end the process; Step 3-8: Calculate p according to the compression encoding parameter setting table i = 1 for channel B i The sum of values ​​B sum1 =∑B i , i∈p i =1; B i The specific method is: determine the d value according to the interval of the dynamic score of the video scene automatic recognition result, and select the value from the parameter table Para according to the d value and the g value representing the higher quality of the compressed image. (d，g) Select the corresponding parameter in and you will get the corresponding bit rate, that is, B i =b (d，g) ; Step 3-9, Determine B sum3 +B sum2 +B sum1 Is it less than B? pcm If yes, go to step 3-12, otherwise go to step 3-10; Step 3-10: Recalculate p according to step 3-8 i = 1 for channel B i The sum of values ​​B sum1 , just choose the g value that represents the average quality of the compressed image, and get a smaller bit rate B i ; Step 3-11, re-judge B sum3 +B sum2 +B sum1 Is it less than B? pcm If yes, go to step 3-12, otherwise do not select p i =1 channel for compression coding and PCM coding transmission; Step 3-12, obtain all compression encoding parameters and the process ends.

Images