Audio and video integrated rocket-borne camera data processing method and device and storage medium

Abstract

The invention belongs to the technical field of spaceflight launching, and discloses a rocket-borne camera data processing method and device and a storage medium. Comprising the following steps: acquiring audio signal data and video signal data acquired by an audio acquisition module and a video acquisition module; analyzing the audio signal data to identify key audio clips, and analyzing the video signal data to identify key frames; generating a priority label based on the identification result; performing hybrid coding on the audio signal data and the video signal data, and adding priority labels and timestamps to form hybrid data; the mixed data is compressed, the key frames and the key audio clips are compressed in a lossless compression mode, and other signals are compressed in a lossy compression mode; meanwhile, the compression ratio is adjusted through real-time communication bandwidth optimization; and packaging the compressed data to form an audio and video data stream for transmission. According to the invention, integrated acquisition and transmission of audio and video data can be realized, and the fidelity effect of received and decoded data is good.

Description

Technical Field

[0001] The present invention belongs to the technical field of aerospace measurement systems, and in particular relates to a method, a device and a storage medium for processing data of an audio and video integrated arrow-borne camera in the field of aerospace telemetry. Background Art

[0002] Limited by telemetry bandwidth and telemetry methods, the telemetry images sent back by rocket-borne cameras in the prior art only contain video images and lack audio information, resulting in insufficient diversity and detail expression of telemetry data, and a poor experience for all parties. Especially during rocket launch and flight, the lack of audio background makes the information appear thin and weak, and the live broadcast of rocket launches is not effective, which is not conducive to popular science, publicity and commercialization. In addition, most existing telemetry data compression schemes are fixed to lossy compression or lossless compression, and lack adaptive switching capabilities. When using a single lossy compression, the space is insufficient, reducing the utilization rate of telemetry bandwidth; when using a single lossless compression, the real-time performance is insufficient, which can easily cause data transmission delays or packet loss.

[0003] With the advent of the era of space travel and the update of various high-bitrate air- and ground-based telemetry methods, the time has come for rocket-borne cameras with audio data acquisition, data correlation compression, and encoding functions to enter the field of modern space telemetry. In a high-dynamic, high-interference, and limited telemetry bandwidth flight environment, how to optimize the use of space telemetry bandwidth and achieve the hybrid acquisition, data correlation compression, and encoding of integrated audio and video for rocket-borne cameras is an urgent problem to be solved in the field of rocket launches. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes an audio and video data processing method, device and storage medium for an audio and video integrated arrow-borne camera, so as to realize the synchronous acquisition and processing of audio and video data, and accelerate the demodulation speed while ensuring the demodulation accuracy, so as to realize the demodulation of different types of signals when the original data is distorted.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for processing audio and video data of an arrow-borne camera with integrated audio and video, comprising the following steps:

[0006] Step 1: Acquire the audio signal data and video signal data collected by the audio acquisition module and the video acquisition module, and align the timestamps of the audio data and the video data through a synchronization mechanism;

[0007] Step 2: Analyze the waveform changes of the audio signal data to identify the key audio segments. At the same time, analyze the video signal data to identify the key frames. Then generate priority tags based on the identification results.

[0008] Step 3: Mix and encode the audio signal data and the video signal data, and add a priority tag and a timestamp to form mixed data;

[0009] Step 4: Compress the mixed data. During compression, key frames and key audio clips are compressed using lossless compression, and other signals are compressed using lossy compression. At the same time, the compression ratio is adjusted through real-time communication bandwidth optimization. The compressed data is then packaged into an audio and video data stream for transmission.

[0010] In the step 1, the video acquisition module acquires video signal data at a dynamic frame rate, and after the audio acquisition module acquires the audio signal data, it also includes the steps of noise reduction and waveform optimization.

[0011] In the step 2, key frames are identified according to the motion complexity and inter-frame variation characteristics of the video signal data.

[0012] In the step three, the compressed audio signal data and video signal data are mixedly encoded through multi-channel encoding, and priority tags and time stamps are added to form mixed data.

[0013] In step 4, the specific method of adjusting the compression ratio by optimizing the communication bandwidth in real time is: setting the communication bandwidth reference M and the reference compression ratio C, obtaining the real-time communication bandwidth M1, and adjusting the compression ratio C1 of the lossy compression method to:

[0014] C1=A×C×M1 / M;

[0015] Where A is the adjustment coefficient.

[0016] The method for processing audio and video data of an arrow-borne camera with integrated audio and video also includes the following steps:

[0017] Step 5: After receiving the audio and video data stream, the ground station decodes and synchronously reconstructs it based on the priority tag and timestamp to achieve data restoration.

[0018] In step 5, the specific method of decoding and synchronously reconstructing based on the priority tag and timestamp to achieve data restoration is:

[0019] Decompress the key data in the audio and video data stream according to the priority tag; ensure the synchronization of the reconstructed audio and video according to the timestamp information.

[0020] The step five also includes the following steps: decoding the corresponding key feature point content according to the input decoding command.

[0021] In addition, the present invention also provides an arrow-borne camera audio and video data processing device, comprising a processor and a memory communicatively connected to the processor;

[0022] The memory stores computer-executable instructions;

[0023] The processor executes the computer execution instructions stored in the memory to implement the method for processing audio and video data of an audio and video integrated arrow-borne camera.

[0024] In addition, the present invention also provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used for the audio and video data processing method of the audio and video integrated arrow-borne camera.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] Compared with the best existing technology, the present invention has the following advantages:

[0027] 1. Realize the integrated collection and processing of audio and video data: Compared with the traditional arrow-borne camera that only collects video data, the present invention can collect audio and video data at the same time, which greatly enriches the content of telemetry data and improves the diversity and expressiveness of information. The synchronous collection of audio and video data can more accurately reflect the multi-dimensional information during the rocket launch process and enhance the application value of the data, especially in scenarios such as scientific research, popular science, and commercial live broadcast.

[0028] 2. Achieve a balance between data quality and bandwidth: The present invention adopts an adaptive compression method based on data characteristics, and can flexibly select lossless compression and lossy compression according to the different contents of audio and video data, thereby achieving a balance between data quality and bandwidth utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flowchart of a method for processing audio and video data of an arrow-borne camera with integrated audio and video is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0031] Embodiment 1

[0032] like Figure 1As shown, the first embodiment of the present invention provides a method for processing audio and video data of an arrow-borne camera with integrated audio and video, comprising the following steps:

[0033] Step 1: Acquire the audio signal data and video signal data collected by the audio acquisition module and the video acquisition module, and align the timestamps of the audio data and the video data through a synchronization mechanism.

[0034] In the step 1, the video acquisition module acquires video signal data at a dynamic frame rate, and after the audio acquisition module acquires the audio signal data, it also includes the steps of noise reduction and waveform optimization.

[0035] In this embodiment, the arrow-borne camera integrates audio and video acquisition modules to achieve synchronous acquisition and preliminary processing of audio and video signals. Specifically, the video module acquires high-resolution images and supports dynamic frame rate adjustment. The audio module acquires ambient sound in real time and performs noise reduction and waveform optimization. The synchronization mechanism ensures the alignment of the timestamps of audio and video data, laying the foundation for subsequent hybrid encoding.

[0036] Step 2: Analyze the waveform changes of the audio signal data and identify the key audio segments. At the same time, analyze the video signal data and identify the key frames; then generate priority tags based on the recognition results.

[0037] Furthermore, in the step 2, key frames are identified according to the motion complexity and inter-frame variation characteristics of the video signal data.

[0038] In this embodiment, the main function of step 2 is to perform data characteristic analysis, perform characteristic analysis on the collected audio and video data, and distinguish key content from ordinary content. Specifically, by analyzing the motion complexity, inter-frame changes and other characteristics of the video content, key frames are identified. In addition, by analyzing the waveform changes of the audio signal, high-dynamic or key audio clips are identified. Based on the recognition results, priority tags can be generated through content priority, providing a basis for subsequent adaptive compression.

[0039] Step 3: Mix and encode the audio signal data and the video signal data, and add a priority tag and a timestamp to form mixed data.

[0040] Furthermore, in the step three, the compressed audio signal data and video signal data are mixedly encoded through multi-channel encoding, and priority tags and timestamps are added during encoding to form mixed data.

[0041] In this embodiment, multi-channel encoding technology is used to efficiently integrate audio and video data. Flight control metadata (such as timestamps and priority tags) are added during data packaging to facilitate ground station analysis.

[0042] Step 4: Compress the mixed data. During compression, key frames and key audio clips are compressed using lossless compression, and other signals are compressed using lossy compression. At the same time, the compression ratio is adjusted through real-time communication bandwidth optimization. The compressed data is then packaged into an audio and video data stream for transmission.

[0043] Specifically, in step 4, the specific method of adjusting the compression ratio by optimizing the communication bandwidth in real time is: setting the communication bandwidth reference M and the reference compression ratio C, obtaining the real-time communication bandwidth M1, and adjusting the compression ratio C1 of the lossy compression method to:

[0044] C1= A×C×M1 / M; (1)

[0045] Wherein, A is the adjustment coefficient. Through the above formula (1), the compression ratio can be optimized and adjusted along with the real-time communication bandwidth, and the dynamic parameter adjustment of the compression process can be realized, so that the subsequent data decoding and restoration can be maximized.

[0046] Furthermore, the audio and video data processing method of the arrow-borne camera with integrated audio and video in this embodiment further includes the following steps:

[0047] Step 5: After receiving the audio and video data stream, the ground station decodes and synchronously reconstructs it based on the priority tag and timestamp to achieve data restoration.

[0048] Furthermore, in step 5, decoding and synchronous reconstruction are performed based on the priority tag and timestamp to achieve data restoration in a specific method as follows:

[0049] Decompress the key data in the audio and video data stream according to the priority tag; ensure the synchronization of the reconstructed audio and video according to the timestamp information.

[0050] Furthermore, the step five also includes the following steps: decoding the corresponding key feature point content according to the input decoding command.

[0051] In this embodiment, after receiving the audio and video data stream, the ground station decodes and reconstructs it synchronously to achieve high-quality data restoration. Specifically, the ground station automatically decompresses key data to ensure high fidelity of the picture and sound. The decoding process is optimized according to the priority tags in the metadata to ensure the synchronization of the reconstructed audio and video. In addition, multi-level decoding options are provided to prioritize the restoration of key feature point content, such as fairing separation, satellite-rocket separation content, etc., to meet different analysis needs.

[0052] An audio and video data processing method of an arrow-borne camera with integrated audio and video provided in an embodiment of the present invention has the following characteristics:

[0053] 1. Realize efficient collection and compression of integrated audio and video telemetry data:

[0054] The embodiments of the present invention integrate audio and video data to construct an integrated audio and video onboard camera system, thereby improving the diversity and detail expression of telemetry data and providing richer telemetry information for space missions.

[0055] 2. Optimize telemetry bandwidth utilization:

[0056] In view of the limitation of limited telemetry bandwidth, the embodiment of the present invention adopts a data correlation compression algorithm to adaptively select lossy or lossless compression method to compress data according to data content characteristics, and adjusts compression parameters according to the communication bandwidth, so as to balance data transmission quality and bandwidth utilization and maximize the utilization of bandwidth resources.

[0057] 3. Improve the event reproducibility and application value of telemetry data:

[0058] The embodiments of the present invention can achieve high-fidelity audio and video recording, especially lossless compression and accurate restoration of key audio and video data, ensure the ability to reproduce events during rocket launch, flight and mission execution, and support scientific research analysis, mission summary and commercial use.

[0059] The feasibility of the embodiment of the present invention has been proved through a series of simulation verifications. The following is part of the implementation source code:

[0060]

[0061]

[0062]

[0063]

[0064]

[0065] Embodiment 2

[0066] A second embodiment of the present invention provides an arrow-borne camera audio and video data processing device, including a processor and a memory communicatively connected to the processor;

[0067] The memory stores computer-executable instructions;

[0068] The processor executes the computer-executable instructions stored in the memory to implement the method for processing audio and video data of an audio and video integrated arrow-borne camera as described in the first embodiment.

[0069] Embodiment 3

[0070] Embodiment 3 of the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method for processing audio and video data of an audio and video integrated arrow-borne camera as described in Embodiment 1.

[0071] In summary, the present invention provides an audio and video data processing method, device and storage medium for an audio and video integrated rocket-borne camera, which can collect audio and video data at the same time, making up for the defect that traditional telemetry images only contain video. The specific audio and video acquisition circuit design and chip selection are well known, and the present invention will not be elaborated. The present invention protects the audio acquisition function of the launch vehicle camera. The present invention emphasizes the collection of audio and video data through the cooperation of a high-sensitivity microphone and a high-resolution camera to improve the three-dimensionality, diversity and restoration of telemetry data. Specifically, the present invention merges audio and video data into one data stream to enhance the integrity and transmission efficiency of the data; at the same time, by dynamically selecting compression, lossy compression and lossless compression are adaptively selected according to the characteristics of the audio and video data, so as to achieve a balance between telemetry bandwidth and data quality.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing audio and video data of an arrow-borne camera with integrated audio and video, characterized in that: The following steps are involved: Step 1: Acquire the audio signal data and video signal data collected by the audio acquisition module and the video acquisition module, and align the timestamps of the audio data and the video data through a synchronization mechanism; Step 2: Analyze the waveform changes of the audio signal data to identify the key audio segments. At the same time, analyze the video signal data to identify the key frames. Then generate priority tags based on the identification results. Step 3: Mix and encode the audio signal data and the video signal data, and add a priority tag and a timestamp to form mixed data; Step 4: Compress the mixed data. During compression, key frames and key audio clips are compressed using lossless compression, and other signals are compressed using lossy compression. At the same time, the compression ratio is adjusted through real-time communication bandwidth optimization. The compressed data is then packaged into an audio and video data stream for transmission.

2. The method for processing audio and video data of an arrow-borne camera with integrated audio and video according to claim 1, characterized in that: In the step 1, the video acquisition module acquires video signal data at a dynamic frame rate, and after the audio acquisition module acquires the audio signal data, it also includes the steps of noise reduction and waveform optimization.

3. The method for processing audio and video data of an arrow-borne camera with integrated audio and video according to claim 1, characterized in that: In the step 2, key frames are identified according to the motion complexity and inter-frame variation characteristics of the video signal data.

4. The method for processing audio and video data of an arrow-borne camera with integrated audio and video according to claim 1, characterized in that: In the step three, the compressed audio signal data and video signal data are mixedly encoded through multi-channel encoding, and priority tags and time stamps are added to form mixed data.

5. The method for processing audio and video data of an arrow-borne camera with integrated audio and video according to claim 1, characterized in that: In step 4, the specific method of adjusting the compression ratio by optimizing the communication bandwidth in real time is: setting the communication bandwidth reference M and the reference compression ratio C, obtaining the real-time communication bandwidth M1, and adjusting the compression ratio C1 of the lossy compression method to: C1=A×C×M1 / M; Among them, A is the adjustment coefficient.

6. The method for processing audio and video data of an arrow-borne camera with integrated audio and video according to claim 1, characterized in that: The following steps are also included: Step 5: After receiving the audio and video data stream, the ground station decodes and synchronously reconstructs it based on the priority tag and timestamp to achieve data restoration.

7. The method for processing audio and video data of an arrow-borne camera with integrated audio and video according to claim 6, characterized in that: In step 5, the specific method of decoding and synchronously reconstructing based on the priority tag and timestamp to achieve data restoration is: Decompress the key data in the audio and video data stream according to the priority tag; ensure the synchronization of the reconstructed audio and video according to the timestamp information.

8. The method for processing audio and video data of an arrow-borne camera with integrated audio and video according to claim 6, characterized in that: The step five also includes the following steps: decoding the corresponding key feature point content according to the input decoding command.

9. An arrow-borne camera audio and video data processing device, characterized in that: comprising a processor and a memory in communication with the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method for processing audio and video data of an audio and video integrated arrow-borne camera as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method for processing audio and video data of an audio and video integrated arrow-borne camera as described in any one of claims 1 to 6.

Images