Audio and video transmission system, audio and video apparatus, and wireless microphone emission apparatus

Abstract

Provided in the embodiments of the present description are an audio and video transmission system, an audio and video apparatus, and a wireless microphone emission apparatus. The system comprises a wireless microphone emission apparatus and an audio and video apparatus, wherein the audio and video apparatus may acquire in real time first timecode data, which is carried by video data collected by a photographic apparatus, and send the currently acquired first timecode data to the wireless microphone emission apparatus; and the wireless microphone emission apparatus may update a local clock on the basis of the received first timecode data, so as to synchronize the local clock with a clock of the photographic apparatus, and on the basis of the synchronized local clock, perform coding processing on original audio data collected by the wireless microphone emission apparatus itself, so as to obtain audio data carrying second timecode data. Since an original transmission link in an audio and video transmission system can be directly reused for transmitting timecode data, so as to realize clock alignment without the need for changing a hardware structure of a device, convenience and rapidness are achieved, and implementation is easy.

Description

Audio and video transmission system, audio and video device, and wireless microphone transmitting device Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to an audio and video transmission system, an audio and video device, and a wireless microphone transmitting device. Background Art

[0002] The production of audio and video content requires capturing video data and collecting audio data, which are then synchronized to produce the final audio and video file. For scenarios where the subject is far away from the camera, the subject is typically equipped with a wireless microphone transmitter, which captures the subject's audio data. This transmitter then transmits the audio data to a wireless microphone receiver located on one side of the camera. The receiver then inputs the received audio data into the camera, allowing the camera to mix the captured video data with the received audio data to produce mixed audio and video data, which is then output.

[0003] Because there is a certain delay in the transmission of audio data from the wireless microphone transmitter to the camera, the audio and video data recorded by the camera also experience a certain delay, resulting in poor synchronization between the two. Furthermore, because audio data is transmitted via wireless technology, some audio data may be lost during the wireless transmission process. If audio data is lost, the recorded audio and video data may be unusable for a long period of time. Therefore, for the above scenarios, a solution is needed to more accurately synchronize audio and video data.

[0004] Summary of the Invention

[0005] Based on this, the present invention provides an audio and video transmission system, an audio and video device, and a wireless microphone transmitting device.

[0006] According to a first aspect of an embodiment of this specification, there is provided an audio and video transmission system, the system comprising a wireless microphone transmitting device and an audio and video device, the wireless microphone transmitting device being wirelessly connected to the audio and video device;

[0007] The audio and video device is used to connect to an external camera device and obtain first time code data carried by video data collected by the camera device in real time, and send the currently obtained first time code data to the wireless microphone transmitting device;

[0008] The wireless microphone transmitting device is used to update the local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and to encode the original audio data collected by the wireless microphone itself based on the synchronized local clock to obtain audio data carrying the second time code data.

[0009] According to the second aspect of the embodiments of this specification, an audio and video device is provided, which is connected to an external camera device, and the audio and video device is wirelessly connected to a wireless microphone transmitting device. The audio and video device is used to obtain first time code data carried by video data collected by the camera device in real time, and send the currently obtained first time code data to the wireless microphone transmitting device, so that the wireless microphone transmitting device updates the local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and encodes the original audio data collected by the wireless microphone itself based on the synchronized local clock to obtain audio data carrying second time code data.

[0010] According to the third aspect of the embodiments of this specification, a wireless microphone transmitting device is provided, which is wirelessly connected to an audio and video device, and the audio and video device is connected to an external camera device. The wireless microphone transmitting device is used to receive first time code data carried by video data collected by the camera device and sent by the audio and video device, and to update a local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and to encode the original audio data collected by itself based on the synchronized local clock to obtain audio data carrying second time code data.

[0011] By applying the embodiment scheme of this specification, an audio and video transmission system is provided, which includes a wireless microphone transmitter and an audio and video device. The audio and video device can obtain first time code data carried by video data collected by a camera device in real time, and send the currently obtained first time code data to the wireless microphone transmitter; the wireless microphone transmitter can update the local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and encode the original audio data collected by itself based on the synchronized local clock to obtain audio data carrying second time code data. By using the existing devices in the system and the transmission link between the devices, the clock of the camera device and the clock of the wireless microphone transmitter are synchronized, and then the synchronized clock is used to obtain audio data carrying time code data and video data carrying time code data, so that the two types of data can be aligned with each other with the help of the time code data. Since the original transmission link in the audio and video transmission system can be directly reused, there is no need to change the hardware structure of the device, which is convenient, fast and easy to implement.

[0012] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0014] FIG1 is a schematic diagram of an audio and video acquisition application scenario according to an embodiment of this specification.

[0015] FIG2 is a schematic diagram of an embodiment of the present specification of using a time code device to achieve clock synchronization between a camera device and a wireless microphone transmitting device.

[0016] FIG3 is a schematic diagram of an audio and video transmission system according to an embodiment of the present specification.

[0017] FIG4 is a schematic diagram of an audio and video transmission system according to an embodiment of this specification.

[0018] FIG5 is a schematic diagram of the logical structure of an audio and video transmission system according to an embodiment of this specification.

[0019] FIG6 is a schematic diagram of the logical structure of an audio and video transmission system according to an embodiment of this specification.

[0020] FIG7 is a schematic diagram of the logical structure of an audio and video transmission system according to an embodiment of this specification.

[0021] FIG8 is a schematic diagram of an audio and video transmission system according to an embodiment of this specification.

[0022] FIG. 9 is a schematic diagram of a first time code data transmission delay according to an embodiment of this specification. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0024] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. As used in this specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0025] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information without departing from the scope of this specification. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0026] In the process of producing audio and video content, it is necessary to complete the shooting of video data and the collection of audio data, and then synchronize the two to produce the final audio and video. Considering the scenario where the subject is far away from the camera device, if the built-in microphone of the camera device is directly used to collect the audio data of the subject, the effect is poor. Therefore, as shown in Figure 1, the subject is usually asked to wear a wireless microphone transmitter, which can collect the audio data of the subject, and then send the audio data of the subject to a wireless microphone receiving device located on one side of the camera device. The wireless microphone receiving device inputs the received audio data into the camera device so that the camera device can mix the collected video data and the received audio data to obtain audio and video mixed data, and then output the audio and video mixed data.

[0027] Because there's a certain delay in the transmission of audio data from the wireless microphone transmitter to the camera, there's also a delay between the audio and video data recorded by the camera. This often results in poor synchronization between the audio and video data in the resulting mixed audio and video data. Furthermore, because audio data is transmitted wirelessly, some audio data may be lost during the wireless transmission process. This loss can render the recorded audio and video data unusable for an extended period of time.

[0028] In the related art, when achieving data synchronization between two devices, a common method is to use an external time code device to synchronize the clocks of the two devices, and then obtain data carrying time code based on the synchronized clock, so as to synchronize the data collected by the two devices based on the time code. For example, as shown in Figure 2, in order to achieve accurate synchronization of audio data collected by a wireless microphone transmitter and video data collected by a camera, one method that is easy to think of is to connect the wireless microphone transmitter and the camera to an external time code device respectively. The two external time codes can be connected by wire or wirelessly. The two time code devices can synchronize the clocks of the wireless microphone transmitter and the camera. Then, the wireless microphone transmitter can obtain audio data carrying time code based on the synchronized clock code, and the camera can obtain video data carrying time code based on the synchronized clock code. Later, the audio data and video data can be aligned based on the time code carried in the two data. Although this method can achieve accurate synchronization of audio data and video data, the addition of two additional time code devices makes the entire system too complicated.

[0029] Since in the scene of video shooting, directors, producers, etc. usually also need to view the audio and video shot in real time, therefore, a set of wireless video transmission devices is usually set in the system to transmit the audio and video mixed data recorded by the camera device to the terminal equipment (such as a monitoring station or a production station) of the director, producer or other staff members so that these staff members can view the recording results in real time. For example, as shown in Figure 3, in some scenarios, the camera device can usually also be connected to a wireless video transmitter. Usually, the camera device can be connected to the wireless video transmitter via a video transmission interface such as SDI / HDMI, and then the audio and video mixed data can be transmitted to the wireless video transmitter via the above-mentioned video transmission interface. The wireless video transmitter sends the mixed audio and video data to the wireless video receiving device, and the wireless video receiving device can send the audio and video data to one or more terminal devices for display to relevant staff members.

[0030] Alternatively, in some scenarios, as shown in Figure 4, in order to simplify the system, the functions of the wireless microphone receiving device and the functions of the wireless video transmitting device can also be integrated into one device (hereinafter referred to as an integrated device). The integrated device can obtain audio data from the wireless microphone transmitting device and then input it into the camera device. At the same time, the audio and video mixed data output by the camera device can be obtained through the integrated device and then sent to the wireless video receiving device.

[0031] Considering the above-mentioned video shooting scenario, since the wireless microphone transmitter and the camera device are both in a system, that is, the wireless microphone transmitter and the camera device can be connected through a series of devices. In addition, since the wireless video transmitter (or the above-mentioned integrated device) itself is provided with a video data transmission interface, it can obtain video data from the camera device, and the video data carries time code data (hereinafter referred to as the first time code data). Therefore, the time code data can be directly obtained from the video data and transmitted to the wireless microphone transmitter, so that the wireless microphone transmitter can synchronize the local clock with the camera device based on the time code data. Therefore, in order to simplify the entire system, the clocks of the two can be directly synchronized with the existing devices in the system and the transmission links between the devices, and then the synchronized clocks can be used to obtain audio data carrying time code data and video data carrying time code data. Since the original transmission links in the system can be directly reused, there is no need to change the hardware structure of the device, which is convenient, fast and easy to implement.

[0032] Based on this, an embodiment of the present application provides an audio and video transmission system, as shown in Figure 5, the audio and video transmission system includes a wireless microphone transmitting device 11 and an audio and video device 12, wherein the wireless microphone transmitting device 11 can be located on the side of the subject being photographed, and the audio and video device 12 can be located on the side of the camera device 13, and the two can be wirelessly connected.

[0033] The wireless microphone transmitting device 11 may include one or more devices (e.g., when the subject is a multi-person scene). For example, the wireless microphone transmitting device 11 may be worn by the subject to collect audio data from the subject, and then the audio and video data may be compressed and encoded before being wirelessly transmitted to the wireless microphone receiving device. Alternatively, the wireless microphone transmitting device 11 may store the collected audio data locally.

[0034] The audio and video device 12 can be connected to the external camera device 13 in a communication manner, and the two can be connected in a wired or wireless manner. For example, in some scenarios, the two can be connected by a wired connection through some hardware interfaces set on the device, such as HDMI, SDI, USB, etc., to transmit video data.

[0035] The audio and video device 12 can obtain the first time code data carried by the video data captured by the camera device 13 in real time, and send the currently obtained first time code data to the wireless microphone transmitter 11. For example, in some scenarios, the audio and video device 12 can be directly connected to the camera device 13 through a hardware interface and obtain the first time code data output by the camera device 13 in real time. In some scenarios, the audio and video device 12 can also be indirectly connected to the camera device 13. For example, the audio and video device 12 can be connected to other devices connected to the camera device 13 to obtain the first time code data from the other devices. After obtaining the first time code data, the audio and video device 12 can send the first time code data to the wireless microphone transmitter 11 via various wireless transmission technologies (such as Bluetooth, WiFi, Zigbee, etc.).

[0036] After receiving the first time code data, the wireless microphone transmitter 11 can use the first time code data to update the local clock to synchronize the local clock with the clock of the camera 13. The wireless microphone transmitter 11 can then encode the original audio data it has collected based on the synchronized local clock to obtain audio data carrying the time code data (hereinafter referred to as the second time code data).

[0037] Timecode data can be used to precisely identify which data was captured at which time. For example, the timecode for video data consists of the following: hour:minute:second:frame. For example, if the timecode is 9:00:00:1, it means the first frame was captured at 9:00:00. Audio data can also be labeled using a similar method.

[0038] In some embodiments, the audio and video device 12 can be a device that only has the function of wirelessly receiving audio data, that is, the audio and video device 12 is similar to the wireless microphone receiving device in Figure 3 above, which can be used to receive audio data from the wireless microphone transmitting device 11 and then input it to the camera device 13, so that the camera device 13 can mix the audio data with the collected video data to obtain audio and video mixed data.

[0039] In some embodiments, the audio device 12 may be a device that has both the function of wirelessly receiving audio data and the function of wirelessly sending video data, that is, the audio and video device 12 may be a device that integrates the functions of a wireless microphone receiving device and a wireless video transmitting device into one device, similar to the integrated device in Figure 4 above.

[0040] In some embodiments, if the audio / video device 12 only has a wireless audio data receiving function, in order to remotely transmit the audio / video data captured by the camera device 13 to a terminal device of a director, producer, or other staff member, as shown in FIG6 , the audio / video system may further include a wireless video transmitter 14. The wireless video transmitter 14 may be wired to the camera device 13 and the audio / video device 12, respectively. After receiving the audio data, the audio / video device 12 may transmit the audio data to the camera device 13. The camera device 13 then mixes the captured video data with the audio / video data to generate the mixed audio / video data, which it then transmits to the wireless video transmitter 14 via a hardware interface. The wireless video transmitter 14 may then extract the first time code data carried in the mixed audio / video data and transmit the first time code data to the audio / video device 12, which then transmits the first time code data to the wireless microphone transmitter 11. Furthermore, the wireless video transmitter 14 may compress and encode the mixed audio / video data before transmitting it to the terminal device 15.

[0041] In some implementations, the wireless video transmitter 14 can be connected to the camera 13 via an SDI interface or an HDMI interface, and the wireless video transmitter 14 can be connected to the audio and video device 12 via a USB interface. Of course, the above are only illustrative examples, and other types of interfaces can also be set according to actual needs when implementing the above data transmission.

[0042] In some embodiments, as shown in FIG7 , if the audio / video device 12 has both the function of wirelessly receiving audio data and the function of wirelessly transmitting video data, the audio / video device 12 can receive audio data from the wireless microphone transmitter 11 and then transmit it to the camera 13. Furthermore, the device can obtain mixed audio / video data from the camera 13, extract the first time code data carried by the video data from the mixed audio / video data, and transmit the first time code data to the wireless microphone transmitter 11. Furthermore, the audio / video device 12 can compress and encode the mixed audio / video data before transmitting it to the terminal device 15.

[0043] In some embodiments, the audio and video transmission system may further include a wireless video receiving device. The audio and video device 12 or the wireless video transmitting device 14 may transmit the mixed audio and video data to the wireless video receiving device, and transmit the data to the terminal device 15 via the wireless video receiving device.

[0044] In some embodiments, considering the transmission delay associated with the first time code data output from camera 13 and then transmitted to wireless microphone transmitter 11 via the transmission link, wireless microphone transmitter 11 can first determine the transmission delay associated with the first time code data being transmitted from camera 13 to wireless microphone transmitter 11 when updating its local clock based on the received first time code data. It can then determine a target time based on the time contained in the first time code data and the transmission delay, and then update its local clock based on the deviation between the target time and the local system time of wireless microphone transmitter 11.

[0045] In some embodiments, a trigger condition may be preset, and the audio and video device 12 may send the currently acquired first time code data to the wireless microphone transmitting device 11 after the preset condition is triggered.

[0046] In some embodiments, the preset condition may be that the video recording state of the camera 13 switches between an on state and an off state. For example, when the camera 13 starts video recording, the wireless microphone transmitter 11 typically also needs to start synchronously collecting audio data. Therefore, the first time code data carried by the video data can be sent to the wireless microphone transmitter 11 so that the wireless microphone transmitter 11 synchronizes its local clock with the clock of the camera 13 and then generates the second time code data of the audio data based on the synchronized clock. This allows the first time code data carried by the video data and the second time code data carried by the audio data to be subsequently aligned based on the two.

[0047] In some embodiments, even if the clocks of the camera device 13 and the wireless microphone transmitter 11 have been synchronized once when video recording begins, as time goes by, the clocks of the two will gradually deviate, and the deviation may be larger when the time duration is longer. For example, assuming that the wireless microphone transmitter uses an RTC clock with an accuracy of ±10PPM, the frequency difference is: 32768.32768Hz ~ 32767.67232Hz, the daily time error is 0.864×1=0.864 seconds, and the total deviation over a year is 5.256 minutes. Assuming that the audio recording time is 4 hours and the local RTC clock deviation is 144ms, if a clock with a large deviation is used to generate time code data, the audio and video will eventually be out of sync, resulting in problems with lip movements and voice mismatch.

[0048] To avoid significant deviations between the local clocks of the camera device 13 and the wireless microphone transmitter 11, synchronization of the two clocks can be performed at preset intervals. Therefore, the preset condition can also be that the time since the last time the first time code data was sent to the wireless microphone transmitter 11 has reached a preset time. For example, the audio and video device 12 can send the current first time code data to the wireless microphone transmitter 11 at preset intervals, so that the wireless microphone transmitter 11 updates its local clock based on the first time code data. The preset duration can be set based on an acceptable delay between the audio and video data. For example, the preset duration can be set to 5 seconds. Assuming the wireless microphone transmitter uses an RTC clock with an accuracy of ±10 PPM and a frequency difference of 32768.32768 Hz to 32767.67232 Hz, if synchronization is achieved every 5 seconds, the clock deviation between the two can be controlled within 50 μs, which is within an acceptable range.

[0049] Of course, in actual processing, the preset duration can be adjusted based on the requirements for audio and video synchronization accuracy. If you want to achieve higher synchronization accuracy, you can set the preset duration longer, otherwise, you can set it shorter.

[0050] In the related art, when using a wireless microphone transmitter 11 to collect audio data and a camera 13 to collect video data, there may usually be a large deviation between the time when recording is started and when it is stopped. For example, it is often possible that the video recorder presses the "start recording" button on the camera 13 and then notifies the subject that recording has started, and then the subject turns on the wireless microphone transmitter 11. Alternatively, the subject turns on the wireless microphone transmitter 11 and notifies the recorder, and then the recorder starts recording the video. It is even possible that after recording has already started, the subject forgets to turn on the wireless microphone transmitter 11. In other words, current technology cannot achieve rapid synchronous recording or pausing of audio and video, resulting in the need for manual editing to eliminate some audio data without video, or video data without audio, which is relatively cumbersome.

[0051] In order to avoid the above problems, in some embodiments, the audio and video device 12 can send a first trigger signal to the wireless microphone transmitting device 11 when it determines that the video recording state of the camera device 13 is switched from the off state to the on state. The first trigger signal is used to control the wireless microphone transmitting device 11 to turn on the audio recording function.

[0052] The audio and video device 12 can send a second trigger signal to the wireless microphone transmitting device 11 when it is determined that the video recording state of the camera device 13 is switched from the on state to the off state. The second trigger signal is used to control the wireless microphone transmitting device 11 to turn off the audio recording function.

[0053] The embodiment of the present application reuses the existing transmission link in the system. When the camera device starts video recording or ends video recording, a trigger signal is sent to the wireless microphone transmitting device 11 to control the wireless microphone transmitting device 11 to synchronously start audio recording or end audio data. Since the signal transmission time is short, the two can start recording or end recording synchronously.

[0054] In some embodiments, the audio / video device 12 can obtain a status indication signal in real time indicating the current video recording state of the camera 13, and determine whether the video recording state of the camera 13 has switched between an on and off state based on the status indication signal. For example, if the audio / video device 12 is directly connected to the camera 13, the status indication signal can be obtained directly from the camera 13. If the audio / video device 12 is connected to the wireless video transmitter 14, the wireless video transmitter 14 can obtain the status indication signal from the camera 13 and then transmit it to the audio / video device 12.

[0055] Considering that if audio data and video data are to be aligned based on time code data, when encoding the captured original audio data, the frame rates of the audio data or video data carrying the time code data must be consistent. Therefore, in some embodiments, the audio and video device 12 is further configured to obtain the frame rate of the video data captured by the camera device 13 in real time, and upon detecting that the video recording state of the camera device 13 has switched from an off state to an on state, transmit the frame rate to the wireless microphone transmitting device 11, so that the wireless microphone transmitting device 11 can encode the audio data based on the frame rate to obtain audio data carrying the second time code data.

[0056] Among them, considering that after the frame rate is set, the frame rate usually remains unchanged during the video recording process, after obtaining the frame rate from the camera device 13, the frame rate can be sent to the wireless microphone transmitting device 11 when it is detected that the recording state of the camera device 13 is switched from the off state to the on state (that is, recording starts), so that the wireless microphone transmitting device 11 can collect the original audio data based on the frame rate and encode the collected original audio data.

[0057] Of course, in actual applications, the audio and video device 12 can also obtain the frame rate of the video data in real time. If a change in the frame rate is detected, the changed frame rate is sent to the wireless microphone transmitting device 11, so that the wireless microphone transmitting device 11 will collect the original audio data based on the changed frame rate and perform encoding processing.

[0058] In the past, the wireless microphone transmitter 11 usually sent the collected audio data to the wireless microphone receiver, which then input the received audio data into the camera 13. The camera 13 then mixed the collected video data and audio data to obtain audio-video mixed data, and then stored the audio-video mixed data locally. In this scenario, there may be a problem of audio data being lost during the wireless transmission process, resulting in the final stored audio-video mixed data being unusable. To avoid the above problem, in some implementations, the wireless microphone transmitter 11 encodes the original audio data collected by itself based on the synchronized local clock to obtain audio data carrying second time code data. The audio data carrying the second time code data can then be stored locally. Later, during audio and video production, the audio data carrying the second time code data and the video data carrying the first time code data can be imported into some audio and video production software. Then, the one-click time code alignment function can be used to align the two based on the time code data carried by the two to obtain an audio and video file.

[0059] In addition, the present application also provides an audio and video device, which is connected to an external camera device, and the audio and video device is wirelessly connected to a wireless microphone transmitting device. The audio and video device is used to obtain the first time code data carried by the video data collected by the camera device in real time, and send the currently obtained first time code data to the wireless microphone transmitting device, so that the wireless microphone transmitting device updates the local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and encodes the original audio data collected by itself based on the synchronized local clock to obtain audio data carrying the second time code data.

[0060] In some embodiments, the audio and video device is used to send the currently acquired first time code data to the wireless microphone transmitting device after a preset condition is triggered.

[0061] In some embodiments, the preset condition includes any one of the following:

[0062] The time duration from the last time the first time code data was sent to the wireless microphone transmitting device reaches a preset time duration;

[0063] The video recording state of the camera device is switched between an on state and an off state.

[0064] In some embodiments, the audio and video device is further configured to send a first trigger signal to the wireless microphone transmitting device when determining that the video recording state of the camera device is switched from an off state to an on state, wherein the first trigger signal is used to control the wireless microphone transmitting device to start an audio recording function;

[0065] And when it is determined that the video recording state of the camera device is switched from the on state to the off state, a second trigger signal is sent to the wireless microphone transmitting device, and the second trigger signal is used to control the wireless microphone transmitting device to turn off the audio recording function.

[0066] In some embodiments, the audio and video device is also used to obtain in real time a status indication signal indicating the current video recording status of the camera device, and determine based on the status indication signal whether the video recording status of the camera device has switched between the on state and the off state.

[0067] In some embodiments, the audio and video device is also used to obtain the frame rate of the camera device in real time, and when it is determined that the video recording state of the camera device is switched from the off state to the on state, the frame rate is sent to the wireless microphone transmitting device, so that the wireless microphone transmitting device is used to encode the audio data based on the frame rate to obtain audio data carrying second time code data.

[0068] In some embodiments, the audio and video device is also used to receive audio data sent by the wireless microphone transmitting device, and transmit the audio data to the camera device, so that the camera device mixes the audio data and video data to obtain audio and video mixed data; and is used to obtain the audio and video mixed data from the camera device, and send the audio and video mixed data to the terminal device.

[0069] The specific functions of the audio and video device can be found in the descriptions of the above-mentioned audio and video system embodiments, which will not be repeated here.

[0070] In addition, the present application also provides a wireless microphone transmitting device, which is wirelessly connected to an audio and video device, and the audio and video device is connected to an external camera device. The wireless microphone transmitting device is used to receive the first time code data carried by the video data collected by the camera device and sent by the audio and video device, and update the local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and encode the original audio data collected by itself based on the synchronized local clock to obtain audio data carrying second time code data.

[0071] In some embodiments, when the wireless microphone transmitting device is used to update the local clock based on the received first time code data, it is specifically used to:

[0072] determining a target time based on the time carried in the first time code data and a transmission delay of the first time code data, wherein the transmission delay is a delay generated by transmitting the first time code data from the camera device to the wireless microphone transmitting device;

[0073] Based on the deviation between the target time and the local system time of the wireless microphone transmitting device, the local clock of the wireless microphone transmitting device is updated.

[0074] In some embodiments, the wireless microphone device is used to turn on the audio recording function when it receives a first trigger signal sent by the audio and video device; and to turn off the audio recording function when it receives a second trigger signal sent by the audio and video device; wherein, when the audio and video device determines that the video recording state of the camera device is switched from the off state to the on state, it sends a first trigger signal to the wireless microphone transmitting device, and when it determines that the video recording state of the camera device is switched from the on state to the off state, it sends a second trigger signal to the wireless microphone transmitting device.

[0075] In some embodiments, the wireless microphone transmitting device is further used to store audio data carrying second time code data locally, wherein the stored audio data carrying second time code data is used to align with video data carrying first time code to produce an audio and video file.

[0076] Among them, the specific functions of the wireless microphone transmitting device can be referred to the description in the above-mentioned audio and video system embodiments, and will not be repeated here.

[0077] In order to further explain the audio and video transmission system provided by the embodiment of the present application, it is explained below in conjunction with a specific embodiment.

[0078] As shown in FIG8 , an audio and video transmission system according to an embodiment of the present application is shown. The system includes a wireless microphone transmitter, an audio and video device, a wireless video transmitter, and a wireless video receiver. The audio and video device and the wireless video transmitter are respectively connected to the camera device by wires, the audio and video device and the wireless video transmitter are connected by wires, the audio device is wirelessly connected to the wireless microphone transmitter, and the wireless video transmitter is wirelessly connected to the wireless video receiver. The process of achieving audio and video data synchronization in the above-mentioned audio and video transmission system is as follows:

[0079] (1) After the camera starts recording video, it can store video data with timecode.

[0080] (2) The wireless video transmitter is connected to the camera device via an HDMI or SDI interface. The video data transmitted by the camera device may carry a timecode, a frame rate, and a status indicator signal indicating the current video recording state, indicating whether the camera device is currently recording or stopped. For ease of description, the above three types of data are referred to as video auxiliary data. The wireless video transmitter can parse the video data output by the camera device to obtain video auxiliary data.

[0081] (3) The wireless video transmitter and the audio / video device are connected via a USB interface, wherein the wireless video transmitter acts as a USB master device and the audio / video device acts as a USB slave device, and the two communicate using USB HID. The wireless video transmitter encapsulates the video auxiliary data corresponding to each frame of collected video data into a USB HID packet and sends it to the audio / video device.

[0082] (4) The audio and video device and the wireless microphone transmitter use the TDMA protocol for wireless air interface transmission, wherein the audio and video device is the main device and is connected to one or more wireless microphone transmitters. The audio and video device can generate a corresponding trigger signal when it determines based on the status indication signal that the video recording state of the camera device has changed (that is, when the camera device turns on or off recording). When it is determined that the video recording state of the camera device is switched from the off state to the on state, the trigger signal is used to control the wireless microphone transmitter to turn on audio recording, and when it is determined that the recording state of the camera device is switched from the on state to the off state, the trigger signal is used to control the wireless microphone transmitter to turn off audio recording.

[0083] The audio and video device encodes and packages the trigger signal, timecode, and frame rate, and then sends it out via 2.4G wireless broadcast within the wireless time slot to which it belongs.

[0084] (5) After the wireless microphone transmitter receives the wireless microphone receiving trigger signal, timecode, and frame rate, it starts or stops audio recording according to the trigger signal. If audio recording is turned on, the local clock is updated using the timecode to synchronize the local clock with the camera clock. Then, the collected raw audio data is LTC-encoded based on the synchronized clock and frame rate to obtain audio data with the timecode, and the audio data with the timecode is stored locally.

[0085] When using the video data's timecode to synchronize the local clock, the transmission delay of the video data's timecode from the camera to the wireless microphone transmitter must be considered. Figure 9 shows a schematic diagram of this transmission delay. Assuming the camera starts recording at T0, meaning the video data's timecode records the first frame of video captured at T0, the time the wireless microphone transmitter triggers audio recording is: T0 + T1 + T2 + T3 + T4.

[0086] T1 is the time it takes for the wireless video transmitter to receive the video auxiliary data, align it, parse it, and forward it to the audio and video device. T1 can be determined by statistical test data with an error range of microseconds.

[0087] T2 is the sum of the time it takes for the audio and video device to wirelessly transmit the trigger signal, timecode, and frame rate to the wireless microphone transmitter, as well as the USB packet unpacking time. Since TDMA wireless transmission technology is used, the wireless transmission time is a constant, and the USB packet unpacking time is also a constant. Therefore, T2 can be determined.

[0088] T3 is the Timecode encoding time, which is also a fixed value.

[0089] T4 is the deviation between the RTC clock of the wireless microphone transmitter and the RTC clock of the camera device, where T0+T1+T2+T3 is the time when data is transmitted to the wireless microphone transmitter. At this time, it can be compared with the local system time of the wireless microphone transmitter, and then the clock deviation T4 between the camera device and the wireless microphone transmitter can be calculated.

[0090] (6) The wireless microphone transmitter uses a high-precision RTC. After synchronizing the local clock with the time code of the camera device, the RTC clock inside the wireless microphone transmitter can be used to generate the timecode and record the audio data with the timecode. However, considering that the clocks of the camera device and the wireless microphone transmitter may deviate greatly over time, in order to ensure that the clocks of the camera device and the wireless microphone transmitter are stably synchronized. The audio and video device can send the timecode of the currently collected video data to the wireless microphone transmitter every 5 seconds. After receiving the timecode, the wireless microphone device can synchronize the clock again to ensure that the local RTC clock is synchronized with the clock on the camera end, thereby avoiding the problem of timecode deviation between the video material and the audio material.

[0091] (7) In addition, in order to enable directors, producers, etc. to remotely view the recorded audio and video mixed data. The wireless microphone transmitter can wirelessly send the collected audio data to the audio and video device, which can be input to the camera device through the audio input interface. The camera device can mix the audio data and video data to obtain audio and video mixed data (the synchronization effect of the audio and video mixed data obtained in this way is poor). The audio and video mixed data can then be output to the wireless video transmitter, which can compress and encode the audio and video mixed data and send it to the wireless video receiver, which will then send it to the terminal equipment of each staff member for viewing.

[0092] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of the various technical features in the above embodiments also falls within the scope of disclosure of this specification.

[0093] Those skilled in the art will readily recognize other implementations of the embodiments of the present invention after considering the specification and practicing the instructions disclosed herein. The embodiments of the present invention are intended to cover any variations, uses, or adaptations of the embodiments of the present invention that follow the general principles of the embodiments of the present invention and include common knowledge or customary techniques in the art not disclosed in the embodiments of the present invention. The description and examples are to be considered as exemplary only, and the true scope and spirit of the embodiments of the present invention are indicated by the following claims.

[0094] It should be understood that the embodiments of the present invention are not limited to the precise structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the embodiments of the present invention is limited only by the appended claims.

[0095] The above description is only a preferred embodiment of the embodiments of this specification and is not intended to limit the embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of this specification should be included in the scope of protection of the embodiments of this specification.

Claims

1. An audio and video transmission system, characterized in that: The system comprises a wireless microphone transmitting device and an audio and video device, wherein the wireless microphone transmitting device is wirelessly connected to the audio and video device; The audio and video device is used to connect to an external camera device, and obtain the first time code data carried by the video data collected by the camera device in real time, and send the currently obtained first time code data to the wireless microphone transmitting device; The wireless microphone transmitting device is used to update the local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and to encode the original audio data collected by itself based on the synchronized local clock to obtain audio data carrying the second time code data.

2. The system according to claim 1, characterized in that When the wireless microphone transmitting device is used to update the local clock based on the received first time code data, it is specifically used to: Determine the target time based on the time carried in the first time code data and the transmission delay of the first time code data, wherein the transmission delay is the delay generated by the transmission of the first time code data from the camera device to the wireless microphone transmitting device; Based on the deviation between the target time and the local system time of the wireless microphone transmitting device, the local clock of the wireless microphone transmitting device is updated.

3. The system according to claim 1, characterized in that The audio and video device is used to send the currently acquired first time code data to the wireless microphone transmitting device after being triggered by a preset condition.

4. The system according to claim 3, characterized in that The preset conditions include any one or more of the following: The time length from the last time the first time code data was sent to the wireless microphone transmitting device reaches a preset time length; The video recording state of the camera device is switched between an on state and an off state.

5. The system according to claim 1, characterized in that The audio and video device is also used to send a first trigger signal to the wireless microphone transmitting device when it is determined that the video recording state of the camera device is switched from the off state to the on state, and the first trigger signal is used to control the wireless microphone transmitting device to start the audio recording function; And when it is determined that the video recording state of the camera device is switched from the on state to the off state, a second trigger signal is sent to the wireless microphone transmitting device, and the second trigger signal is used to control the wireless microphone transmitting device to turn off the audio recording function.

6. The system according to claim 4 or 5, characterized in that: The audio and video device is also used to obtain in real time a status indication signal indicating the current video recording status of the camera device, and determine based on the status indication signal whether the video recording status of the camera device has switched between an on state and an off state.

7. The system according to claim 5, characterized in that The audio and video device is also used to obtain the frame rate of the camera device in real time, and send the frame rate to the wireless microphone transmitting device when it is determined that the video recording state of the camera device is switched from the off state to the on state; The wireless microphone transmitting device is used to encode the audio data based on the frame rate to obtain audio data carrying second time code data.

8. The system according to any one of claims 1 to 7, characterized in that: The system also includes a wireless video transmitting device, which is respectively connected to the camera device and the audio and video device by wire, and is used to obtain the first time code data from the camera device and transmit the first time code data to the audio and video device.

9. The system according to claim 8, characterized in that The audio and video device is also used to receive audio data sent by the wireless microphone transmitting device, and transmit the audio data to the camera device, so that the camera device mixes the audio data and video data to obtain audio and video mixed data, and sends the audio and video mixed data to the terminal device through the wireless video transmitting device.

10. The system according to claim 8, characterized in that The wireless video transmitting device is connected to the camera device via an SDI interface or an HDMI interface, and the wireless video transmitting device is connected to the audio and video device via a USB interface.

11. The system according to any one of claims 1 to 7, characterized in that: The audio and video device is also used to receive audio data sent by the wireless microphone transmitting device, and transmit the audio data to the camera device, so that the camera device mixes the audio data and video data to obtain audio and video mixed data; and is used to obtain the audio and video mixed data from the camera device and send the audio and video mixed data to the terminal device.

12. The system according to claim 9 or 11, characterized in that The system also includes a wireless video receiving device, and the wireless video transmitting device or the audio and video device is used to send the audio and video mixed data to the wireless video receiving device, so as to send the audio and video mixed data to the terminal device through the wireless video receiving device.

13. The system according to claim 1, characterized in that The wireless microphone transmitting device is also used to store audio data carrying the second time code data locally, wherein the stored audio data carrying the second time code data is used to align with the video data carrying the first time code to produce an audio and video file.

14. An audio and video device, characterized in that: The audio and video device is connected to an external camera device, and the audio and video device is wirelessly connected to a wireless microphone transmitting device. The audio and video device is used to obtain the first time code data carried by the video data collected by the camera device in real time, and send the currently obtained first time code data to the wireless microphone transmitting device, so that the wireless microphone transmitting device updates the local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and encodes the original audio data collected by itself based on the synchronized local clock to obtain audio data carrying the second time code data.

15. The audio and video device according to claim 14, characterized in that: The audio and video device is used to send the currently acquired first time code data to the wireless microphone transmitting device after being triggered by a preset condition.

16. The audio and video device according to claim 15, characterized in that: The preset conditions include any one or more of the following: The time length from the last time the first time code data was sent to the wireless microphone transmitting device reaches a preset time length; The video recording state of the camera device is switched between an on state and an off state.

17. The audio and video device according to claim 14, characterized in that: The audio and video device is also used to send a first trigger signal to the wireless microphone transmitting device when it is determined that the video recording state of the camera device is switched from the off state to the on state, and the first trigger signal is used to control the wireless microphone transmitting device to start the audio recording function; And when it is determined that the video recording state of the camera device is switched from the on state to the off state, a second trigger signal is sent to the wireless microphone transmitting device, and the second trigger signal is used to control the wireless microphone transmitting device to turn off the audio recording function.

18. The audio and video device according to claim 14, characterized in that: The audio and video device is also used to obtain in real time a status indication signal indicating the current video recording status of the camera device, and determine based on the status indication signal whether the video recording status of the camera device has switched between an on state and an off state.

19. The audio and video device according to claim 14, characterized in that: The audio and video device is also used to obtain the frame rate of the camera device in real time, and when it is determined that the video recording state of the camera device is switched from an off state to an on state, the frame rate is sent to the wireless microphone transmitting device, so that the wireless microphone transmitting device is used to encode the audio data based on the frame rate to obtain audio data carrying second time code data.

20. The audio and video device according to claim 14, characterized in that: The audio and video device is also used to receive audio data sent by the wireless microphone transmitting device, and transmit the audio data to the camera device, so that the camera device mixes the audio data and video data to obtain audio and video mixed data; and is used to obtain the audio and video mixed data from the camera device and send the audio and video mixed data to the terminal device.

21. A wireless microphone transmitting device, characterized in that: The wireless microphone transmitting device is wirelessly connected to the audio and video device, and the audio and video device is connected to an external camera device. The wireless microphone transmitting device is used to receive the first time code data carried by the video data collected by the camera device and sent by the audio and video device, and to update the local clock based on the received first time code data to synchronize the local clock with the clock of the camera device, and to encode the original audio data collected by itself based on the synchronized local clock to obtain audio data carrying the second time code data.

22. The wireless microphone transmitting device according to claim 21, characterized in that: When the wireless microphone transmitting device is used to update the local clock based on the received first time code data, it is specifically used to: Determine the target time based on the time carried in the first time code data and the transmission delay of the first time code data, wherein the transmission delay is the delay generated by the transmission of the first time code data from the camera device to the wireless microphone transmitting device; Based on the deviation between the target time and the local system time of the wireless microphone transmitting device, the local clock of the wireless microphone transmitting device is updated.

23. The wireless microphone transmitting device according to claim 21, characterized in that: The wireless microphone device is used to start the audio recording function when receiving the first trigger signal sent by the audio and video device; And when receiving a second trigger signal sent by the audio and video device, turning off the audio recording function; wherein, when the audio and video device determines that the video recording state of the camera device is switched from the off state to the on state, the audio and video device sends a first trigger signal to the wireless microphone transmitting device, and when it determines that the video recording state of the camera device is switched from the on state to the off state, the audio and video device sends a second trigger signal to the wireless microphone transmitting device.

24. The wireless microphone transmitting device according to claim 21, characterized in that: The wireless microphone transmitting device is also used to store audio data carrying the second time code data locally, wherein the stored audio data carrying the second time code data is used to align with the video data carrying the first time code to produce an audio and video file.

Images