Method and system for managing a first audio / video stream and a second audio / video stream.
The method and system synchronize audio and video streams using timecodes to enable synchronized replay and mixing during post-production, addressing the lack of video access for sound engineers.
Patent Information
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- GRILLERE PASCAL
- Filing Date
- 2024-05-23
- Publication Date
- 2026-04-24
AI Technical Summary
During video or film shooting, sound recording is managed separately from video capture, and sound engineers lack access to video streams during post-production, making it difficult to synchronize audio and video streams for remixing or replay.
A method and system that utilize a primary and secondary device to manage audio and video streams, where the primary device transmits a timecode to the secondary device, allowing the secondary device to record or read streams based on the timecode and recording state commands, ensuring synchronization.
Enables synchronous replay of audio and video streams before post-production, facilitating seamless audio mixing and remixing by maintaining synchronization between the two streams.
Abstract
Description
Title of the invention: Method and system for managing a first audio / video stream and a second audio / video stream. Technical field of the invention
[0001] The present invention relates to a method and a system for managing a first stream and a second stream, the first stream and the second stream each being of the audio or video type. State of the art
[0002] During video or film shooting, the shooting and sound recording are managed separately. The video stream is captured by the camera and the audio stream is captured by a sound recorder.
[0003] On the audio side, the engineer in charge of the audio recording also receives the video stream, which he views on a monitor to guide him in his mixing work during the recording. Once his recording is finished, however, if he listens back to the sound or remixes the take, he no longer has the original video stream.
[0004] There is currently no solution for the sound engineer to have access to the images when he has to replay the captured sound, upstream of the post-production phase.
[0005] The aim of the invention is to propose a suitable solution to address this problem. Description of the invention
[0006] This objective is achieved by a method for managing a first stream and a second stream, the first and second streams each being of audio or video type, the first stream being received as input by a so-called primary device and the second stream being received as input by a so-called secondary device, the first and second streams each being composed of one or more elements, the primary and secondary devices having storage means configured to store each element of the first and second streams as well as an associated timecode, the method comprising: - A step of transmission by the primary device of a time code to the secondary device, said time code being associated with each element of the first stream; - The secondary device is configured to: - Receive the timecode from the primary device, - When it receives a first activation command from the primary device, it enters a recording state for the second stream. each element of the second stream being recorded, associated with the received timecode; - To enter a so-called reading state of each element of the second stream stored in its storage means when the first command is not received or is inactive and the timecode is already stored in its storage means.
[0007] According to a particular feature, the secondary device is configured to remain in a so-called transparent state when the received time code is not stored in its storage means.
[0008] According to another feature, the recording state has an active or inactive status defined by the value of a bit of the timecode sent by the primary device to the secondary device.
[0009] The invention also relates to a system for managing a first stream and a second stream, the first stream and the second stream each being of audio or video type, the first stream being received as input by a so-called primary device and the second stream being received as input by a so-called secondary device, the first stream and the second stream each being composed of one or more elements, the primary device and the secondary device having memorization means configured to memorize each element of the first stream and the second stream as well as an associated time code, - The primary device being configured to emit a time code to the secondary device, said time code being associated with each element of the first stream; - The secondary device is configured to: - Receive the timecode from the primary device (Dl), - When it receives a first activation command from the primary device, switch to a recording state of the second stream, each element of the second stream being recorded, associated with the received time code; - To enter a so-called reading state of each element of the second stream stored in its storage means when the first command is not received or is inactive and the timecode is already stored in its storage means.
[0010] According to a particular feature, the secondary device is configured to remain in a so-called transparent state when the received time code is not stored in its storage means.
[0011] According to another feature, the recording state has an active or inactive status defined by the value of a bit of the timecode sent by the primary device to the secondary device. Brief description of the figures
[0012] Other features and advantages will become apparent in the detailed description that follows, given in relation to the accompanying drawings in which: - Fig. 1 illustrates the principle of implementation of the invention's management system and its operating principle; - Fig. 2A represents a diagram illustrating the algorithm executed by the primary device in the operation of the management process of the invention; - Fig. 2B represents a diagram illustrating the algorithm executed by the secondary device in the operation of the management process of the invention; - Figures 3A and 3B show two variant embodiments of the management system of the invention;
[0013] Detailed description of at least one embodiment
[0014] The invention applies to a system which conventionally comprises a device for recording a first stream, for example a video stream, and a device for recording a second stream, for example an audio stream. The reverse would also be conceivable.
[0015] The invention aims to enable the replay of one stream in synchronization with the other stream, at any time, particularly before post-production. This applies more specifically to the audio stream, which must be able to be replayed by the sound engineer while having the synchronous video stream in parallel.
[0016] For this purpose, with reference to [Fig.1], the system uses a device called primary DI and a device called secondary D2.
[0017] The primary device DI is considered as the master device (or “leader”) and the secondary device D2 is considered as the slave device (or “follower”).
[0018] The primary device DI is configured to: - Generate a TC timecode on its incoming stream FX_1; - Transmit a generated TC timecode to the secondary device D2 in line with the flow being processed; - Record an incoming stream FX_1 with the associated time code TC in its storage means Ml; - Read a previously recorded stream, associated with its TC timecode; - Generate and send commands to the secondary device D2 when the primary device DI switches to an active recording status;
[0019] The secondary device D2 is configured to: - Receive the TC timecode from the primary device DI; - Receive the command on the status of the recording state from the primary device DI; - Record an incoming FX_2 stream when it receives a command from the primary device DI indicating that it is in an active recording status state; - Search for a TC timecode in M2 storage means when it is not in an active recording status; - Read a stream already recorded when it is not in an active recording state and it finds a time code TC in its storage means M2;
[0020] As is known, a timecode is a temporal reference used in the fields of sound and image for the synchronization and marking of recorded material. The timecode (TC) is a temporal reference assigned to each frame of a video in the form of a sequence of numbers generated at regular intervals by a timekeeping system. The data is recorded concurrently with each frame or sound.
[0021] For the remainder of the description, it is thus considered that each FX_1, FX_2 stream can be recorded in the form of one or more elements, a timecode TC being associated with each element of the recorded stream.
[0022] Based on these principles, the process of the invention can be separated into two distinct parts, one of them being associated with the primary device DI and the other being associated with the secondary device D2.
[0023] The primary device DI and the secondary device D2 are each equipped with a processing and control unit. Each processing and control unit is configured to send / receive data / commands and execute instructions according to a specific operating algorithm. It is associated with storage means M_1, M_2 of each device D1, D2.
[0024] The primary device D1 and the secondary device D2 are each implemented in both software and hardware form. In software form, the device is implemented as software instructions executable by its processing and control unit. The processing and control unit may be specific to the device or shared with other existing means.
[0025] For the remainder of the description, it is assumed that: - Each device in the system is capable of switching from a recording state to an active status in which it records the elements of the incoming stream into its storage means; - Each device is capable of switching from a reading state to an active state in which it is capable of reading one or more elements stored in its storage means;
[0026] It should be noted that the active (ON) or inactive (OFF) status of the recording state sent by the primary device DI to the secondary device D2 can be manifested by separate data sent, or by the presence or absence of data.
[0027] By way of non-limitation, the time code TC can be transmitted from the primary device DI to the secondary device D2 via a wired or wireless connection (Bluetooth, Wi-Fi, radio, etc.). Similarly, the signal corresponding to the recording state can be transmitted from the primary device DI to the secondary device D2 in the form of an electrical signal representing the two states and / or, for example, by reusing a bit of the message representing the time code TC.
[0028] In this latter case, advantageously, bit 11, commonly known as the "color frame" of the LTC (Linear Time Code - SMPTE 12M-1-2008 standard), can be used. In this solution, the processing and control unit of the primary device DI is configured to set bit 11 of the time code representative message to 1 when the recording state is active (ON) or to 0 when the recording state is inactive (OFF). Thus, the time code representative message carries both the information relating to the time required for stream synchronization and the information relating to the recording state status (S_rec).
[0029] Figures 2A and 2B show two examples of algorithms that can be executed respectively by the primary device DI and by the secondary device D2. On these diagrams, the letter Y corresponds to the Yes branch and the branch N corresponds to the No branch.
[0030] By way of non-limitation, in conjunction with [Fig.2A], on the side of the primary device Dl, the operating algorithm is for example as follows:
[0031] E0: start of the algorithm.
[0032] El: The processing and control unit of the primary device Dl continuously emits a time code TC. Its recording state S_rec is by default inactive (S_rec=OFF).
[0033] E2: User action on the primary device Dl, to switch the recording state to active status (S_rec=ON) or the reading state to active status (S_read=ON).
[0034] E3: The processing and control unit determines whether the user action results in the recording state being switched to active status (S_rec=ON?).
[0035] E4: If the recording state is active (S_rec=ON), the processing and control unit of the primary device Dl sends a command to the device secondary D2 to inform it that the recording state is in active status and emits the timecode TC associated with each element of its stream that is recorded.
[0036] E5: The processing and control unit of the primary device activates the recording of the incoming stream FX_1 and writes (W_l) each element into its storage means M_l, a timecode TC being associated with each element of the recorded stream. Stream recording continues until a new action is received from the user (this new action causing a return to step E2).
[0037] E6: If the record state is inactive status (S_rec=OFF), the processing and control unit determines whether the user action has switched the read state to active status (S_read=ON?).
[0038] E7: If the read state is in inactive status (S_read=OFF), the processing and control unit of the primary device DI returns to step El described above.
[0039] E8: If the read state is in active status (S_read=ON), the processing and control unit of the primary device DI maintains the inactive recording state (S_rec=OFF) and sends this command to the secondary device D2, accompanied by the time code of each element read.
[0040] E8: For reading, the processing and control unit of the primary device reads the selected elements (R_l) of the selected stream into its storage means M_l.
[0041] By way of non-limitation, in conjunction with [Fig.2B], on the side of the secondary device D2, the operating algorithm is for example as follows:
[0042] E10: Start of algorithm.
[0043] E20: By default, the secondary device D2 is in a so-called transparent state, that is- that is to say that the incoming FX_2 stream received at the input is duplicated on the output, without any action being taken on it.
[0044] E30: The secondary device D2 receives a timecode from the primary device DI TC. It may receive a command from the primary device DI regarding the recording state status (S_rec=ON / OFF). As mentioned above, the state of this command can be integrated into the message representing the timecode TC.
[0045] E40: The processing and control unit then determines whether the command relative to the recording state is in active status (S_rec=ON ?).
[0046] E50: If the command relating to the recording state is in active status (S_rec=ON), The processing and control unit of the secondary device D2 activates the writing (W_2) of each element of the incoming stream (FX_2) into its storage means, with the associated time code received at step E30. Once the writing of each element is complete, the secondary device D2 returns to the transparent state (step E20).
[0047] E60: If the command related to the recording state is in an inactive state (S_rec=OFF), the processing and control unit of secondary device D2 will check if the timecode TC received at step E30 is present in its storage means M_2.
[0048] E70: If the processing and control unit of the secondary device D2 identifies The timecode TC is stored in its memory M_2. It then switches the secondary device D2, which has a read state (R_2), to an active state, resulting in a read of the element associated with the found timecode TC and sending this element to its output. The operator can thus read, in a synchronized manner, the stream recorded by the primary device and the stream recorded by the secondary device.
[0049] E80: If the processing and control unit of the secondary device does not find If the timecode TC is not found in its storage M_2, this means that the device is in an inactive recording state (S_rec=OFF) and also in an inactive reading state (timecode TC not found). It therefore remains in a transparent state and rechecks the timecode TC received at step E30, as well as the status of the recording state.
[0050] In summary:
[0051] It is observed that the two devices will record their respective FX_1, FX_2 streams, the two streams being time-stamped by the same time code TC.
[0052] When secondary device D2 is not in an active recording state, it is configured to: - Read the TC timecode emitted by the primary device DI; - Constantly search through already saved items for a code temporal TC corresponding to that received;
[0053] If the timecode emitted by the primary device DI is found: - The secondary device D2 switches to an active reading state; - The secondary device D2 feeds its output stream with the elements recorded while remaining synchronized with the received timecode;
[0054] If the timecode TC emitted by the primary device DI is not found: - The device remains in a transparent state; its incoming flow is sent to its output;
[0055] The timecode generated at the level of the primary device DI and received continuously by the secondary device D2 always allows for a synchronous replay of the two streams, in particular at the level of the secondary device D2, before the post-production period.
[0056] Thus, according to the invention, when an operator switches the primary device DI from the read state to the active state, the secondary device is automatically switched to the read state in the active state if the same timecode TC is shared by the two devices D1, D2. During this reading, the primary device DI receives the elements recorded at the level of the storage means M_2 of the secondary device D2, which correspond to the TC timecode being read. The operator can thus read the two streams in parallel synchronously.
[0057] It should be noted that the principle of the invention can be duplicated in a system comprising a primary device DI and N secondary devices D2_1, D2_2, D2_N (with N greater than or equal to 2), according to the diagram shown in [Fig. 3A] or according to the diagram shown in [Fig. 3B]. In these figures, each secondary device manages a flow FX_2_1, FX_2_2, FX_2_N.
[0058] In this system, the primary device DI sends information (timecode and recording state) to all secondary devices connected in parallel ([Fig.3A]) or connected in a chain ([Fig.3B]).
[0059] Furthermore, it is also possible to use the same secondary device D2 capable of managing several flows in parallel.
[0060] The invention has many advantages, including: - The operating process is automatic because the secondary device D2 dynamically changes its state based solely on information sent by the primary device D1, without any intervention from an operator on the flow. - The system thus described is synchronized by the use of timecode which allows synchronous replay of the two streams. - The combination of these two characteristics makes it possible to deliver a stream in line with the operator's expectations, in all phases of their activity (waiting for the next recording, recording, listening back or remixing). - Little information is exchanged between the two devices, just timecode and commands related to the recording status.
Claims
Demands
1. Method for managing a first stream (FX_1) and a second stream (FX_2), the first stream and the second stream each being of audio or video type, the first stream (FX_1) being received as input by a device called primary (D1) and the second stream (FX_2) being received as input by a device called secondary (D2), the first stream (FX_1) and the second stream (FX_2) each being composed of one or more elements, the primary device and the secondary device having storage means (M_1, M_2) configured to store each element of the first and second streams as well as an associated time code (TC), characterized in that it comprises: - A step of transmission by the primary device (D1) of a time code (TC) to the secondary device (D2), said time code being associated with each element of the first stream;- The secondary device (D2) is configured to: - Receive the time code (TC) from the primary device (D1), - When it receives a first activation command from the primary device, switch to a recording state of the second stream, each element of the second stream being recorded, associated with the received time code; - Switch to a so-called reading state of each element of the second stream stored in its storage means when the first command is not received or is inactive and the time code (TC) is already stored in its storage means (M2).
2. Method according to claim 1, characterized in that the secondary device (D2) is configured to remain in a so-called transparent state when the received time code (TC) is not stored in its storage means (M_2).
3. Method according to claim 1 or 2, characterized in that the recording state has an active or inactive status defined by the value of a bit of the timecode (TC) sent by the primary device (D1) to the secondary device (D2).
4. System for managing a first stream (FX_1) and a second stream (FX_2), the first stream (FX_1) and the second stream (FX_2) each being of audio or video type, the first stream being received as input by a device called primary (D1) and the second stream being received as input by a device called secondary (D2), the first stream and the second stream each being composed of one or more elements, the primary device and the secondary device having storage means (M_1, M_2) configured to store each element of the first and second streams as well as an associated time code, characterized in that: - The primary device (D1) is configured to emit a time code (TC) to the secondary device (D2), said time code being associated with each element of the first stream; - The secondary device (D2) is configured to: - Receive the timecode (TC) from the primary device (Dl), - When it receives a first activation command from the primary device, switch to a recording state of the second stream, each element of the second stream being recorded, associated with the received time code; - To enter a so-called reading state of each element of the second stream stored in its storage means when the first command is not received or is inactive and the time code (TC) is already stored in its storage means (M_2).
5. System according to claim 4, characterized in that the secondary device (D2) is configured to remain in a so-called transparent state when the received time code (TC) is not stored in its storage means (M_2).
6. System according to claim 4 or 5, characterized in that the recording state has an active or inactive status defined by the value of a bit of the timecode (TC) sent by the primary device (D1) to the secondary device (D2).