Video collaborative editing process optimization method with multi-track audio synchronization

Abstract

The present invention discloses a method for optimizing a video collaborative editing process with multi-track audio synchronization, and relates to the technical field of audio and video editing. The method comprises: determining a video acquisition device and a plurality of independent recording devices; performing synchronous data acquisition to generate initial audio and video data and a plurality of independent audio track data; separating the picture and audio of the initial audio and video data, and aligning and mapping according to the time code to generate original picture data and original audio data; performing time code alignment correction on the original audio data and the plurality of independent audio track data, and then performing environmental sound processing on the corrected original audio data based on the background environment requirements, and then fusing it with the plurality of corrected independent audio track data to generate fused audio track data; performing synchronous fusion of the fused audio track data and the original picture data to generate a target fused video file. The technical problem of low efficiency of multi-track audio and video synchronization in the prior art is solved, and the technical effect of improving the efficiency of audio and video editing is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of audio and video editing, and in particular to a method for optimizing a video collaborative editing process with multi-track audio synchronization. Background Art

[0002] In the current field of audio and video editing technology, with the continuous improvement of multimedia production needs, the synchronous editing of multi-track audio and video has become one of the core links in the production of high-quality audio and video content. However, in the prior art, the synchronous processing efficiency of multi-track audio and video data is low, and it usually relies on manual operation or primary time code matching algorithms for processing. This method is not only time-consuming and labor-intensive, but also prone to insufficient synchronization accuracy due to time code errors or deviations between device recordings, affecting the final editing quality. In addition, with the increase in diversified scene requirements, audio and video production has put forward higher requirements for the processing of background sound. However, existing methods often lack flexibility and cannot effectively process and optimize multi-track audio data according to different scene background requirements, which leads to poor fusion of background sound and target picture, making it difficult to meet the audience's needs for high immersion and high-quality audio and video. Summary of the invention

[0003] The present application provides a method for optimizing the process of video collaborative editing with multi-track audio synchronization, which solves the technical problem of low efficiency of multi-track audio and video synchronization in the prior art.

[0004] In view of the above problems, the present application provides a method for optimizing the collaborative video editing process with multi-track audio synchronization.

[0005] The present application provides a method for optimizing a video collaborative editing process with multi-track audio synchronization, the method comprising:

[0006] Determine a video acquisition device and multiple independent recording devices for audio and video acquisition of a target scene; perform synchronous data acquisition through the video acquisition device and the multiple independent recording devices to generate initial audio and video data and multiple independent audio track data; receive the background environment requirements of the target scene; separate the picture and audio of the initial audio and video data, and align and map them according to the time code to generate original picture data and original audio data; perform time code alignment correction on the original audio data and the multiple independent audio track data, perform environmental sound processing on the corrected original audio data based on the background environment requirements, and fuse them with the multiple corrected independent audio track data to generate fused audio track data; synchronously fuse the fused audio track data and the original picture data to generate a target fused video file.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] First, determine the video acquisition device and multiple independent recording devices used to acquire audio and video of the target scene. Then, synchronous data acquisition is performed through the video acquisition device and multiple independent recording devices to generate initial audio and video data and multiple independent audio track data. At the same time, the background environment requirements of the target scene are received. Then, the initial audio and video data is separated into picture and audio, and aligned and mapped according to the time code to generate original picture data and original audio data. Further, after the original audio data and multiple independent audio track data are aligned and corrected for time code, the corrected original audio data is processed for environmental sound based on the background environment requirements, and then merged with multiple corrected independent audio track data to generate fused audio track data. Finally, the fused audio track data and the original picture data are synchronously merged to generate a target fused video file. The technical problem of low efficiency of multi-track audio and video synchronization in the prior art is solved, and the technical effect of improving the efficiency of audio and video editing is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram of a method for optimizing a collaborative video editing process with multi-track audio synchronization provided in an embodiment of the present application;

[0011] Figure 2 A schematic diagram of the process of generating fused audio track data in the video collaborative editing process optimization method with multi-track audio synchronization provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The present application solves the technical problem of low efficiency of multi-track audio and video synchronization in the prior art by providing a method for optimizing the video collaborative editing process with multi-track audio synchronization.

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0014] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or are inherent to these processes, methods, products or devices.

[0015] Examples, such as Figure 1 As shown, the embodiment of the present application provides a method for optimizing a video collaborative editing process with multi-track audio synchronization, wherein the method includes:

[0016] Determine the video capture device and multiple independent recording devices used to capture audio and video of the target scene.

[0017] When capturing audio and video of the target scene, it is necessary to determine suitable video capture equipment and multiple independent recording devices based on the scene requirements to ensure that high-quality images and audio data are captured. Video capture equipment includes high-definition cameras, movie cameras, mobile phone cameras, etc. Determine whether anti-shake function, wide-angle lens or high frame rate support are required based on the scene requirements; for example, static scenes can use fixed cameras, while dynamic scenes require equipment that supports stable shooting. Independent recording devices are used to capture different audio tracks, such as directional microphones for picking up sounds in a specific direction, lavalier microphones for recording dialogues, environmental recording devices for capturing background sounds, and multi-track recorders that can record multiple audio tracks at the same time.

[0018] Synchronous data acquisition is performed through the video acquisition device and the multiple independent recording devices to generate initial audio and video data and multiple independent audio track data.

[0019] The video acquisition device is responsible for recording the image of the target scene and its own audio track, while multiple independent recording devices collect high-quality audio track data from different sound sources, such as dialogue, ambient sound, or specific sound source tracks. By simultaneously collecting data from the video acquisition device and multiple independent recording devices, initial audio and video data and multiple independent audio track data are generated. The initial audio and video data refers to the video data that carries both the video image and the recorded background sound, and the independent audio track data refers to the audio data collected by each independent recording device. During the data collection process, all devices need to record synchronously based on a unified time code to ensure that the generated initial audio and video data and the independent audio track data are consistent in time. This synchronous acquisition method can effectively avoid the problem of audio and video being out of sync due to time deviations of different devices in traditional audio and video recording, and lay the foundation for subsequent time code alignment, audio track processing, and audio and video fusion.

[0020] Receive background environment requirements of the target scene.

[0021] Different target scenes have different requirements for background sound processing, such as the need to increase ambient sound or highlight dialogue sound. By receiving the background demand information of the target scene, specific requirements for audio processing and video editing are obtained. The requirements for background sound in different scenes may vary significantly. For example, in interview or interview scenes, in order to ensure the clarity of voice content, it is usually necessary to remove all environmental noise, including background wind and noise, to create a quiet auditory environment. In life videos, in order to preserve the realism and atmosphere of the scene, it is necessary to appropriately retain environmental sounds, such as the background sound of the street, the natural wind sound, or the whispered conversation of the crowd, while reducing noise that is too harsh or disruptive.

[0022] The initial audio and video data is separated into picture and audio, and aligned and mapped according to the time code to generate original picture data and original audio data.

[0023] Use audio and video separation tools or algorithms (such as FFmpeg or other professional video processing software) to extract audio tracks and video images from the initial audio and video data; read the time code information embedded in the initial audio and video data, and align and map the separated audio data and image data according to the time code. During the alignment process, ensure that the time code starting points are consistent, and correct any potential frame rate differences or device delays to ensure that the audio and image data are strictly synchronized on the timeline; after completing the time code alignment, output the original image data and original audio data to ensure that in the subsequent audio correction, processing and audio and video fusion process, the audio and image data can be accurately matched to achieve high-quality synchronization effects.

[0024] After time code alignment correction is performed on the original audio data and the multiple independent audio track data, the corrected original audio data is processed for environmental sound based on the background environment requirements, and then merged with the multiple corrected independent audio track data to generate merged audio track data.

[0025] Extract the time code information embedded in the original audio data and multiple independent audio track data, and use audio editing tools (such as Adobe Audition or professional time code calibration software) to compare the time axis of each audio data; for problems such as inconsistent time codes or delays in recording devices, accurately correct the time code of the audio track by adjusting the starting point of the time axis, scaling the time axis, or inserting compensation segments to ensure that all audio data is completely synchronized on the time axis.

[0026] After completing the time code alignment correction, the corrected original audio data is processed for environmental sound according to the received background environment requirements. For background noise processing such as interviews or interview scenes, noise reduction algorithms (such as spectral noise reduction or AI noise reduction technology) can be used to completely remove environmental noise and enhance the clarity of the human voice; for background sound processing in life videos that need to retain a sense of real scenes, the performance of environmental sound is optimized by adjusting audio equalization, reducing high-frequency noise, or appropriately enhancing the natural part of background sound. The original audio data that has been processed for environmental sound is fused with multiple independent audio track data after time code correction to generate fused audio track data. The fused audio track data can not only meet the background sound requirements of the target scene, but also ensure the synchronization and auditory effects between the audio tracks, laying a high-quality audio foundation for the subsequent synchronous fusion with the picture data.

[0027] Furthermore, performing time code alignment correction on the original audio data and the plurality of independent audio track data includes:

[0028] Collect the time codes of the starting sampling points in the original audio data and the multiple independent audio track data, and calculate the time code offsets of the multiple independent audio track data relative to the original audio data; perform time code offset correction on the original audio data and the multiple independent audio track data according to the time code offset, and generate corrected original audio data and multiple corrected independent audio track data.

[0029] Preferably, the time code of the starting sampling point in the original audio data and multiple independent audio track data is collected, and the time code information of each audio track is parsed using an audio processing tool to record the time code of the starting sampling point of the original audio data and each independent audio track data; by comparing the starting time code of each audio track, the time code offset of multiple independent audio track data relative to the original audio data is calculated to obtain the time code deviation value of each audio track; according to the calculated time code offset, the original audio data and multiple independent audio track data are corrected for time code offset, which can be achieved by inserting a blank audio segment, cutting the excess part or resetting the time code starting point. During the correction process, the audio track with a large time code offset is moved or aligned on the time axis to ensure that the data starting point of all audio tracks is consistent with the time code of the original audio data. After completing the time code offset correction, the corrected original audio data and multiple corrected independent audio track data are generated. The correction result ensures that all audio track data are completely synchronized in the time dimension, providing an accurate time reference for subsequent audio track fusion and audio and video synchronization, thereby effectively avoiding audio synchronization problems caused by differences in recording equipment or operating errors, and improving the quality and efficiency of audio and video processing.

[0030] Furthermore, if Figure 2As shown, based on the background environment requirements, the corrected original audio data and the multiple corrected independent audio track data are processed for environmental sound and then merged to generate merged audio track data, including:

[0031] Based on the background environment requirements, determine whether to remove all background environment sounds; if not, extract the background sound retention features; establish a pure noise fingerprint based on the background sound retention features, and construct a filtering module with the pure noise fingerprint; perform denoising on the corrected original audio data and the multiple corrected independent audio track data through the filtering module; perform audio track synchronization and volume balance on the corrected original audio data after denoising and the multiple corrected independent audio track data to generate the fused audio track data.

[0032] Specifically, according to the background environment requirements, determine whether the target scene needs to completely remove the background environment sound. For example, if the scene is a conversation or other situation that needs to highlight the human voice, choose to remove all background sounds; if it is a situation such as a life video that needs to retain the real scene atmosphere, choose to retain some background sounds; if the background environment sound does not need to be completely removed, analyze and extract specific features in the background sound. The extraction process can be based on spectral features, energy distribution or timbre features to mark the background sound part that needs to be retained (such as natural wind sound, slight ambient sound, etc.) for subsequent processing; based on the extracted background sound retention features, identify the pure noise part that needs to be removed in the audio (such as mechanical noise, current noise or irregular noise), and generate a pure noise fingerprint through spectral analysis. Using the pure noise fingerprint, build an adaptive filtering module (such as an adaptive noise filter) The method comprises the following steps: first, performing denoising on the corrected original audio data and multiple independent audio track data using a filtering module to remove the redundant pure noise part in the audio while retaining the required background sound characteristics and optimizing the audio quality; performing track synchronization and volume balance processing on the corrected original audio data after denoising and multiple corrected independent audio track data; performing synchronization processing to verify the corrected audio data again through the time code to ensure that the time base of each audio track is consistent; and adjusting the volume ratio of the audio track according to the scene requirements, such as increasing the volume of the vocal track and reducing the volume of the background track to ensure the overall auditory effect of the fused audio track data; and merging the processed corrected original audio data with the multiple independent audio track data to generate the final fused audio track data.

[0033] Furthermore, performing track synchronization and volume balance processing on the corrected original audio data after denoising and the plurality of corrected independent audio track data to generate the fused audio track data includes:

[0034] The volume and frequency imbalance of the corrected multiple independent audio track data are adjusted to generate multiple independent audio track adjustment data; the audio track masters corresponding to the multiple independent recording devices are determined, and a dynamic volume adjustment correlation curve between the audio track master sound and the ambient sound is constructed; based on the dynamic volume adjustment correlation curve, the corrected original audio data after denoising processing and the multiple corrected independent audio track data are synchronized with the time code to generate the fused audio track data.

[0035] Specifically, the audio processing tool is used to analyze the data of multiple independent audio tracks one by one, detect the volume peak and frequency distribution range, and identify the volume and frequency imbalance between the audio tracks; according to the analysis results, the equalizer is used to adjust the frequency distribution of the audio track to eliminate abnormal frequency components, and the dynamic range compressor is used to optimize the volume peak of the audio track to generate independent audio track adjustment data after volume and frequency balance. Next, the recording device corresponding to each audio track and its collection object, that is, the owner of the audio track, such as the speaker in the interview or the participant of the dialogue, are determined, and the characteristic parameters of the owner's voice of the audio track are extracted, including the dynamic range of volume, the main frequency band range, and the clarity requirements of the sound. Combined with the background environment requirements, a dynamic volume adjustment correlation curve between the owner's voice of the audio track and the environmental sound is constructed. The curve defines the priority of the owner's voice of the audio track and the volume ratio relationship with the background sound in different time periods to ensure the prominent effect of the main sound track and the coordination with the background sound. Subsequently, based on the time code information embedded in the audio data, the time code alignment check is performed on the corrected original audio data after denoising and the multiple independent audio track adjustment data to ensure that each audio track is completely synchronized on the time axis. During the synchronization process, the possible time deviation is corrected by comparing and correcting the starting point and rate of the time code. On the basis of time code synchronization, the volume of the audio track in each time period is dynamically adjusted according to the constructed dynamic volume adjustment curve, while matching the clarity requirements of the master sound of the audio track to ensure the balance and auditory effect of the final audio track fusion. Finally, the corrected original audio data after synchronization and dynamic volume adjustment is integrated with the adjusted independent audio track data to generate fused audio track data. The output audio has both the clarity of the master sound of the audio track and the harmonious fusion with the background sound, laying a high-quality audio foundation for subsequent audio and video synchronization editing.

[0036] Furthermore, determining the master audio tracks corresponding to the plurality of independent recording devices and constructing a dynamic volume adjustment correlation curve between the master audio track sounds and the ambient sounds includes:

[0037] Collect historical audio fusion records and corresponding historical fusion effect evaluation parameters; analyze the historical audio fusion records to generate contrast data between the fused audio track master data and the environmental noise; select the optimal contrast corresponding to the optimal evaluation effect parameter based on the contrast data and the historical fusion effect evaluation parameters; and construct the dynamic volume adjustment related curve with the optimal contrast.

[0038] Specifically, historical audio fusion records and corresponding historical fusion effect evaluation parameters are collected, and these evaluation parameters may include audio clarity scores, environmental noise interference levels, user satisfaction feedback, etc., to reflect the comprehensive evaluation of historical audio fusion effects; data analysis is performed on the collected historical audio fusion records to extract the fused track master data (i.e., the characteristic data of the target sound in the main sound track, such as vocal signal strength, main frequency distribution, etc.) and environmental noise data, and the contrast between the two is calculated to generate a set of track master and environmental noise contrast data corresponding to the fusion effect; based on these contrast data and the corresponding historical fusion effect evaluation parameters, a screening algorithm (such as a multivariate optimization method or a correlation analysis method) is used to match the evaluation parameters with the contrast data to screen out the optimal contrast corresponding to the evaluation parameters that can achieve the best historical fusion effect; based on the screened optimal contrast as a benchmark, combined with the target scene requirements and evaluation criteria, a dynamic volume adjustment correlation curve of the track master sound and the environmental sound is constructed to ensure that the curve can achieve a dynamic balance between the main sound track and the background sound track in different time periods and scenes, and provide a precise control basis for subsequent track synchronization and volume adjustment.

[0039] The fused audio track data and the original picture data are synchronously fused to generate a target fused video file.

[0040] The fused audio track data is synchronously merged with the original picture data to ensure perfect synchronization of audio and video and generate the final target fused video file.

[0041] Furthermore, the fused audio track data and the original picture data are synchronously fused to generate a target fused video file, including:

[0042] A lip shape-sound matching analysis network is configured, wherein the lip shape-sound matching analysis network is constructed based on training of lip shape feature samples and audio feature samples having a corresponding relationship; based on the lip shape-sound matching analysis network, matching analysis is performed on the audio data and the picture data belonging to the same time zone in the fused audio track data and the original picture data; if the match is successful, the fused audio track data and the original picture data are synchronized by time code to generate the target fused video file.

[0043] Specifically, first, a lip shape-sound matching analysis network is configured, which is trained based on lip shape feature samples and audio feature samples with corresponding relationships. The network is learned through a large amount of annotated audio and lip shape feature data, so that it can accurately identify the corresponding relationship between audio and lip shape movements. Then, the configured lip shape-sound matching analysis network is used to perform frame-by-frame matching analysis on the audio and picture data belonging to the same time period in the fused audio track data and the original picture data. By analyzing the characteristics of the lip shape movement in the picture and the pronunciation characteristics of the corresponding audio, it is determined whether the two match to ensure the synchronization of audio and video. If the matching analysis result shows that the audio and the picture are successfully matched in the current time period, the fused audio track data and the original picture data are synchronized and fused in time code based on this, and the possible synchronization errors are eliminated through the correction and binding of the time code. Finally, the matched and synchronized data are integrated and output to generate the target fused video file. In this way, it is ensured that the audio track and the picture in the generated video file are not only completely aligned in time, but also achieve high-precision matching in the coordination of voice and lip shape movements, thereby significantly improving the overall quality and user experience of audio and video fusion.

[0044] Furthermore, performing matching analysis on the audio data and the picture data belonging to the same time zone in the fused audio track data and the original picture data based on the lip shape-sound matching analysis network also includes:

[0045] Perform voiceprint feature analysis on audio data belonging to the same time zone to establish speaker identity information; extract lip shape features of the corresponding speaker in the picture data of the same time zone based on the speaker identity information; input the lip shape feature extraction result and the audio data into the lip shape-sound matching analysis network for matching analysis.

[0046] Specifically, voiceprint feature analysis is performed on audio data in the same time zone, and voiceprint recognition technology is used to extract voiceprint features in the audio data, and the identity information of the speaker is identified and established to determine the specific sound-making subject corresponding to the current audio; based on the extracted speaker identity information, the picture area corresponding to the speaker is located in the picture data in the same time zone, and an image processing algorithm is used to extract the lip shape features of the speaker, including the movement trajectory of the lips, the degree of opening and closing, and other key point change information, to form complete lip shape feature data; the extracted lip shape feature data and the corresponding audio data are input into the configured lip shape-sound matching analysis network, and the matching analysis function of the network is used to determine whether the voiceprint features of the audio data are consistent with the lip shape features in the picture data.

[0047] Furthermore, performing matching analysis on the audio data and the picture data belonging to the same time zone in the fused audio track data and the original picture data based on the lip shape-sound matching analysis network also includes:

[0048] If the matching fails, the forward time zone and the backward time zone are re-matched according to the preset step size in the matching failure time zone; if the re-matching result is successful, the offset correction is performed on the fused audio track data and the original picture data based on the re-matching result to generate the target fused video file; if the re-matching result fails, the original matching failure time zone is sent to the editing management terminal for reminder.

[0049] When the audio data and picture data belonging to the same time zone in the fused audio track data and the original picture data are matched and analyzed based on the lip shape-sound matching analysis network, if the match fails, the forward time zone and the backward time zone are rematched according to the preset time step in the time zone where the match fails. During the rematching process, the matching range is gradually expanded, and the audio data and picture data in the adjacent time zones are analyzed to try to find a segment that matches the voiceprint features of the audio data and the lip shape features of the picture. If the rematching result is successful, the offset correction of the time code of the fused audio track data and the original picture data is performed based on the result obtained by the rematching to ensure the consistency of the audio and picture on the time axis, and then the final target fused video file is generated. If the rematching result still fails, the time zone where the original match fails is recorded and sent to the editing management terminal to remind the user that the audio and video matching of the time segment is abnormal, which is convenient for manual intervention and further processing. Through this method, a reliable remediation mechanism can be provided in the case of automatic matching failure, which can not only maximize the matching success rate, but also effectively reduce errors and omissions in audio and video synchronization editing, and improve the overall quality and efficiency of audio and video fusion.

[0050] In summary, the embodiments of the present application have at least the following technical effects:

[0051] First, determine the video acquisition device and multiple independent recording devices used to acquire audio and video of the target scene. Then, synchronous data acquisition is performed through the video acquisition device and multiple independent recording devices to generate initial audio and video data and multiple independent audio track data. At the same time, the background environment requirements of the target scene are received. Then, the initial audio and video data is separated into picture and audio, and aligned and mapped according to the time code to generate original picture data and original audio data. Further, after the original audio data and multiple independent audio track data are aligned and corrected for time code, the corrected original audio data is processed for environmental sound based on the background environment requirements, and then merged with multiple corrected independent audio track data to generate fused audio track data. Finally, the fused audio track data and the original picture data are synchronously merged to generate a target fused video file. The technical problem of low efficiency of multi-track audio and video synchronization in the prior art is solved, and the technical effect of improving the efficiency of audio and video editing is achieved.

[0052] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0053] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0054] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A method for optimizing the collaborative video editing process with multi-track audio synchronization, characterized in that: The method comprises: Determine the video capture device and multiple independent recording devices for audio and video capture of the target scene; Performing synchronous data acquisition through the video acquisition device and the multiple independent recording devices to generate initial audio and video data and multiple independent audio track data; Receiving background environment requirements of the target scene; Separating the picture and the audio of the initial audio and video data, and aligning and mapping them according to the time code to generate original picture data and original audio data; After time code alignment correction is performed on the original audio data and the multiple independent audio track data, environmental sound processing is performed on the corrected original audio data based on the background environment requirements, and then the corrected original audio data is merged with the multiple corrected independent audio track data to generate merged audio track data; The fused audio track data and the original picture data are synchronously fused to generate a target fused video file.

2. The method for optimizing the video collaborative editing process with multi-track audio synchronization as claimed in claim 1, characterized in that: Based on the background environment requirement, the corrected original audio data and the multiple corrected independent audio track data are subjected to environmental sound processing and then merged to generate merged audio track data, including: Determine whether to remove all background environment sounds based on the background environment requirements; If not, extract the background sound retention features; Establishing a pure noise fingerprint based on the background sound retention feature, and constructing a filtering module with the pure noise fingerprint; Performing denoising processing on the corrected original audio data and the plurality of corrected independent audio track data by the filtering module; The corrected original audio data after the denoising process and the multiple corrected independent audio track data are subjected to audio track synchronization and volume balance processing to generate the fused audio track data.

3. The method for optimizing the video collaborative editing process with multi-track audio synchronization as claimed in claim 2, characterized in that: Performing track synchronization and volume balance processing on the corrected original audio data after denoising and the multiple corrected independent audio track data to generate the fused audio track data, including: Performing volume and frequency imbalance adjustment on the corrected plurality of independent audio track data to generate a plurality of independent audio track adjustment data; Determine the master audio tracks corresponding to the multiple independent recording devices, and construct a dynamic volume adjustment correlation curve between the master audio track sound and the ambient sound; Based on the dynamic volume adjustment related curve, the corrected original audio data after denoising is synchronized with a plurality of corrected independent audio track data to generate the fused audio track data.

4. The method for optimizing the video collaborative editing process with multi-track audio synchronization as claimed in claim 3, characterized in that: Determining the audio track masters corresponding to the multiple independent recording devices, and constructing a dynamic volume adjustment correlation curve between the audio track master sounds and the ambient sounds, including: Collect historical audio fusion records and corresponding historical fusion effect evaluation parameters; Analyze the historical audio fusion record to generate contrast data between fused audio track master data and environmental noise; Selecting the optimal contrast corresponding to the optimal evaluation effect parameter based on the contrast data and the historical fusion effect evaluation parameter; The dynamic volume adjustment related curve is constructed with the optimal contrast.

5. The method for optimizing the video collaborative editing process with multi-track audio synchronization as claimed in claim 1, characterized in that: Synchronously fusing the fused audio track data and the original picture data to generate a target fused video file, including: Configuring a lip shape-sound matching analysis network, wherein the lip shape-sound matching analysis network is constructed based on training of lip shape feature samples and audio feature samples having a corresponding relationship; Performing matching analysis on the audio data and the picture data belonging to the same time zone in the fused audio track data and the original picture data based on the lip shape-sound matching analysis network; If the match is successful, the fused audio track data and the original picture data are fused in time code synchronization to generate the target fused video file.

6. The method for optimizing the video collaborative editing process with multi-track audio synchronization as claimed in claim 5, characterized in that: The method further comprises: performing matching analysis on the audio data and the picture data belonging to the same time zone in the fused audio track data and the original picture data based on the lip shape-sound matching analysis network; Perform voiceprint feature analysis on audio data belonging to the same time zone to establish speaker identity information; Extracting lip shape features of the corresponding speaker in the picture data of the same time zone based on the speaker identity information; The lip shape feature extraction result and the audio data are input into the lip shape-sound matching analysis network for matching analysis.

7. The method for optimizing the video collaborative editing process with multi-track audio synchronization as claimed in claim 6, characterized in that: Based on the lip shape-sound matching analysis network, matching analysis is performed on the audio data and the picture data belonging to the same time zone in the fused audio track data and the original picture data, further comprising: If the matching fails, rematch the forward time zone and the backward time zone according to the preset step length in the matching failure time zone; If the re-matching result is successful, the fused audio track data and the original picture data are offset-corrected based on the re-matching result to generate the target fused video file; If the re-matching result fails, the original matching failure time zone will be sent to the editing management terminal for reminder.

8. The method for optimizing the video collaborative editing process with multi-track audio synchronization as claimed in claim 2, characterized in that: Performing time code alignment correction on the original audio data and the plurality of independent audio track data, comprising: Collecting time codes of starting sampling points in the original audio data and the plurality of independent audio track data, and calculating time code offsets of the plurality of independent audio track data relative to the original audio data; The original audio data and the plurality of independent audio track data are corrected for time code offset according to the time code offset to generate corrected original audio data and a plurality of corrected independent audio track data.

Images