Audio waveform graph generation method and video striping method and device, and electronic device

Abstract

Embodiments of the present application provide an audio waveform chart generation method and video splitting method and device, electronic equipment and storage medium, by acquiring an audio slice file, and performing coarse filtering sampling on the audio slice file, a coarse-grained waveform chart is quickly generated, realizing fast acquisition of audio waveform features to cope with the need for strong real-time, and the slice file is also decoded, and a more accurate amplitude sequence is obtained by fine sampling on the decoded slice file, so as to draw a fine-grained audio waveform. And by storing and comparing the coarse-grained amplitude data of the nearest slice, the target waveform feature is identified, so as to roughly determine the position of the target waveform, automatically add an entry point in the splitting track and create a temporary splitting segment. The present application solves the problem that the browser memory may be too large to cause failure in the audio waveform chart drawing of a large file or direct current stream, and ensures the efficiency and accuracy of the audio waveform chart drawing.

Description

Technical Field

[0001] This application relates to the field of audio and video processing, and in particular to an audio waveform generation method, video segmentation method, apparatus, electronic device, and storage medium. Background Technology

[0002] In existing technologies, generating waveforms on the web first requires downloading the entire audio or video file from the server over the network to obtain its binary data, then creating an audio context to decode the audio data, and finally using a canvas to draw the audio graph. However, existing solutions only support drawing audio waveforms from static audio files of normal size. When drawing audio waveforms from large files, the browser may encounter excessive memory usage, leading to failure. Summary of the Invention

[0003] This invention provides an audio waveform generation method, a video segmentation method, apparatus, electronic device, and storage medium, which enables rapid drawing of audio waveforms and reduces memory usage.

[0004] In a first aspect, the present invention provides a method for generating an audio waveform, comprising: Obtain the original audio, slice the original audio, and generate slice files; According to the preset sampling rules, the sliced ​​file is sampled to obtain the first sampling data; The first sampled data is decoded to generate a first waveform.

[0005] Further, acquiring the first sampling data includes: The number of sampling times is determined based on the width of the waveform diagram in the slice file; The sampling operation is performed uniformly on each of the slice files for the specified number of sampling times to obtain the first sampled data.

[0006] Furthermore, after generating the first waveform, the process also includes: Perform a full decoding operation on the slice file to obtain the decoded data of the slice file; The decoded data of the slice file is sampled to obtain second sampled data, wherein the sampling number of the second sampled data is greater than the sampling number of the first sampled data; A second waveform is generated based on the second sampled data.

[0007] Secondly, the present invention also provides a video segmentation method, comprising: Obtain the original video file and perform audio data separation operation on the original video file to obtain the audio data file; Perform the above-described audio waveform generation method on the audio data file to obtain the first waveform; Based on the characteristics of the preset target waveform, determine the target position of the target waveform in the first waveform diagram; The target location is confirmed to be the in-segment entry point of the original video file.

[0008] Further, the audio waveform generation method described above is performed on the audio data file to obtain the second waveform; Based on the characteristics of the preset target waveform, the target position of the target waveform in the second waveform diagram is determined; Adjust the in-point of the original video file based on the target position shown in the second waveform diagram.

[0009] Furthermore, after confirming that the target location is the in-segment point of the original video file, the process also includes: The strip disassembly entry point is shown in the first waveform diagram and / or the second waveform diagram. or The original video file is split into segments based on the stated segmentation entry point.

[0010] Thirdly, the present invention also provides an audio waveform generation apparatus, comprising: The slice file generation module is used to acquire the original audio, slice the original audio, and generate slice files; The first sampling data acquisition module is used to sample the slice file according to a preset sampling rule to acquire first sampling data; The first waveform generation module is used to decode the first sampled data and generate the first waveform.

[0011] Furthermore, it also includes: The PCM data acquisition module is used to perform full decoding of the slice file to acquire the pulse code modulation data of the slice file. If the value of the pulse code modulation data is within [-1, 1], the second sampling data acquisition module resamples the slice file to acquire second sampling data, wherein the number of second sampling data is greater than the number of first sampling data. The second waveform generation module is used to generate a second waveform based on the second sampled data.

[0012] Fourthly, the present invention also provides a video strip splitting device, comprising: The audio data acquisition module is used to acquire the original video file and perform audio data separation operation on the original video file to obtain an audio data file; The first waveform acquisition module is used to perform the audio waveform generation method as described in the first aspect on the audio data file to acquire the first waveform data. The target waveform position determination module is used to determine the target position of the target waveform in the first waveform diagram based on the characteristics of the preset target waveform. The segment entry point confirmation module is used to confirm that the target position is the segment entry point of the original video file.

[0013] Fifthly, the present invention provides an electronic device, comprising: At least one memory and at least one processor; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the at least one processor performs the steps of the audio waveform generation method and video segmentation method as described in the first and second aspects.

[0014] In a sixth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the audio waveform generation method and the video segmentation method as described in the first and second aspects.

[0015] This invention acquires raw audio, segments it to generate segment files, samples the segment files according to preset sampling rules to obtain sampled data, and then decodes the sampled data to generate a waveform. The coarse sampling based on the audio segment files in this invention can quickly generate coarse-grained waveforms, enabling rapid acquisition of audio waveform features to meet the needs of high real-time requirements. Furthermore, by identifying target waveform features in the coarsely sampled waveform, the approximate location of the target waveform can be quickly determined, and the entry point for segmentation can be quickly identified when segmentation is required at the target waveform location. This application solves the problem of failure due to excessive browser memory usage when drawing audio waveforms from large files or DC streams, ensuring the efficiency of audio waveform drawing. Simultaneously, this waveform drawing method facilitates the identification of segmentation locations for large files or live video streams, improving the efficiency of video segmentation.

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Attached Figure Description

[0017] Figure 1 This is a flowchart of the steps of an audio waveform generation method provided in an exemplary embodiment; Figure 2 A flowchart illustrating the steps of a video segmentation method provided in an exemplary embodiment; Figure 3 This is a schematic diagram of a module for an audio waveform generation apparatus provided in an exemplary embodiment; Figure 4 This is a schematic diagram of a video splitting device provided in an exemplary embodiment; Figure 5 This is a schematic diagram of an electronic device provided in one exemplary embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0019] It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0020] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0021] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0022] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0023] In a live streaming scenario using Apple devices and HTTP Live Streaming (HLS) technology, live video data is captured by cameras to generate a real-time video stream, which is then sent to the server. The server then transcodes the video stream into segments based on the HLS protocol, and finally distributes them to different clients via a Content Delivery Network (CDN). To promote the live stream content, it needs to be edited and distributed across multiple platforms. For example, in a concert live stream, the singer's performance segments often need to be edited out of the live stream and produced as separate videos. Currently, this editing work is often done manually, and the editors need a certain level of understanding of the live stream content to determine the appropriate editing points based on their professional experience. This means that the efficiency and accuracy of live stream editing depend entirely on the professional level of the editor. Even so, when special situations arise during the live stream that cannot be accurately judged based on experience, it will greatly affect the completion of the editing work.

[0024] Therefore, based on the above considerations and the problems raised in the background technology, such as Figure 1 As shown in the figure, this application provides a method for generating an audio waveform, which specifically includes the following steps: S201: Obtain the original audio, slice the original audio, and generate slice files.

[0025] The original audio can be audio file data directly obtained through an audio acquisition device, or it can be an audio file obtained by separating video file data, including audio tracks, obtained through a video acquisition device.

[0026] Slicing is the process of dividing a video stream into TS files according to a certain duration. Since large files or long live streams are common, the client loads and buffers a certain number of slices at a time for decoding and playback. Once a slice is downloaded locally, the cached data avoids the need to download the file again over the network, reducing latency and improving page performance.

[0027] In this embodiment, specifically, the image acquisition device acquires audio and video files captured in real-time from the live broadcast and uploads them to the server. The server slices the audio and video files to obtain TS file data. By separating the video track and audio track from the TS file data, the sliced ​​audio file is obtained.

[0028] S202: Sample the sliced ​​file according to the preset sampling rules to obtain the first sampled data.

[0029] In a preferred embodiment, obtaining the first sampled data includes the following steps: The number of sampling times is determined based on the width of the waveform diagram in the slice file; The sampling operation is performed uniformly on each of the slice files for the specified number of sampling times to obtain the first sampled data.

[0030] In this embodiment, specifically, the server slices the audio / video file or the pulled live stream to generate an m3u8 list. The client downloads the m3u8 file and then parses it to obtain the slice list information. For the initial rendering operation, the slice files corresponding to the trimFrom and trimTo time periods are downloaded. Then, the number of samples is determined based on the width of the DOM of the slice file. In a preferred example, each pixel corresponds to one sampling, based on the number of pixels in the horizontal direction, ultimately determining to perform M samplings. Then, the sampling number M of this slice is mapped to all slice files, that is, M samples are uniformly performed after calculating the offset from each slice file. After sampling all slice files, the collection of sampled data obtained is the first sampled data.

[0031] S203: Decode the first sampled data to generate a first waveform.

[0032] In this embodiment, a sampling number threshold can be preset. When the sampling number reaches the threshold, the acquired sampling data is combined and decoded. Then, a waveform is drawn based on the horizontal displacement. This waveform is a rough waveform based on the sampling data of the slice file, which can reflect the waveform characteristics of the audio file to a certain extent and reduce the waiting time for subsequent rendering.

[0033] In a preferred example, since the obtained coarse waveform cannot accurately describe the waveform characteristics of the audio file, the following steps are included after obtaining the first waveform: Perform a full decoding operation on the slice file to obtain the decoded data of the slice file; The decoded data of the slice file is sampled to obtain second sampled data, wherein the sampling number of the second sampled data is greater than the sampling number of the first sampled data; A second waveform is generated based on the second sampled data.

[0034] This application embodiment first performs coarse sampling on the audio file, then decodes and plots a waveform only on the coarsely sampled data. This method of quickly generating a coarse-grained waveform by coarsely sampling the undecoded data is to meet the high real-time requirements of live streaming applications. However, since the coarse waveform is plotted based on the data before decoding, which lacks time information, the correspondence between the waveform plotted by coarse sampling and time is not accurate enough. Furthermore, the sampling rate corresponding to the coarse waveform is low, and the accuracy of the waveform plotted based on it is also low, failing to accurately display the current audio waveform. Therefore, this application embodiment also provides a method that, after obtaining the coarse waveform, decodes all file segments, and then performs higher-precision sampling based on the decoded data to plot a precise waveform. Since the decoded data includes the time information corresponding to the audio data, the precise waveform plot has a more accurate correspondence between the waveform and time, and the higher sampling rate of the precise waveform plot allows for the display of a finer-grained sound waveform.

[0035] However, if a detailed waveform is drawn directly instead of a rough one, firstly, the decoding time is too long to meet the real-time requirements of live video streaming; secondly, for large video files and live video streaming, the large amount of data that needs to be downloaded and decoded can easily cause browser memory explosion and lead to failure. The method provided in this application not only ensures the real-time performance required by the application scenario and avoids browser memory pressure, but also provides a more finely granular audio waveform.

[0036] The following section will describe in detail the specific implementation process of the embodiments of this application, combined with practical application operations.

[0037] First, let's introduce the general process of generating waveforms based on HLS slicing. The server slices the audio / video file or the pulled live stream, generating an m3u8 list. The client downloads the m3u8 file and parses it to obtain the slice list information. The binary data of the video and audio tracks, the encoder used, and meta information are extracted from the slice file. Then, the audio track data of the slice is added to an ArrayBuffer list. Adding audio track data stops when the end time of the slice is greater than or equal to the preset end time for drawing the waveform. Before drawing the waveform, the data in the ArrayBuffer list is concatenated, and then the resulting buffer data is decoded to obtain the audio data. Because the start and end times of the sound wave are not necessarily the start and end times of the slice, it is necessary to segment the required audio data parts. Finally, the waveform is drawn based on the obtained audio data. After the audio track ArrayBuffer data is decoded through the WebAudio API, an AudioBuffer object is returned, which stores information such as the audio sampling rate, channels, and PCM data. The PCM data of a specific channel is obtained using the getChannelData method. PCM (Programmable Character Set) technology converts sound from analog to digital signals; it involves sampling and quantizing the sound. PCM data values ​​range from -1 to 1. Higher sampling frequencies and bit depths in PCM result in more realistic and natural sound reproduction, but also require more decoding data. Resampling is necessary for accurate waveform rendering. In a preferred example, sampling can be calculated every X milliseconds based on the width of the waveform container canvas, and the resampled audio data can be saved for use in subsequent renderings without needing to download it again.

[0038] Secondly, the generation of the second waveform, namely the generation of the refined audio waveform, includes the following steps. First, initialize the HLS player. The HLS player uses MSE to mount the media output to a web-based H5 audio player. Then, download and parse the m3u8 to obtain the segment list. By default, the first TS segment is buffered, and then the BUFFER_APPENDING event listened to by the player is triggered, which allows obtaining the data, sequence number sn, duration, and bufferLength of the TS segment. Create map segments to store the data information of the segments (M~N) within the time period. By default, segments with sn equal to 0 are loaded. If sn is not equal to 0 and the segment data has not been stored, it is stored in segments with sn as the key and an object composed of sn, duration, bufferLength, and data as the value. Then, set the current time of the audio to currentTime = Math.min(Math.max(fragEnd, start), end), where start and end are the start and end times of this waveform drawing, and fragEnd is the end time of the segment. After setting the current time, the HLS player loads the corresponding slice and continues to trigger the BUFFER_APPENDING event callback, repeating the above logic until fragEnd is greater than or equal to end, at which point waveform drawing is executed. First, an arrayBuffer is extracted from the recorded segments, concatenating the data from slices M to N into an arrayBuffer. Then, the WebAudio API is used to decode the arrayBuffer data, obtaining audioData. Next, the offset and frame number are calculated based on the start and end times (start-firstFragStart, end-firstFragStart) and the sampling rate. Data is then segmented from the audio channels as the audioData for waveform drawing. Finally, using the resampling technique mentioned above, the amplitude data of the peaks and troughs is sampled, and the waveform is drawn based on this data.

[0039] This application also provides a video segmentation method based on the above-described audio waveform generation method, such as... Figure 2 As shown, it includes the following steps: S301: Obtain the original video file and perform audio data separation operation on the original video file to obtain the audio data file.

[0040] S302: Perform the operation of any audio waveform generation method as described in the embodiments of this application on the audio data file to obtain a first waveform.

[0041] S303: Determine the target position of the target waveform in the first waveform diagram based on the characteristics of the preset target waveform.

[0042] After acquiring the waveform data, the waveform diagram can intuitively display the waveform of the sound information contained in the current audio. Since different sound information has different waveform characteristics, given the waveform characteristics of the target sound, target waveform recognition can be performed in the waveform diagram to quickly determine the position of the target sound waveform in the current waveform diagram.

[0043] In this embodiment, the target sound is human voice, and the target waveform feature is the human voice waveform feature. In a specific application scenario, taking a concert as an example, the acquired audio data file generally includes human voice information and background sound information. The waveform features of the background sound often exhibit continuous correlation, while the waveform features of the human voice are discontinuous relative to the background sound waveform features, and its waveform amplitude is larger. When the frequency range (peaks and troughs) continuously exceeds the threshold range of the background sound waveform features, the presence of human voice can be determined. By distinguishing the aforementioned waveform features, the position of the human voice waveform in the waveform diagram can be confirmed. In some other examples, the target waveform can also be other types of sound, such as a whistle. In some other examples, the target waveform can also be identified based on other sound waveform features, such as using frequency and other features to determine the position of the target waveform.

[0044] S304: Confirm that the target location is the in-segment entry point of the original video file.

[0045] When there is a need to segment a video file at the location of a target sound, the position where the target sound appears in the current video file can be determined based on the target location, and this position can be used as the entry point for segmentation. The entry point indicates the location where the target sound waveform appears in the current waveform graph, and video segmentation can be performed at this point. If the target waveform feature is not identified in the current waveform graph, it is determined that the target sound does not appear in the current video file, and segmentation will not be performed at the current video file location.

[0046] In a preferred embodiment, since the first waveform is a coarse waveform and can only partially represent the waveform features of the audio file, a fine waveform needs to be drawn for the audio file to better confirm the target waveform features, thereby obtaining second waveform data. The audio waveform generation method described above is performed on the audio data file to obtain the second waveform. Based on the characteristics of the preset target waveform, the target position of the target waveform in the second waveform is determined. Then, based on the target position in the second waveform, the in-segmentation point of the original video file is adjusted.

[0047] Because the second waveform is a refined waveform drawn after full decoding of the sliced ​​file, it can obtain a more accurate amplitude (peak and trough) sequence. Furthermore, because it contains time information, the target waveform located in the second waveform is more representative of the location of the target sound within the video file. Therefore, the same target waveform feature recognition operation performed in the first waveform can be performed on the second waveform.

[0048] At this point, the following three situations may exist: (1) If there is a strip entry point in the first waveform diagram, it is determined that there is a target waveform near the strip entry point in the second waveform diagram. The target position of the target waveform in the second waveform diagram is re-identified according to the characteristics of the target waveform. Since the second waveform diagram has higher precision than the first waveform diagram, the target waveform position confirmed in the second waveform diagram is more accurate than that in the first waveform diagram. Therefore, the strip entry point position is adjusted according to the target waveform position in the second waveform diagram.

[0049] (2) If there is no strip entry point in the first waveform diagram, the target waveform feature is identified in the second waveform diagram. If there is a target waveform in the second waveform diagram, the position of the target waveform confirmed in the second waveform diagram is confirmed as the strip entry point position.

[0050] (3) If no target waveform features are detected in either the first waveform diagram or the second waveform diagram, it is confirmed that the video file does not contain target sound information, and the video file is not split at the current video file position.

[0051] Coarse sampling of undecoded data quickly generates coarse-grained waveforms, enabling rapid user feedback to meet the demands of high real-time performance. Simultaneously, the coarse-grained waveform pre-determines the location of the segmentation entry point. When identifying the target waveform in the fine-grained waveform, this entry point is used as a "temporary segmentation entry point." Identifying target waveform features near this temporary entry point allows for faster location of the actual target waveform. The actual segmentation entry point is then adjusted based on this location, leading to even faster confirmation of the video segmentation position.

[0052] However, due to the lack of timing information in the pre-decoding data, the coarse waveform diagram's correspondence with time is not precise enough, and the number of samples is limited. Therefore, the audio information contained in the coarse waveform diagram is limited. It's possible that a target sound appears in the video file, but the corresponding waveform is not detected by target waveform feature identification in the coarse waveform diagram. Therefore, it's necessary to perform target waveform feature identification again on the precise waveform diagram generated from the decoded data, which includes the timing information corresponding to the audio data, to prevent the omission of target sounds. Simultaneously, because the adjusted segmentation entry point is more precise, it also avoids the possibility of needing manual secondary adjustments due to inaccurate segmentation entry points identified in the coarse waveform diagram.

[0053] In a preferred embodiment, after confirming that the target position is the entry point for splitting the original video file, the method further includes: displaying the entry point for splitting the original video file in the waveform diagram, or splitting the original video file according to the first entry point for splitting the original video file.

[0054] After confirming the final splitting entry point, the system can automatically add this entry point to the video splitting track and create a temporary splitting segment to achieve video splitting. Alternatively, the position of the splitting entry point can be displayed in the first and / or second waveform graphs, prompting relevant personnel to split the video at this location, thus assisting in the rapid splitting and generation of short video content. The position of the splitting entry point can be displayed in both the first and second waveform graphs. When the splitting entry point exists in the second waveform graph, it will be prioritized or highlighted because the second waveform graph has higher resolution.

[0055] This application embodiment acquires audio segment files and performs coarse sampling on the audio segment files to quickly generate coarse-grained waveform diagrams, achieving rapid acquisition of audio waveform features to meet the needs of strong real-time requirements. Simultaneously, the segment files are decoded, and fine sampling is performed on the decoded segment files to obtain a more accurate amplitude sequence, thereby drawing fine-grained audio waveforms. Furthermore, by storing and comparing the coarse-grained amplitude data of the most recent segments, target waveform features are identified, thereby roughly determining the position of the target waveform, automatically adding an in-point to the segmentation track, and creating temporary segmentation fragments. After obtaining the decoded audio waveform diagram, the target waveform is identified again and its position is obtained, and the segmentation in-point is adjusted. This application solves the problem of failure due to excessive browser memory usage when drawing audio waveform diagrams of large files or DC streams, ensuring the efficiency and accuracy of audio waveform diagram drawing. At the same time, this waveform diagram drawing method can realize automatic segmentation of large files or live video streams, saving labor costs.

[0056] This application embodiment also provides an audio waveform generation device 400, such as... Figure 3 As shown, it includes: The slice file generation module is used to acquire the original audio, slice the original audio, and generate slice files; The first sampling data acquisition module is used to sample the slice file according to a preset sampling rule to acquire first sampling data; The first waveform generation module is used to decode the first sampled data and generate the first waveform.

[0057] This application embodiment also provides a video strip splitting device 500, such as... Figure 4 As shown, it includes: The audio data acquisition module is used to acquire the original video file and perform audio data separation operation on the original video file to obtain an audio data file; The first waveform acquisition module is used to perform the above-described audio waveform generation method on the audio data file to acquire the first waveform data; The target waveform position determination module is used to determine the target position of the target waveform in the first waveform diagram based on the characteristics of the preset target waveform. The segment entry point confirmation module is used to confirm that the target position is the segment entry point of the original video file.

[0058] It should be noted that the audio waveform generation device and the audio waveform generation method provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here. Similarly, the video segmentation device and the video segmentation method provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will also not be repeated here.

[0059] like Figure 5 As shown, Figure 5 This is a structural block diagram of an electronic device according to an exemplary embodiment of this application.

[0060] The electronic device includes a processor 910 and a memory 920. The main control chip may contain one or more processors 910. Figure 5 Taking a processor 910 as an example, the main control chip can contain one or more memory modules 920. Figure 5 Take a memory chip 920 as an example.

[0061] The memory 920, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the audio waveform generation method and video segmentation method program described in any embodiment of this application, and the program instructions / modules corresponding to the audio waveform generation method and video segmentation method described in any embodiment of this application. The memory 920 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created according to the use of the device, etc. Furthermore, the memory 920 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 920 may further include memory remotely located relative to the processor 910, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0062] The processor 910 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 920, thereby implementing an audio waveform generation method and a video segmentation method as described in any of the above embodiments.

[0063] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an audio waveform generation method and a video segmentation method as described in any of the above embodiments.

[0064] This invention can take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0065] It should be understood that the embodiments of this application are not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from their scope. The scope of the embodiments of this application is limited only by the appended claims.

[0066] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the embodiments of this application, and these all fall within the protection scope of the embodiments of this application.

Claims

1. A method for generating audio waveforms, characterized in that, include: Obtain the original audio, slice the original audio, and generate slice files; According to the preset sampling rules, the sliced ​​file is sampled to obtain the first sampling data; The first sampled data is decoded to generate a first waveform. The first waveform diagram is used to determine the in-segment entry point of the target waveform in the original video file; The acquisition of the first sampled data includes: The number of samplings is determined based on the width of the waveform of the slice file; the sampling operation is performed uniformly on each slice file for the number of samplings to obtain the first sampled data; wherein, the number of samplings is determined based on the number of pixels in the horizontal direction of the dom width of the slice file, and each pixel corresponds to one sampling count; After generating the first waveform, the following is also included: Perform a full decoding operation on the slice file to obtain the decoded data of the slice file; The decoded data of the slice file is sampled to obtain second sampled data, wherein the sampling number of the second sampled data is greater than the sampling number of the first sampled data; A second waveform is generated based on the second sampled data; the second waveform is used to adjust the in-point of the target waveform in the original video file.

2. A method for segmenting video clips, characterized in that, Includes the following steps: Obtain the original video file and perform audio data separation operation on the original video file to obtain the audio data file; The audio data file is subjected to the operation of the audio waveform generation method as described in claim 1 to obtain the first waveform and the second waveform; Based on the characteristics of the preset target waveform, determine the target position of the target waveform in the first waveform diagram; Confirm that the target location is the in-segment entry point of the original video file; Based on the characteristics of the preset target waveform, determine the target position of the target waveform in the second waveform diagram; Adjust the in-point of the original video file based on the target position in the second waveform diagram.

3. The video segmentation method according to claim 2, characterized in that, After confirming that the target location is the in-point of the original video file segmentation, the method further includes: The strip removal entry point is shown in the first waveform diagram and / or the second waveform diagram. or The original video file is split into segments based on the stated segmentation entry point.

4. An audio waveform generation device, characterized in that, include: The slice file generation module is used to acquire the original audio, slice the original audio, and generate slice files; The first sampling data acquisition module is used to sample the slice file according to a preset sampling rule to acquire first sampling data; The first waveform generation module is used to decode the first sampled data and generate a first waveform. The first waveform diagram is used to determine the in-segment entry point of the target waveform in the original video file; The acquisition of the first sampled data includes: The number of samplings is determined based on the width of the waveform of the slice file; the sampling operation is performed uniformly on each slice file for the number of samplings to obtain the first sampled data; wherein, the number of samplings is determined based on the number of pixels in the horizontal direction of the dom width of the slice file, and each pixel corresponds to one sampling count; After generating the first waveform, the following is also included: Perform a full decoding operation on the slice file to obtain the decoded data of the slice file; The decoded data of the slice file is sampled to obtain second sampled data, wherein the sampling number of the second sampled data is greater than the sampling number of the first sampled data; A second waveform is generated based on the second sampled data; the second waveform is used to adjust the in-point of the target waveform in the original video file.

5. A video strip splitting device, characterized in that, include: The audio data acquisition module is used to acquire the original video file and perform audio data separation operation on the original video file to obtain an audio data file; The first waveform acquisition module is used to perform the audio waveform generation method as described in claim 1 on the audio data file to acquire the first waveform and the second waveform. The target waveform position determination module is used to determine the target position of the target waveform in the first waveform diagram based on the characteristics of the preset target waveform. The splitting entry point confirmation module is used to confirm that the target position is the splitting entry point of the original video file; Based on the characteristics of the preset target waveform, determine the target position of the target waveform in the second waveform diagram; based on the target position in the second waveform diagram, adjust the in-point of the original video file.

6. An electronic device, characterized in that, include: At least one memory and at least one processor; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the at least one processor performs the steps of the method as described in any one of claims 1 to 2.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 2.

Images