Waveform Diagram Drawing Method, Device, Electronic Device and Storage Medium

Abstract

The present disclosure relates to a waveform drawing method, device, electronic device, and storage medium. The method includes: when a first waveform of audio is displayed in a video editing interface, receiving an instruction for adjusting the insertion time of the audio, the first waveform including waveform segments corresponding to each audio segment; in response to the instruction, determining the time interval corresponding to each waveform segment according to the adjusted insertion time, the time interval being the time interval corresponding to the waveform segment on the video timeline after the audio is superimposed on the video according to the adjusted insertion time; determining a time interval that overlaps with at least one variable speed playback interval of the video as a target time interval; and when at least one target time interval is determined, redrawing the waveform segment corresponding to the target time interval according to the video playback speed corresponding to the target time interval to obtain a second waveform, the second waveform being used to update the first waveform.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a waveform graph drawing method, device, electronic device, and storage medium. Background Art

[0002] Users can use the audio and video editing application installed on the terminal to overlay music into the video and add music to the video, for example, use the music as background music for a certain video segment or the entire video.

[0003] In related art, audio and video editing applications display a waveform graph with music volume as the vertical axis and video length as the horizontal axis, allowing users to control points on the waveform graph to perform editing operations such as pausing or cutting the audio and video. However, when users re-edit audio and video, that is, re-overlay music on the video, the music waveform graph takes a long time to update, resulting in long-term lags on the user's terminal device and increased battery consumption. Summary of the Invention

[0004] The present disclosure provides a waveform graph drawing method, device, electronic device, and storage medium to at least solve the problem of long waveform graph update time in related technologies. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a waveform drawing method is provided, comprising: when a first waveform of audio is displayed in an editing interface of a video, receiving an instruction for adjusting the insertion time of the audio, the audio comprising at least one audio segment, and the first waveform comprising a waveform segment corresponding to each audio segment; in response to the instruction, determining a time interval corresponding to each waveform segment according to the adjusted insertion time, the time interval referring to a time interval corresponding to the waveform segment on a video timeline after the audio is superimposed on the video according to the adjusted insertion time; determining a time interval overlapping with at least one variable-speed playback interval of the video as a target time interval, the variable-speed playback interval referring to a time interval corresponding to a video segment whose playback speed occurs relative to a preset playback speed; when at least one target time interval is determined, redrawing the waveform segment corresponding to the target time interval according to the video playback speed corresponding to the target time interval to obtain a second waveform graph, the second waveform graph being used to update the first waveform graph.

[0006] In one possible implementation, based on the video playback speed corresponding to the target time interval, the waveform segment corresponding to the target time interval is redrawn to obtain a second waveform graph, including: determining the multiple sampling time points included in the target time interval based on a preset sampling period; determining the video length corresponding to each preset sampling period based on the video playback speed corresponding to the sampling time point; and drawing the waveform segment corresponding to each target time interval based on the video length and audio volume corresponding to each sampling time period to obtain a second waveform graph.

[0007] In another possible implementation, a time interval that overlaps with at least one variable speed playback interval of a video is determined as a target time interval, including: obtaining the latest variable speed playback interval sequence and the latest time interval sequence, the latest variable speed playback interval sequence refers to the variable speed playback interval sequence or the initial variable speed playback interval sequence updated according to the last comparison result, the latest variable speed playback interval sequence includes at least one variable speed playback interval arranged in chronological order, the latest time interval sequence refers to the time interval sequence or the initial time interval sequence updated according to the last comparison result, the latest time interval sequence includes at least one time interval arranged in chronological order; comparing the first variable speed playback interval in the latest variable speed playback interval sequence with the first time interval in the latest time interval sequence to obtain a comparison result; determining whether the first time interval in the latest time interval sequence is the target time interval based on the comparison result, and updating the latest variable speed playback interval sequence or the latest time interval sequence based on the comparison result.

[0008] In another possible implementation, before obtaining the latest variable speed playback interval sequence and the latest time interval sequence, the method also includes: arranging all variable speed playback intervals in chronological order to obtain an initial variable speed playback interval sequence, and arranging all time intervals in chronological order to obtain an initial time interval sequence.

[0009] In another possible implementation, whether the first time interval in the latest time interval sequence is the target time interval is determined based on the comparison result, including: if the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval, and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval, and less than or equal to the end time of the first time interval, the first time interval is determined as the target time interval.

[0010] In another possible implementation, the latest variable speed playback interval sequence or the latest time interval sequence is updated according to the comparison result, including: if the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval, and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval, and less than or equal to the end time of the first time interval, then the first time interval is deleted from the latest time interval sequence to update the latest time interval sequence; if the end time of the first variable speed playback interval is less than the start time of the first time interval, then the first variable speed playback interval is deleted from the latest variable speed playback interval sequence to update the latest variable speed playback interval sequence; if the start time of the first variable speed playback interval is greater than the end time of the first time interval, then the first time interval is deleted from the latest time interval to update the latest time interval sequence.

[0011] In another possible implementation, a time interval that overlaps with at least one variable-speed playback interval of the video is determined as a target time interval. The method also includes: comparing each variable-speed playback interval with each time interval in turn, and taking the time interval that overlaps with any variable-speed playback interval as the target time interval.

[0012] In another possible implementation, before determining the time interval corresponding to each waveform segment based on the adjusted insertion time in response to an instruction, the method further includes: dividing the first waveform into at least one waveform segment in chronological order based on the start time and end time of the audio in the first waveform to be superimposed on the video.

[0013] According to a second aspect of an embodiment of the present disclosure, a waveform drawing device is provided, the device comprising: a receiving unit, configured to receive an instruction for adjusting the insertion time of audio when a first waveform diagram of audio is displayed in an editing interface of a video, the audio comprising at least one audio segment, and the first waveform comprising a waveform segment corresponding to each audio segment; a first determining unit, configured to determine, in response to the instruction, a time interval corresponding to each waveform segment according to the adjusted insertion time, the time interval being a time interval on a video timeline corresponding to the waveform segment after the audio is superimposed on the video according to the adjusted insertion time; a second determining unit, configured to determine a time interval that overlaps with at least one variable-speed playback interval of the video as a target time interval, the variable-speed playback interval being a time interval corresponding to a video segment whose playback speed is relative to a preset playback speed; a drawing unit, configured to, upon determining at least one target time interval, redraw the waveform segment corresponding to the target time interval according to the video playback speed corresponding to the target time interval to obtain a second waveform, the second waveform being used to update the first waveform.

[0014] In one possible embodiment, the drawing unit is configured to specifically perform: determining multiple sampling time points included in the target time interval based on a preset sampling period; determining the video length corresponding to each preset sampling period based on the video playback speed corresponding to the sampling time point; and drawing the waveform segment corresponding to each target time interval based on the video length and audio volume corresponding to each sampling time period to obtain a second waveform.

[0015] In another possible implementation, the second determination unit is configured to specifically perform: obtaining the latest variable speed playback interval sequence and the latest time interval sequence, the latest variable speed playback interval sequence refers to the variable speed playback interval sequence or the initial variable speed playback interval sequence updated according to the last comparison result, the latest variable speed playback interval sequence includes at least one variable speed playback interval arranged in chronological order, the latest time interval sequence refers to the time interval sequence or the initial time interval sequence updated according to the last comparison result, the latest time interval sequence includes at least one time interval arranged in chronological order; comparing the first variable speed playback interval in the latest variable speed playback interval sequence with the first time interval in the latest time interval sequence to obtain a comparison result; determining whether the first time interval in the latest time interval sequence is the target time interval based on the comparison result, and updating the latest variable speed playback interval sequence or the latest time interval sequence based on the comparison result.

[0016] In another possible implementation, before obtaining the latest variable speed playback interval sequence and the latest time interval sequence, the second determination unit is configured to further specifically perform: arranging all variable speed playback intervals in chronological order to obtain an initial variable speed playback interval sequence, and arranging all time intervals in chronological order to obtain an initial time interval sequence.

[0017] In another possible implementation, the second determination unit is configured to specifically execute: if the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval, and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval, and less than or equal to the end time of the first time interval, determine the first time interval as the target time interval.

[0018] In another possible implementation, if the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, then the first time interval is deleted from the latest time interval sequence to update the latest time interval sequence; if the end time of the first variable speed playback interval is less than the start time of the first time interval, then the first variable speed playback interval is deleted from the latest variable speed playback interval sequence to update the latest variable speed playback interval sequence; if the start time of the first variable speed playback interval is greater than the end time of the first time interval, then the first time interval is deleted from the latest time interval to update the latest time interval sequence.

[0019] In another possible implementation, the second determining unit is configured to specifically perform: comparing each variable speed playback interval with each time interval in turn, and taking a time interval that overlaps with any variable speed playback interval as a target time interval.

[0020] In another possible implementation, the device further includes: a segmentation unit configured to segment the first waveform into at least one waveform segment in chronological order according to the start time and end time of the audio in the first waveform to be superimposed on the video.

[0021] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor and a memory for storing processor executable instructions; wherein the processor is configured to execute the executable instructions to implement a waveform drawing method as in the first aspect and any possible implementation manner thereof.

[0022] According to the fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an image processing device or an electronic device, the image processing device or the electronic device is enabled to perform a waveform graph drawing method as in the first aspect and any possible implementation method thereof.

[0023] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor as the waveform diagram drawing method of the first aspect and any possible implementation thereof.

[0024] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: based on the adjusted insertion time, the time interval for the audio to be re-superimposed on each waveform segment in the video is determined. Then, based on the overlap between the time interval of each waveform segment and the variable speed playback interval of the video, the waveform segment in the waveform graph where the waveform has undergone local changes is detected and identified, so that when the audio and video resources are re-edited, only the waveform segment corresponding to the changed local waveform needs to be redrawn to update the entire waveform graph. Compared with the related art, which requires the entire waveform graph to be redrawn, the waveform graph drawing time is greatly shortened, thereby greatly reducing the waveform graph update time in the terminal device, and further greatly reducing the power consumption of the terminal device.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0027] Figure 1 is a schematic diagram of an electronic device according to an exemplary embodiment;

[0028] Figure 2 is a flow chart showing a method for drawing a waveform diagram according to an exemplary embodiment;

[0029] Figure 3 is a schematic diagram showing a waveform diagram before adjustment according to an exemplary embodiment;

[0030] Figure 4 is a schematic diagram showing an adjusted waveform diagram according to an exemplary embodiment;

[0031] Figure 5 The figure is a block diagram of a waveform graph drawing device according to an exemplary embodiment. DETAILED DESCRIPTION

[0032] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0033] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0034] Before introducing the waveform graph drawing method provided by the embodiment of the present disclosure in detail, a brief introduction to the implementation environment and application scenarios involved in the embodiment of the present disclosure is first given.

[0035] First, a brief introduction to the application scenarios involved in this disclosure is given.

[0036] In related technologies, a waveform is drawn by overlaying the music volume and the length of the video at each time point in the video. This waveform is then displayed on the audio and video editing application interface, allowing users to control points on the waveform to perform editing operations such as pausing or cutting the audio and video. However, when users re-edit the audio and video, that is, re-overlay the music on the video, the update time of the music waveform is very long, resulting in long-term lag and increased battery consumption on the user's terminal device.

[0037] In response to the above-mentioned problem, the present disclosure provides a waveform graph drawing method, which redetermines the time interval corresponding to each waveform graph segment included in the first waveform graph when the insertion time of audio superimposed on the video changes. At the same time, the time interval corresponding to the variable-speed playback video clip that changes relative to the preset playback speed is determined as the variable-speed playback interval of the video. Among the time intervals of the above-mentioned re-determined waveform graph segments, the time interval of the waveform graph segments that overlap with the variable-speed playback interval of the video is determined as the target time interval, so as to detect the local waveform graph in which the waveform in the first waveform graph changes, which is the waveform graph segment corresponding to the target time interval. Thus, by redrawing the waveform graph segment corresponding to the target time interval, the desired second waveform graph is obtained.

[0038] The waveform drawing method provided by the embodiment of the present disclosure determines the time interval for re-superimposing the audio onto each waveform segment in the video based on the adjusted insertion time. Furthermore, based on the overlap between the time interval of each waveform segment and the variable speed playback interval of the video, the waveform segments in the waveform where local waveform changes occur are detected and identified, so that when re-editing the audio and video resources, only the waveform segments corresponding to the changed local waveforms need to be redrawn to update the entire waveform. Compared to the related art, which requires redrawing the entire waveform, the waveform drawing time is greatly shortened, thereby greatly reducing the waveform update time in the terminal device, and thus greatly reducing the power consumption of the terminal device.

[0039] Next, the implementation environment (implementation architecture) involved in this disclosure is briefly introduced below.

[0040] The waveform graph drawing method provided in the embodiments of the present disclosure can be applied to electronic devices. The electronic device can be a terminal device or a server. The terminal device can be a smartphone, tablet computer, PDA, vehicle-mounted terminal, desktop computer, laptop computer, etc. The server can be any server or server cluster, which is not limited in this disclosure.

[0041] In addition, it should be noted that the user information involved in this disclosure (including but not limited to user device information, user personal information, etc.) is all information authorized by the user account or fully authorized by all parties.

[0042] Figure 1 Schematic diagram of an electronic device provided by the present disclosure. Figure 1 The electronic device 10 may include at least one processor 101 and a memory 103 for storing processor-executable instructions. The processor 101 is configured to execute instructions in the memory 103 to implement the waveform drawing method in the following embodiment.

[0043] In addition, the electronic device 10 may further include a communication bus 102 , at least one communication interface 104 , an output device 105 and an input device 106 .

[0044] The processor 101 may be a central processing unit (CPU), a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of the program of the disclosed solution.

[0045] The communication bus 102 may include a pathway for transmitting information between the aforementioned components.

[0046] The communication interface 104 , which uses any transceiver or other device, is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0047] The memory 103 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processing unit via a bus. The memory may also be integrated with the processing unit.

[0048] The memory 103 is used to store instructions for executing the solution of the present disclosure, and the execution is controlled by the processor 101. The processor 101 is used to execute the instructions stored in the memory 103, thereby realizing the functions of the method of the present disclosure.

[0049] In a specific implementation, as an embodiment, the processor 101 may include one or more CPUs, such as Figure 1 CPU0 and CPU1 in.

[0050] In a specific implementation, as an embodiment, the electronic device 10 may include multiple processors, such as Figure 1 1 and 107. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0051] For ease of understanding, the waveform diagram drawing method provided by the present disclosure is specifically introduced below with reference to the accompanying drawings.

[0052] Figure 2FIG. 1 is a flow chart showing a method for drawing a waveform diagram according to an exemplary embodiment. Figure 2 As shown, the waveform diagram drawing method is used in an electronic device and includes the following steps.

[0053] Step S21: When the first waveform of the audio is displayed in the video editing interface, an instruction for adjusting the insertion time of the audio is received.

[0054] The audio includes at least one audio segment, and the first waveform includes a waveform segment corresponding to each audio segment.

[0055] In one embodiment, when an instruction for adjusting the insertion time of audio is received, that is, when the superimposition time of audio onto video is changed, the waveform segments included in the first waveform are acquired.

[0056] As a possible implementation method, before implementing step S21 or after implementing step S21, the waveform segment of the first waveform is obtained through the following step S211, that is, the waveform segment of the original waveform of the audio displayed on the video editing interface before the change is obtained.

[0057] Step S211: According to the start time and end time of the audio in the first waveform to be superimposed on the video, the first waveform is divided into at least one waveform segment in chronological order.

[0058] In this embodiment, the first waveform is divided into N waveform segments based on the start and end times of the audio being superimposed on the video, so that each waveform segment of the first waveform corresponds one-to-one with the time on the video's timeline. This approach allows the waveform segments included in the first waveform to be retrieved based on the start and end times of the audio being superimposed on the video. This ensures that the time intervals corresponding to the retrieved waveform segments and the time intervals corresponding to the variable-speed playback of the video are identical, namely, the time on the video's timeline, thereby ensuring the accuracy and reliability of the waveform drawing.

[0059] Step S22: In response to the instruction, determine the time interval corresponding to each waveform segment according to the adjusted insertion time.

[0060] The time interval of each waveform segment mentioned above refers to the time interval on the video time axis corresponding to the waveform segment after the audio is superimposed on the video according to the adjusted insertion time.

[0061] As an embodiment, the time interval corresponding to each waveform segment is determined by the time difference between the adjusted insertion time and the original insertion time of the first waveform graph displaying the audio in the video editing interface. For example, the original insertion time of the first waveform graph is 1 second, including 5 waveform time periods, and the original time intervals corresponding to the 5 waveform time periods are [1s, 2s], [2s, 3s], [3s, 4s], [4s, 5s] and [5s, 6s]. If the adjusted insertion time is 2 seconds, the time difference between the two insertion times is (-1) second, and the time interval corresponding to the original waveform segment is subtracted from the time difference to obtain the time interval corresponding to the adjusted waveform segment. That is, the time interval corresponding to the adjusted waveform segment is [2s, 3s], [3s, 4s], [4s, 5s], [5s, 6s] and [6s, 7s].

[0062] As another embodiment, the adjusted insertion time and the total time duration of the entire first waveform on the video axis are used. For example, if the entire duration of the first waveform is 5 seconds, if the adjusted insertion time is 2 seconds, and the number of waveform segments included in the first waveform is determined to be 5, then the time intervals corresponding to the adjusted waveform segments are [2s, 3s], [3s, 4s], [4s, 5s], [5s, 6s], and [6s, 7s].

[0063] Step S23: determining a time interval that overlaps with at least one variable-speed playback interval of the video as a target time interval.

[0064] It should be noted that the variable speed playback interval refers to the time interval corresponding to the video clip whose playback speed changes relative to the preset playback speed. It can also be understood as the time interval corresponding to the variable speed playback video clip whose playback speed on the video timeline changes relative to the preset playback speed.

[0065] Among them, the time interval that overlaps with at least one variable speed playback interval of the video can be understood as that the time intervals of each waveform segment that overlaps with any variable speed playback interval among the time intervals corresponding to all waveform segments are determined as target time intervals.

[0066] In some embodiments, the variable speed playback interval can be a pre-stored variable speed playback interval, or can be detected by detecting the video playback speed when the insertion time changes. The overlap of the time interval and the variable speed playback interval can be referred to as the time interval and the variable speed playback interval having an intersection.

[0067] As a possible implementation, the above step S23 can be specifically implemented through the following steps S231 to S233.

[0068] Step S231: Obtain the latest variable-speed playback interval sequence and the latest time interval sequence.

[0069] Among them, the latest variable speed playback interval sequence refers to the variable speed playback interval sequence or the initial variable speed playback interval sequence updated according to the last comparison result, and the latest variable speed playback interval sequence includes at least one variable speed playback interval arranged in chronological order. The latest time interval sequence refers to the time interval sequence or the initial time interval sequence updated according to the last comparison result, and the latest time interval sequence includes at least one time interval arranged in chronological order.

[0070] The last comparison result is the last comparison result relative to the first updated time interval and the first updated variable-speed playback interval.

[0071] In one embodiment, before performing the above step S231 , the following step S2311 may be further performed.

[0072] Step S2311: Arrange all variable speed playback intervals in chronological order to obtain an initial variable speed playback interval sequence, and arrange all time intervals in chronological order to obtain an initial time interval sequence.

[0073] In this embodiment, all time intervals are sorted chronologically to form an initial time interval sequence, and all intervals to be speed-changed are arranged chronologically to form an initial speed-changed playback interval sequence. The latest speed-changed playback interval sequence is then designated as the initial speed-changed playback interval sequence, and the latest time interval sequence is designated as the initial speed-changed playback interval sequence. By initializing and assigning values to the latest speed-changed playback interval sequence and the latest time interval sequence, the feasibility of the aforementioned cyclic comparison and update process for determining the target interval is ensured.

[0074] Step S232: Compare the first variable speed playback interval in the latest variable speed playback interval sequence with the first time interval in the latest time interval sequence to obtain a comparison result.

[0075] Step S233: determining whether the first time interval in the latest time interval sequence is the target time interval based on the comparison result, and updating the latest variable speed playback interval sequence or the latest time interval sequence based on the comparison result.

[0076] In one embodiment, step S233 can be implemented by executing steps S2331 through S2334. Step S2331 determines whether the first time interval in the latest time interval sequence is the target time interval. Steps S2332 through S2334 update the latest variable-speed playback interval sequence or the latest time interval sequence. This application does not specifically limit the order in which steps S2331 through S2334 are executed.

[0077] Step S2331: If the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, the first time interval is determined as the target time interval.

[0078] In this embodiment, the target time interval is determined by comparing the start time and end time corresponding to the updated first time interval and the updated first variable speed playback interval. The implementation process logic is simple and easy to operate, which simplifies the detection and determination process of the waveform segments in the waveform graph where the waveform has changed, thereby speeding up the waveform graph drawing process.

[0079] Step S2332: If the start time of the first variable-speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, and / or if the end time of the first variable-speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, then delete the first time interval from the latest time interval sequence to update the latest time interval sequence;

[0080] Step S2333: If the end time of the first variable speed playback interval is less than the start time of the first time interval, the first variable speed playback interval is deleted from the latest variable speed playback interval sequence to update the latest variable speed playback interval sequence;

[0081] Step S2334: If the start time of the first variable-speed playback interval is greater than the end time of the first time interval, the first time interval is deleted from the latest time interval to update the latest time interval sequence.

[0082] In this embodiment, the start and end times of the updated first time interval and the updated first variable-speed playback interval are compared to update the variable-speed playback interval sequence and the time interval sequence. This implementation process is simple and easy to operate, and simplifies the process of detecting and determining waveform segments in the waveform graph where the waveform has changed. This simplification of the process of detecting and determining waveform segments in the waveform graph where the waveform has changed speeds up the waveform graph drawing process.

[0083] In a specific embodiment, first, all time intervals are sorted in chronological order to form a time interval sequence, and all intervals to be speed-changed are arranged in chronological order to form a speed-changed playback interval sequence. Secondly, the first time interval and the first speed-changed playback interval in the above-mentioned time interval sequence and speed-changed playback interval sequence are respectively extracted and compared to obtain a comparison result. Thirdly, based on the comparison result, it is determined whether the first time interval is a target interval, and the time interval sequence or speed-changed playback interval sequence is updated. Thirdly, the updated time interval sequence and the first time interval and the first speed-changed playback interval in the speed-changed playback interval sequence are compared to obtain a comparison result. Thirdly, based on the comparison result, it is determined whether the updated first time interval is a target interval, and the updated time interval sequence or the updated speed-changed playback interval sequence is updated again. The comparison and update process is repeated in sequence until the updated time interval sequence does not contain any time intervals or the speed-changed playback interval sequence does not contain any speed-changed playback intervals.

[0084] In some embodiments, the process of determining the target interval may be achieved by using a priority queue algorithm.

[0085] In the above embodiment, the first variable-speed playback interval in the latest variable-speed playback interval sequence is compared with the first time interval in the latest time interval sequence. Based on the comparison results, a target time interval, an updated variable-speed playback interval sequence, or an updated time interval sequence is determined, and the updated variable-speed playback interval sequence and the updated time interval sequence are used as the latest variable-speed playback interval sequence and the latest time interval sequence, respectively. By sequentially repeating the above steps of comparing the updated first time interval with the updated first variable-speed playback interval, and updating the variable-speed playback interval sequence and the time interval sequence, the target time interval can be quickly and accurately determined, thereby improving the speed and accuracy of determining waveform segments in which waveform changes occur in the waveform graph, thereby improving the speed of waveform drawing.

[0086] As another possible implementation, the above step S23 may also be specifically implemented through the following step S234.

[0087] Step S234: Compare each variable speed playback interval with each time interval in turn, and take the time interval that overlaps with any variable speed playback interval as the target time interval.

[0088] In this embodiment, all speed-varying playback intervals are searched once for each waveform segment from the beginning to the end. In this manner, when there are only a few speed-varying playback intervals or waveform segments, the speed-varying playback intervals or waveform segments are directly compared one by one to detect and determine the waveform segments in which the waveform has changed.

[0089] Step S24: When at least one target time interval is determined, the waveform segment corresponding to the target time interval is redrawn according to the video playback speed corresponding to the target time interval to obtain a second waveform, which is used to update the first waveform.

[0090] As an implementation method, the original waveform segment corresponding to the target time in the first waveform is removed, and the waveform segment corresponding to the target time interval is redrawn on the removed first waveform directly according to the video playback speed corresponding to the target time interval, and the second waveform is directly obtained after drawing.

[0091] As another implementation method, the waveform segment corresponding to the target time interval is redrawn according to the video playback speed corresponding to the target time interval, and the redrawn waveform segment replaces the original waveform segment of the target time interval to obtain a second waveform.

[0092] Furthermore, as a possible implementation, the above step S24 can be specifically implemented through the following steps S241 to S243.

[0093] Step S241: determining a plurality of sampling time points included in the target time interval according to a preset sampling period;

[0094] Step S242: determining the video length corresponding to each preset sampling period according to the video playback speed corresponding to the sampling time point;

[0095] Step S243: According to the video length and audio volume corresponding to each sampling time period, a waveform segment corresponding to each target time interval is drawn to obtain a second waveform.

[0096] As a possible implementation method, the target interval is divided into N equal parts according to a preset sampling period, and the above steps S241 to S243 can be specifically implemented in the following steps 1 to 4.

[0097] Step 1: determine the start time of the waveform segment of each target interval;

[0098] Step 2: Determine the sampling time width, i.e., the preset sampling period;

[0099] Step 3: Obtain the video length corresponding to each sampling period and the audio volume corresponding to each sampling period;

[0100] Step 4: Redraw the waveform segment corresponding to the target interval using the video length as the horizontal axis and the audio volume as the vertical axis.

[0101] Among them, the drawing of the waveform segment of the target interval is the same as Figure 3 and Figure 4 The waveform drawing process is the same as that in the video. The video length unit is frame and the audio volume unit is decibel. The waveform segment corresponding to the target time interval to be redrawn is as follows Figure 4 The waveform segment corresponding to [11s, 15s] in .

[0102] In this embodiment, according to a preset sampling period, the video playback speed and audio volume at each sampling time point are obtained by sampling, thereby achieving the drawing of a local waveform graph where the waveform in the waveform graph changes.

[0103] In the above embodiment, when the insertion time of audio superimposed on the video changes, the time intervals corresponding to the various waveform segments included in the first waveform graph are re-determined. Simultaneously, the time interval corresponding to the variable-speed video segment that changes relative to the preset playback speed is determined as the variable-speed playback interval of the video. Among the re-determined time intervals of the waveform segments, the time intervals of the waveform segments that overlap with the variable-speed playback interval of the video are determined as the target time interval, so that the local waveform graph in the first waveform graph where the waveform changes is detected as the waveform segment corresponding to the target time interval. Thus, by redrawing the waveform segments corresponding to the target time interval, the desired second waveform graph is obtained.

[0104] The waveform drawing method provided by the embodiment of the present disclosure determines the time interval for re-superimposing the audio onto each waveform segment in the video based on the adjusted insertion time. Then, based on the overlap between the time interval of each waveform segment and the variable speed playback interval of the video, the waveform segment in the waveform graph where the waveform has undergone local changes is detected and identified, so that when re-editing the audio and video resources, only the waveform segment corresponding to the changed local waveform needs to be redrawn to update the entire waveform graph. Compared to the related art, which requires redrawing the entire waveform graph, the waveform drawing time is greatly shortened, thereby greatly reducing the waveform update time in the terminal device, and further greatly reducing the power consumption of the terminal device.

[0105] For example, Figure 3 As shown, the editing interface of the video before adjustment is displayed as follows Figure 3In the first waveform of the audio shown, the preset sampling period T is 1s, the insertion time before adjustment is 1s, and the first waveform before adjustment includes three waveform segments. The first waveform segment to the third waveform segment correspond to the time intervals of [1s, 5s], [5s, 9s], and [9s, 13s], respectively. Figure 3 ,like Figure 4 As shown, when the audio insertion time is changed to 3s, the adjusted first waveform includes three waveform segments, from the first waveform segment to the third waveform segment, and the time intervals corresponding to the three adjusted waveform segments are: [3s, 7s], [7s, 11s] and [11s, 15s]. At the same time, the variable speed playback intervals are obtained as: [0s, 1s], [13s, 14s], [14s, 15s] and [17s, 18s]. The speeds corresponding to the four variable speed playback intervals are 2 times the preset playback speed, 2 times the preset playback speed, 4 times the preset playback speed and non-uniform changes relative to the preset playback speed, such as speed being the cube of time. Among them, in the time interval, the time interval that overlaps with the variable speed playback interval is [11s, 15s], then the target time interval is [11s, 15s]. After redrawing the target time interval [11s, 15s], the following is obtained. Figure 4 The second waveform is shown.

[0106] The process of determining the target time interval of [11s, 15s] may be implemented by using the above steps S231 to S233 or by using the above step S234.

[0107] For example, when steps S231 to S233 are implemented, initially, the time interval sequence is [3s, 7s], [7s, 11s], and [11s, 15s], and the variable speed playback interval sequence is [0s, 1s], [13s, 14s], [14s, 15s], and [17s, 18s]. The first time interval of the initial time interval sequence is [3s, 7s], the first variable speed playback interval of the initial variable speed playback interval sequence is [0s, 1s], the first time interval [3s, 7s] and the first variable speed playback interval [0s, 1s] do not overlap, and the end time 1s of the first variable speed playback interval is less than the start time 3s of the first time interval, then the first variable speed playback interval [0s, 1s] is deleted from the variable speed playback interval sequence to update the variable speed playback interval sequence, and the time interval sequence is still updated to the original time interval sequence. The first playback interval of the updated variable speed playback interval sequence is [13s, 14s] and the first time interval of the updated time interval sequence is still [3s, 7s]. The end time 7s of the updated first time interval is less than the start time 13s of the updated first variable speed playback interval. Then the first time interval [3s, 7s] is deleted from the updated time interval sequence to update the time interval sequence.

[0108] The present disclosure also provides a Figure 5 The waveform diagram drawing device 500 shown includes: a receiving unit 501 , a first determining unit 502 , a second determining unit 503 , a drawing unit 504 and a segmenting unit 505 .

[0109] The receiving unit 501 is configured to receive an instruction for adjusting the insertion time of the audio when the first waveform of the audio is displayed in the video editing interface, wherein the audio includes at least one audio segment and the first waveform includes a waveform segment corresponding to each audio segment. Figure 2 The first determining unit 502 is configured to execute, in response to the instruction, determining the time interval corresponding to each waveform segment according to the adjusted insertion time, where the time interval refers to the time interval on the video time axis corresponding to the waveform segment after the audio is superimposed on the video according to the adjusted insertion time; for example, the first determining unit 502 is configured to execute Figure 2 The second determining unit 503 is configured to determine a time interval that overlaps with at least one variable speed playback interval of the video as a target time interval, where the variable speed playback interval refers to a time interval corresponding to a variable speed playback video segment on the video timeline that changes relative to a preset playback speed; for example, the second determining unit 503 is configured to execute Figure 2The drawing unit 504 is configured to execute, when at least one target time interval is determined, redraw the waveform segment corresponding to the target time interval according to the video playback speed corresponding to the target time interval to obtain a second waveform, and the second waveform is used to update the first waveform. For example, the drawing unit 504 is configured to execute Figure 2 Step S23 in .

[0110] In one possible embodiment, the drawing unit 504 is configured to specifically perform: determining multiple sampling time points included in the target time interval based on a preset sampling period; determining the video length corresponding to each preset sampling period based on the video playback speed corresponding to the sampling time point; and drawing the waveform segment corresponding to each target time interval based on the video length and audio volume corresponding to each sampling time period to obtain a second waveform.

[0111] In another possible implementation, the second determination unit 503 is configured to specifically perform: obtaining the latest variable speed playback interval sequence and the latest time interval sequence, the latest variable speed playback interval sequence refers to the variable speed playback interval sequence or the initial variable speed playback interval sequence updated according to the last comparison result, the latest variable speed playback interval sequence includes at least one variable speed playback interval arranged in chronological order, the latest time interval sequence refers to the time interval sequence or the initial time interval sequence updated according to the last comparison result, the latest time interval sequence includes at least one time interval arranged in chronological order; comparing the first variable speed playback interval in the latest variable speed playback interval sequence with the first time interval in the latest time interval sequence to obtain a comparison result; determining whether the first time interval in the latest time interval sequence is the target time interval based on the comparison result, and updating the latest variable speed playback interval sequence or the latest time interval sequence based on the comparison result.

[0112] In another possible implementation, before obtaining the latest variable speed playback interval sequence and the latest time interval sequence, the second determination unit 503 is configured to further specifically perform: arranging all variable speed playback intervals in chronological order to obtain an initial variable speed playback interval sequence, and arranging all time intervals in chronological order to obtain an initial time interval sequence.

[0113] In another possible implementation, the second determination unit 503 is configured to specifically execute: if the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval, and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval, and less than or equal to the end time of the first time interval, determine the first time interval as the target time interval.

[0114] In another possible implementation, if the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, then the first time interval is deleted from the latest time interval sequence to update the latest time interval sequence; if the end time of the first variable speed playback interval is less than the start time of the first time interval, then the first variable speed playback interval is deleted from the latest variable speed playback interval sequence to update the latest variable speed playback interval sequence; if the start time of the first variable speed playback interval is greater than the end time of the first time interval, then the first time interval is deleted from the latest time interval to update the latest time interval sequence.

[0115] In another possible implementation, the second determining unit 503 is configured to specifically perform: comparing each variable speed playback interval with each time interval in turn, and taking a time interval that overlaps with any variable speed playback interval as a target time interval.

[0116] In another possible implementation, the device further includes: a segmentation unit 505 configured to segment the first waveform into at least one waveform segment in chronological order according to the start time and end time of the audio in the first waveform to be superimposed on the video.

[0117] The present disclosure also provides an electronic device. Figure 1 The system includes a processor 101 and a memory 103 for storing instructions executable by the processor 101. The processor 101 is configured to execute the instructions to implement the waveform drawing method according to any of the above-described possible implementations. The methods can achieve the same technical effects, and are not described here in detail to avoid repetition.

[0118] The present disclosure also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an image processing device or electronic device, the image processing device or electronic device can perform the waveform graph drawing method described in any of the above possible implementations. To avoid repetition, the method is not further described here.

[0119] The present disclosure also provides a computer program product including a computer program or instructions, wherein the computer program or instructions are executed by a processor to perform the waveform graph drawing method according to any of the above possible implementations. The method can achieve the same technical effects, and to avoid repetition, it will not be described here.

[0120] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0121] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for drawing a waveform graph, characterized in that: include: When a first waveform of audio is displayed in a video editing interface, an instruction for adjusting the insertion time of the audio is received, wherein the audio includes at least one audio segment, and the first waveform includes a waveform segment corresponding to each of the audio segments; In response to the instruction, determining a time interval corresponding to each waveform segment according to the adjusted insertion time, wherein the time interval refers to a time interval on a video timeline corresponding to the waveform segment after the audio is superimposed on the video according to the adjusted insertion time; determining the time interval that overlaps with at least one variable-speed playback interval of the video as a target time interval, wherein the variable-speed playback interval refers to a time interval corresponding to a video segment whose playback speed varies relative to a preset playback speed; When at least one target time interval is determined, determining a plurality of sampling time points included in the target time interval according to a preset sampling period; Determining the video length corresponding to each of the preset sampling periods according to the video playback speed corresponding to the sampling time point; According to the video length and audio volume corresponding to each sampling time period, a waveform segment corresponding to each target time interval is drawn to obtain a second waveform graph, and the second waveform graph is used to update the first waveform graph.

2. The waveform drawing method according to claim 1, wherein: The step of determining the time interval that will overlap with at least one variable-speed playback interval of the video as the target time interval includes: Obtaining a latest variable-speed playback interval sequence and a latest time interval sequence, wherein the latest variable-speed playback interval sequence refers to a variable-speed playback interval sequence updated according to a previous comparison result or an initial variable-speed playback interval sequence, and the latest variable-speed playback interval sequence includes at least one of the variable-speed playback intervals arranged in chronological order; and the latest time interval sequence refers to a time interval sequence updated according to a previous comparison result or an initial time interval sequence, and the latest time interval sequence includes at least one of the time intervals arranged in chronological order; Comparing the first variable speed playback interval in the latest variable speed playback interval sequence with the first time interval in the latest time interval sequence to obtain a comparison result; Determine whether the first time interval in the latest time interval sequence is the target time interval based on the comparison result, and update the latest variable speed playback interval sequence or the latest time interval sequence based on the comparison result.

3. The waveform drawing method according to claim 2, wherein: Before obtaining the latest variable-speed playback interval sequence and the latest time interval sequence, the method further includes: All variable speed playback intervals are arranged in time sequence to obtain the initial variable speed playback interval sequence, and all time intervals are arranged in time sequence to obtain the initial time interval sequence.

4. The waveform diagram drawing method according to claim 2 or 3, characterized in that: Determining, based on the comparison result, whether the first time interval in the latest time interval sequence is a target time interval includes: If the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, the first time interval is determined as the target time interval.

5. The waveform drawing method according to claim 2 or 3, characterized in that: The updating of the latest variable-speed playback interval sequence or the latest time interval sequence according to the comparison result includes: If the start time of the first variable-speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, and / or if the end time of the first variable-speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, deleting the first time interval from the latest time interval sequence to update the latest time interval sequence; If the end time of the first variable speed playback interval is less than the start time of the first time interval, deleting the first variable speed playback interval from the latest variable speed playback interval sequence to update the latest variable speed playback interval sequence; If the start time of the first variable-speed playback interval is greater than the end time of the first time interval, the first time interval is deleted from the latest time interval to update the latest time interval sequence.

6. The waveform graph drawing method according to any one of claims 1 to 3, characterized in that: The time interval that overlaps with at least one variable-speed playback interval of the video is determined as a target time interval, and the method further includes: Each of the variable speed playback intervals is compared with each of the time intervals in turn, and a time interval that overlaps with any variable speed playback interval is used as the target time interval.

7. The waveform graph drawing method according to any one of claims 1 to 3, characterized in that: Before determining the time interval corresponding to each waveform segment according to the adjusted insertion time in response to the instruction, the method further includes: According to the start time and the end time of the audio in the first waveform being superimposed on the video, the first waveform is divided into at least one waveform segment in chronological order.

8. A waveform graph drawing device, characterized in that: The device comprises: a receiving unit configured to receive an instruction for adjusting the insertion time of audio when a first waveform of audio is displayed in a video editing interface, the audio including at least one audio segment, the first waveform including a waveform segment corresponding to each of the audio segments; a first determining unit configured to, in response to the instruction, determine a time interval corresponding to each of the waveform segments according to the adjusted insertion time, the time interval being a time interval on a video timeline corresponding to the waveform segment after the audio is superimposed on the video according to the adjusted insertion time; a second determining unit configured to determine, as a target time interval, the time interval that overlaps with at least one variable-speed playback interval of the video, wherein the variable-speed playback interval refers to a time interval corresponding to a video segment whose playback speed varies relative to a preset playback speed; The drawing unit is configured to, upon determining at least one target time interval, determine a plurality of sampling time points included in the target time interval according to a preset sampling period; determine a video length corresponding to each preset sampling period according to a video playback speed corresponding to the sampling time point; and draw a waveform segment corresponding to each target time interval according to the video length and audio volume corresponding to each sampling time period to obtain a second waveform, wherein the second waveform is used to update the first waveform.

9. The waveform graph drawing device according to claim 8, characterized in that: The second determining unit is configured to specifically perform: Obtaining a latest variable-speed playback interval sequence and a latest time interval sequence, wherein the latest variable-speed playback interval sequence refers to a variable-speed playback interval sequence updated according to a previous comparison result or an initial variable-speed playback interval sequence, and the latest variable-speed playback interval sequence includes at least one of the variable-speed playback intervals arranged in chronological order; and the latest time interval sequence refers to a time interval sequence updated according to a previous comparison result or an initial time interval sequence, and the latest time interval sequence includes at least one of the time intervals arranged in chronological order; Comparing the first variable speed playback interval in the latest variable speed playback interval sequence with the first time interval in the latest time interval sequence to obtain a comparison result; Determine whether the first time interval in the latest time interval sequence is the target time interval based on the comparison result, and update the latest variable speed playback interval sequence or the latest time interval sequence based on the comparison result.

10. The waveform graph drawing device according to claim 9, characterized in that: Before obtaining the latest variable-speed playback interval sequence and the latest time interval sequence, the second determining unit is configured to further execute: All variable speed playback intervals are arranged in time sequence to obtain the initial variable speed playback interval sequence, and all time intervals are arranged in time sequence to obtain the initial time interval sequence.

11. The waveform graph drawing device according to claim 9 or 10, characterized in that: The second determining unit is configured to specifically perform: If the start time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, and / or if the end time of the first variable speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, the first time interval is determined as the target time interval.

12. The waveform graph drawing device according to claim 9 or 10, characterized in that: The second determining unit is configured to specifically perform: If the start time of the first variable-speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, and / or if the end time of the first variable-speed playback interval is greater than or equal to the start time of the first time interval and less than or equal to the end time of the first time interval, deleting the first time interval from the latest time interval sequence to update the latest time interval sequence; If the end time of the first variable speed playback interval is less than the start time of the first time interval, deleting the first variable speed playback interval from the latest variable speed playback interval sequence to update the latest variable speed playback interval sequence; If the start time of the first variable-speed playback interval is greater than the end time of the first time interval, the first time interval is deleted from the latest time interval to update the latest time interval sequence.

13. The waveform graph drawing device according to any one of claims 8 to 10, characterized in that: The second determining unit is configured to specifically perform: Each of the variable speed playback intervals is compared with each of the time intervals in turn, and a time interval that overlaps with any variable speed playback interval is used as the target time interval.

14. The waveform graph drawing device according to any one of claims 8 to 10, characterized in that: The device further comprises: The segmentation unit is configured to segment the first waveform into at least one waveform segment in chronological order according to the start time and end time of the audio in the first waveform to be superimposed on the video.

15. An electronic device, characterized in that: include: A processor and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the executable instructions to implement the waveform drawing method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an image processing apparatus or an electronic device, the image processing apparatus or the electronic device is enabled to execute the waveform graph drawing method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the waveform graph drawing method according to any one of claims 1 to 7 is implemented.

Images