Subtitle generation method and device, electronic equipment and storage medium

Abstract

The invention discloses a subtitle generation method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a target audio; respectively processing the target audio according to at least one predetermined subtitle generation mode to obtain at least one piece of candidate subtitle information; and comparing the candidate subtitle information, and determining the candidate subtitle information with the maximum same quantity in the candidate subtitle information as target subtitle information corresponding to the target audio. By adopting the technical scheme of the invention, when the target subtitle information is determined, the determination efficiency of the target subtitle information can be improved, so that the generation of the subtitle information is accelerated, the subtitles corresponding to the audio with longer audio time can be generated, and the use experience of a user is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software technology, and in particular to a subtitle generation method, device, electronic equipment and storage medium. Background Art

[0002] Videos have become a common way of presenting information today due to their length and high degree of freedom in style.

[0003] To ensure the accuracy of video information transmission, video creators often add subtitles to videos to help users who watch the videos understand the video content more easily. However, as the length of the video increases, the accuracy of subtitle generation by existing methods gradually decreases, and it is very easy for the subtitle audio time to be out of sync, affecting the user experience of video viewers. Summary of the invention

[0004] The present invention provides a subtitle generation method, device, electronic device and storage medium to solve the problem of high error rate when generating subtitles for a long video.

[0005] According to one aspect of the present invention, a subtitle generation method is provided, the method comprising:

[0006] Determine the target audio;

[0007] According to at least one predetermined subtitle generation method, the target audio is processed separately to obtain at least one candidate subtitle information;

[0008] The candidate subtitle information is compared to determine the candidate subtitle information with the largest number of identical numbers among the candidate subtitle information as the target subtitle information corresponding to the target audio.

[0009] According to another aspect of the present invention, there is provided a subtitle generating device, the device comprising:

[0010] A target audio module, used to determine the target audio;

[0011] A candidate subtitle generation module, used to process the target audio separately according to at least one predetermined subtitle generation method to obtain at least one candidate subtitle information;

[0012] The target subtitle selection module is used to compare the candidate subtitle information and determine the candidate subtitle information with the largest number of identical numbers among the candidate subtitle information as the target subtitle information corresponding to the target audio.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the subtitle generation method of any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores computer instructions. The computer instructions are used to enable a processor to implement the subtitle generation method of any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention determines the target audio, and processes the target audio separately according to at least one predetermined subtitle generation method to obtain at least one candidate subtitle information, compares each candidate subtitle information, and determines the candidate subtitle information with the largest number of the same among the candidate subtitle information as the target subtitle information corresponding to the target audio, so that when determining the target subtitle information, the determination efficiency of the target subtitle information can be improved, thereby speeding up the generation of subtitle information. The combined use of the above steps can generate subtitles corresponding to audio with a longer audio time, thereby improving the user experience.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flowchart of a subtitle generation method provided according to Embodiment 1 of the present invention;

[0022] Figure 2 is a flowchart of another subtitle generation method provided according to Embodiment 2 of the present invention;

[0023] Figure 3 is a structural schematic diagram of a subtitle generating device provided according to Embodiment 3 of the present invention;

[0024] Figure 4 It is a schematic diagram of the structure of an electronic device for implementing the subtitle generation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1 A flowchart of a subtitle generation method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of generating subtitles for a long audio. The method can be executed by a subtitle generation device. The subtitle generation device can be implemented in the form of hardware and / or software. The subtitle generation device can be configured in an electronic device with data processing capabilities. Figure 1 As shown, the method includes:

[0029] S110: Determine target audio.

[0030] The target audio may be audio for which subtitles need to be generated.

[0031] The target audio may be extracted from a video file, or may be an audio file only. For example, the corresponding audio file may be extracted from an MP4 file, or an MP3 file may be directly used as the target audio. The target audio format of the present application may be an audio format such as MP3, WAV, AAC, and WMA.

[0032] In an optional solution, determining the target audio may include steps A1-A4:

[0033] Step A1: determine the audio to be processed.

[0034] Step A2: Determine the audio duration of the audio to be processed.

[0035] Step A3: If the audio duration of the audio to be processed is less than the first preset audio duration, the audio to be processed is used as the target audio.

[0036] Step A4: If the audio duration of the audio to be processed is greater than or equal to the first preset audio duration, the audio to be processed is segmented to obtain at least one target audio.

[0037] The audio to be processed may be audio that is waiting for caption generation but the length of the audio is unknown.

[0038] As the audio length increases, the amount of audio data increases, which makes it more likely that subtitle errors will occur when generating the corresponding subtitles. Therefore, to avoid this problem, the audio length of the audio to be processed will be determined before processing the audio to be processed, so as to use the audio length to determine whether the audio to be processed is prone to errors in subtitle generation due to the large amount of data.

[0039] When the audio duration of the audio to be processed is less than the first preset audio duration, it is determined that the data volume of the audio to be processed is not too large, and thus the audio to be processed is directly processed as the target audio.

[0040] When the audio duration of the audio to be processed is greater than or equal to the first preset audio duration, it will be determined that the data volume of the audio to be processed is too large. At this time, it will be determined that directly generating corresponding subtitles based on the audio to be processed will result in large errors. At this time, the audio to be processed will be segmented to shorten the audio duration of the audio to be processed and reduce the data volume of the audio to be processed, thereby obtaining a target audio whose audio duration is less than the first preset audio duration.

[0041] In an optional solution, segmenting the audio to be processed to obtain at least one target audio may include steps B1-B5:

[0042] Step B1, determining the audio amplitude spectrum corresponding to the audio to be processed.

[0043] Step B2: determining the audio segmentation point of the audio to be processed according to the predetermined video segmentation duration, where the audio segmentation point is the time point at which the video is segmented.

[0044] Step B3: Determine the audio amplitude corresponding to each audio segmentation point based on the audio amplitude spectrum.

[0045] Step B4: If the audio amplitude corresponding to the audio segmentation point is greater than the preset amplitude, the audio segmentation point is moved forward or backward according to the time axis sequence of the video to be processed until the audio amplitude is less than or equal to the preset amplitude.

[0046] Step B5: Split the audio to be processed according to each audio split point to obtain at least one target audio.

[0047] When splitting the audio to be processed, if the audio split point is exactly at the midpoint of a sentence, it may split a complete sentence into two parts, thus changing the semantic relationship of the sentence.

[0048] For example, if the audio split point is exactly at the position of the character "ge" in the sentence "This is an apple", and it is directly split at this time, the sentence will be split into two sentences: "This is yi" and "ge apple". If subtitle generation is performed separately at this time, errors may occur.

[0049] To avoid the above problems, the audio amplitude spectrum corresponding to the audio to be processed will be extracted, and before splitting the audio to be processed, the audio split points of the audio to be processed will be determined according to the pre-determined video split duration.

[0050] After obtaining each audio split point and the audio amplitude spectrum, the audio amplitude corresponding to each audio split point can be determined.

[0051] When the audio amplitude corresponding to the audio split point is greater than the preset amplitude, it is determined that the audio split point is exactly in a sentence. At this time, the audio split point will be moved forward or backward until the audio amplitude is less than or equal to the preset amplitude, so as to avoid splitting a complete sentence.

[0052] In an alternative solution, moving the audio split point forward or backward according to the time axis order of the video to be processed may include steps C1-C2:

[0053] Step C1: Determine the target amplitude point closest to the audio split point, the amplitude of the target amplitude point is less than or equal to the preset amplitude, and the target amplitude point is an amplitude point in the audio amplitude spectrum.

[0054] Step C2: Adjust the audio split point to the target amplitude point.

[0055] When moving the audio split point forward or backward according to the time axis order of the video to be processed, in order to improve the moving efficiency of the audio split point, the target amplitude point closest to the audio split point with an amplitude less than or equal to the preset amplitude will be found, and the audio split point will be adjusted to the target amplitude point.

[0056] S120: Process the target audio separately according to at least one pre-determined subtitle generation method to obtain at least one candidate subtitle information.

[0057] The subtitle generation method can be existing subtitle generation software and neural network models with subtitle generation functions, etc.

[0058] Since different subtitle generation methods are good at different audio types when generating subtitles, multiple subtitle generation methods can be determined in advance, and after obtaining the target audio, the target audio will be processed separately by at least one pre-selected subtitle generation method to obtain at least one candidate subtitle information.

[0059] For example, when the target audio is "This is an apple", the subtitle generation methods include method A, method B and method C.

[0060] At this time, the target audio can be processed by subtitle generation method A to obtain candidate subtitle information. The target audio can be processed by subtitle generation method B to obtain candidate subtitle information. The target audio can be processed by subtitle generation method C to obtain candidate subtitle information.

[0061] S130: Compare the candidate subtitle information and determine the candidate subtitle information with the largest number of identical information among the candidate subtitle information as the target subtitle information corresponding to the target audio.

[0062] The target subtitle information may be the subtitle information that is finally determined and matches the target video.

[0063] After obtaining each candidate subtitle information, since different candidate subtitle information is generated by different subtitle generation methods, the target subtitle information can be determined by using the accuracy of different subtitle generation methods. The reason is that after the target audio has been fixed, the correct subtitle information corresponding to the target audio has also been fixed. Therefore, when multiple candidate subtitle information is the same, the possibility that the candidate subtitle information is the correct subtitle information corresponding to the target audio is the greatest. Therefore, the candidate subtitle information with the largest number of the same number will be selected as the target subtitle information corresponding to the target audio.

[0064] For example, when the number of candidate subtitle information finally obtained is 3, if two of the candidate subtitle information are the same and the other one is different from the two candidate subtitle information, any one of the two identical candidate subtitle information will be used as the target subtitle information.

[0065] By comparing each candidate subtitle information, the candidate subtitle information with the largest number of identical information among the candidate subtitle information is determined as the target subtitle information corresponding to the target audio, so that when determining the target subtitle information, the efficiency of determining the target subtitle information can be improved, thereby speeding up the generation of subtitle information.

[0066] According to the technical solution of the embodiment of the present invention, by determining the target audio, and processing the target audio separately according to at least one predetermined subtitle generation method, at least one candidate subtitle information is obtained, and each candidate subtitle information is compared, and the candidate subtitle information with the largest number of the same among the candidate subtitle information is determined as the target subtitle information corresponding to the target audio, so that when determining the target subtitle information, the efficiency of determining the target subtitle information can be improved, thereby speeding up the generation of subtitle information. By using the above steps in combination, subtitles corresponding to audio with a longer audio time can be generated, thereby improving the user experience.

[0067] Embodiment 2

[0068] Figure 2 A flowchart of another subtitle generation method is provided for an embodiment of the present invention. Based on the above embodiment, this embodiment further optimizes the process of comparing each candidate subtitle information and determining the candidate subtitle information with the largest number of identical numbers among the candidate subtitle information as the target subtitle information corresponding to the target audio. This embodiment can be combined with each optional solution in one or more of the above embodiments. Figure 2 As shown, the subtitle generation method of this embodiment may include the following steps:

[0069] S210: Determine target audio.

[0070] S220: Process the target audio separately according to at least one predetermined subtitle generation method to obtain at least one candidate subtitle information.

[0071] S230: Compare the candidate subtitle information and determine the candidate subtitle information with the largest number of identical information as the target subtitle information corresponding to the target audio.

[0072] S240: If the candidate subtitle information is different from each other, select one of the candidate subtitle information as the target subtitle information corresponding to the target audio.

[0073] After determining multiple candidate subtitle information, it may happen that each candidate subtitle information is different. At this time, it is determined that different subtitle generation methods may have certain problems when generating candidate subtitle information. At this time, no matter which candidate subtitle information is selected, there will be certain errors. You can choose any candidate subtitle information as the target subtitle information corresponding to the target audio, but the target audio will be marked to inform the user that the target subtitle information corresponding to this target audio may be inaccurate.

[0074] When the candidate subtitle information is different from each other, one of the candidate subtitle information is selected as the target subtitle information corresponding to the target audio, so that when it is impossible to decide to select the final candidate subtitle information, the target subtitle information can be quickly determined.

[0075] In an optional solution, after comparing the candidate subtitle information and determining the candidate subtitle information with the largest number of identical numbers among the candidate subtitle information as the target subtitle information corresponding to the target audio, steps D1-D3 are also included:

[0076] Step D1: If the candidate subtitle information is different from each other, then determine the similarity between the candidate subtitle information.

[0077] Step D2: Determine the comprehensive similarity corresponding to each candidate subtitle information according to the similarity between each candidate subtitle information.

[0078] Step D3: According to the comprehensive similarities corresponding to the candidate subtitle information, the candidate subtitle information with the largest comprehensive similarity is selected as the target subtitle information.

[0079] When the candidate subtitle information is different from each other, in order to improve the accuracy of determining the final target subtitle information, the similarities between the candidate subtitle information can be determined first, and then the comprehensive accuracy corresponding to the candidate subtitle information can be determined with the help of the similarity, and finally the candidate subtitle information with the highest comprehensive similarity can be selected and used as the target subtitle information.

[0080] For example, among candidate subtitle information A, candidate subtitle information B, and candidate subtitle information C, the similarity between candidate subtitle information A and candidate subtitle information B is 50%, the similarity between candidate subtitle information A and candidate subtitle information C is 60%, and the similarity between candidate subtitle information B and candidate subtitle information C is 30%. At this time, it can be determined that A has the highest similarity with other candidate subtitle information, so candidate subtitle information A can be used as target subtitle information.

[0081] The comprehensive similarity corresponding to each candidate subtitle information is determined by similarity, and the candidate subtitle information with the largest comprehensive similarity is selected as the target subtitle information, so that the accuracy of the target subtitle information finally determined is higher.

[0082] Optionally, according to the similarities between the respective candidate subtitle information, a comprehensive similarity corresponding to each candidate subtitle information is determined, including:

[0083] The similarities between each candidate subtitle information and other candidate subtitle information are summed to obtain the comprehensive similarity of each candidate subtitle information.

[0084] The comprehensive similarity can be determined by summing up the similarities between each candidate subtitle information and other candidate subtitle information, or by weighted summation or other methods, which is not limited in this application.

[0085] According to the technical solution of the embodiment of the present invention, when the candidate subtitle information is different from each other, one of the candidate subtitle information is selected as the target subtitle information corresponding to the target audio, so that when it is impossible to decide on the final candidate subtitle information, the target subtitle information can be quickly determined.

[0086] Embodiment 3

[0087] Figure 3 The present invention provides a block diagram of a subtitle generating device. The present invention is applicable to the case of generating subtitles for a long audio. The subtitle generating device can be implemented in the form of hardware and / or software. The subtitle generating device can be configured in an electronic device with data processing capabilities. Figure 3 As shown, the subtitle generation device of this embodiment may include: a target audio module 310, a candidate subtitle generation module 320 and a target subtitle selection module 330. Among them:

[0088] A target audio module 310, configured to determine a target audio;

[0089] A candidate subtitle generation module 320 is used to process the target audio separately according to at least one predetermined subtitle generation method to obtain at least one candidate subtitle information;

[0090] The target subtitle selection module 330 is used to compare the candidate subtitle information and determine the candidate subtitle information with the largest number of identical numbers among the candidate subtitle information as the target subtitle information corresponding to the target audio.

[0091] Based on the above embodiment, optionally, the target audio module 310 includes:

[0092] Determine the audio to be processed;

[0093] Determine the audio duration of the audio to be processed;

[0094] If the audio duration of the audio to be processed is less than the first preset audio duration, the audio to be processed is used as the target audio;

[0095] If the audio duration of the audio to be processed is greater than or equal to the first preset audio duration, segmenting the audio to be processed to obtain at least one target audio;

[0096] Accordingly, after the target subtitle selection module 330, the following steps are also included:

[0097] The target subtitle information is concatenated to obtain the subtitle information corresponding to the audio to be processed.

[0098] Based on the above embodiment, optionally, the audio to be processed is segmented to obtain at least one target audio, including:

[0099] Determine the audio amplitude spectrum corresponding to the audio to be processed;

[0100] Determine the audio segmentation point of the audio to be processed according to the predetermined video segmentation duration, where the audio segmentation point is the time point at which the video is segmented;

[0101] Based on the audio amplitude spectrum, determine the audio amplitude corresponding to each audio segmentation point;

[0102] If the audio amplitude corresponding to the audio segmentation point is greater than the preset amplitude, the audio segmentation point is moved forward or backward according to the timeline sequence of the video to be processed until the audio amplitude is less than or equal to the preset amplitude;

[0103] The audio to be processed is segmented according to each audio segmentation point to obtain at least one target audio.

[0104] Based on the above embodiment, optionally, the audio segmentation point is moved forward or backward according to the time axis sequence of the video to be processed, including:

[0105] Determine a target amplitude point closest to the audio segmentation point, the amplitude of the target amplitude point is less than or equal to the preset amplitude, and the target amplitude point is an amplitude point in the audio amplitude spectrum;

[0106] Adjust the audio split point to the target amplitude point.

[0107] Based on the above embodiment, optionally, after the target subtitle selection module 330, the following further includes:

[0108] If the candidate subtitle information is different from each other, then determining the similarity between the candidate subtitle information;

[0109] Determine the comprehensive similarity corresponding to each candidate subtitle information according to the similarity between each candidate subtitle information;

[0110] According to the comprehensive similarities corresponding to the respective candidate subtitle information, the candidate subtitle information with the greatest comprehensive similarity is selected as the target subtitle information.

[0111] On the basis of the above embodiment, optionally, according to the similarity between each candidate subtitle information, the comprehensive similarity corresponding to each candidate subtitle information is determined, including:

[0112] The similarities between each candidate subtitle information and other candidate subtitle information are summed to obtain the comprehensive similarity of each candidate subtitle information.

[0113] Based on the above embodiment, optionally, after the target subtitle selection module 330, the following further includes:

[0114] If the candidate subtitle information is different from each other, one of the candidate subtitle information is selected as the target subtitle information corresponding to the target audio.

[0115] The subtitle generating device provided in the embodiment of the present invention can execute the subtitle generating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Embodiment 4

[0117] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0118] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0119] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0120] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a subtitle generation method.

[0121] In some embodiments, the subtitle generation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the subtitle generation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the subtitle generation method in any other appropriate manner (e.g., by means of firmware).

[0122] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0124] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0127] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0128] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0129] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A subtitle generation method, characterized in that: include: Determine the target audio; According to at least one predetermined subtitle generation method, the target audio is processed separately to obtain at least one candidate subtitle information; The candidate subtitle information is compared to determine the candidate subtitle information with the largest number of identical numbers among the candidate subtitle information as the target subtitle information corresponding to the target audio.

2. The method according to claim 1, characterized in that Determine the target audio, including: Determine the audio to be processed; Determine the audio duration of the audio to be processed; If the audio duration of the audio to be processed is less than the first preset audio duration, taking the audio to be processed as the target audio; If the audio duration of the audio to be processed is greater than or equal to the first preset audio duration, segmenting the audio to be processed to obtain at least one target audio; Accordingly, after comparing the candidate subtitle information and determining the candidate subtitle information with the largest number of identical numbers among the candidate subtitle information as the target subtitle information corresponding to the target audio, the method further includes: The target subtitle information is concatenated to obtain the subtitle information corresponding to the audio to be processed.

3. The method according to claim 2, characterized in that Segmenting the audio to be processed to obtain at least one target audio, including: Determine an audio amplitude spectrum corresponding to the audio to be processed; Determining an audio segmentation point of the audio to be processed according to a predetermined video segmentation duration, wherein the audio segmentation point is a time point at which the video is segmented; Based on the audio amplitude spectrum, determine the audio amplitude corresponding to each audio segmentation point; If the audio amplitude corresponding to the audio segmentation point is greater than the preset amplitude, the audio segmentation point is moved forward or backward according to the time axis sequence of the video to be processed until the audio amplitude is less than or equal to the preset amplitude; The audio to be processed is segmented according to each of the audio segmentation points to obtain at least one target audio.

4. The method according to claim 3, characterized in that The audio segmentation point is moved forward or backward according to the time axis sequence of the video to be processed, including: Determine a target amplitude point closest to the audio segmentation point, wherein the amplitude of the target amplitude point is less than or equal to a preset amplitude, and the target amplitude point is an amplitude point in the audio amplitude spectrum; The audio segmentation point is adjusted to the target amplitude point.

5. The method according to claim 1, characterized in that: After comparing each candidate subtitle information and determining the candidate subtitle information with the largest number of identical numbers among each candidate subtitle information as the target subtitle information corresponding to the target audio, the method further includes: If the candidate subtitle information is different from each other, then determining the similarity between the candidate subtitle information; Determine the comprehensive similarity corresponding to each candidate subtitle information according to the similarity between each candidate subtitle information; According to the comprehensive similarities corresponding to the respective candidate subtitle information, the candidate subtitle information with the greatest comprehensive similarity is selected as the target subtitle information.

6. The method according to claim 5, characterized in that According to the similarities between the various candidate subtitle information, the comprehensive similarity corresponding to each candidate subtitle information is determined, including: The similarities between each candidate subtitle information and other candidate subtitle information are summed to obtain the comprehensive similarity of each candidate subtitle information.

7. The method according to claim 1, characterized in that After comparing each candidate subtitle information and determining the candidate subtitle information with the largest number of identical numbers among each candidate subtitle information as the target subtitle information corresponding to the target audio, the method further includes: If the candidate subtitle information is different from each other, one of the candidate subtitle information is selected as the target subtitle information corresponding to the target audio.

8. A subtitle generating device, characterized in that: include: A target audio module, used to determine the target audio; A candidate subtitle generation module, configured to process the target audio separately according to at least one predetermined subtitle generation method to obtain at least one candidate subtitle information; The target subtitle selection module is used to compare the candidate subtitle information and determine the candidate subtitle information with the largest number of identical information among the candidate subtitle information as the target subtitle information corresponding to the target audio.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the subtitle generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the subtitle generation method according to any one of claims 1 to 7 when executed.

Images