Methods, equipment, and media for generating subtitles for medical science videos

Abstract

This application relates to a method, device, and medium for generating subtitles for medical science videos. The method includes: acquiring the medical science video and its corresponding metadata, and retrieving multiple historical scripts matching the metadata from a historical video database; evaluating the quality of each historical script from multiple dimensions and generating a reference script based on the evaluation results; extracting audio segments containing human voices from the medical science video, and fusing the metadata with the reference scripts to obtain fused text; extracting medical terms from the fused text and constructing a hot word list based on the medical terms; generating initial subtitle text based on the audio segments and the hot word list, and optimizing the initial subtitle text using the reference scripts to obtain target subtitle text matching the medical science video. This solves the problem of difficulty in generating accurate medical science video subtitles due to the lack of high-quality reference scripts.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method, device and medium for generating subtitles for medical science videos. Background Technology

[0002] Medical science videos serve as an important medium for disseminating health knowledge, and the quality of subtitles directly impacts viewers' comprehension of professional medical content. Currently, subtitle generation technology primarily relies on general speech recognition systems to convert audio into text; however, this approach has several limitations in practical applications. Firstly, the medical field contains numerous and obscure medical terms, and general speech recognition systems are not accurate enough in recognizing these terms, resulting in subtitles that lack professionalism and may even disseminate incorrect information. Secondly, the lack of an effective script reference mechanism makes it difficult to guarantee the quality of subtitle generation.

[0003] Therefore, the lack of high-quality reference scripts makes it difficult to generate accurate medical science video subtitles with professional terminology, which has become a technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a method, device, and medium for generating subtitles for medical science videos, in order to solve the technical problem mentioned above that it is difficult to generate accurate subtitles for medical science videos due to the lack of high-quality reference scripts.

[0005] According to one aspect of the embodiments of this application, this application provides a method for generating subtitles for medical science videos, including: acquiring a medical science video and its corresponding metadata, and retrieving multiple historical scripts matching the metadata from a historical video database; performing quality assessment on each historical script from multiple dimensions, and generating a reference script based on the assessment results; extracting audio segments containing human voices from the medical science video, and fusing the metadata with the reference scripts to obtain fused text; extracting medical terms from the fused text, and constructing a hot word list based on the medical terms; generating initial subtitle text based on the audio segments and the hot word list, and optimizing the initial subtitle text using the reference scripts to obtain target subtitle text matching the medical science video.

[0006] Optionally, multiple historical scripts matching the metadata are retrieved from the historical video database, including: extracting doctor identifiers, title text, content description text, and disease domain from the metadata; performing weighted searches in the historical video database based on the doctor identifier, title text, content description text, and disease domain respectively, and obtaining search results sorted by relevance; extracting each historical video ranked before the first preset ranking in the search results from the historical video database, and obtaining each historical script corresponding to each historical video.

[0007] Optionally, the quality of each historical script is evaluated from multiple dimensions, including: obtaining the original script, final subtitles, script word count, and script generation time of the historical video to which each historical script belongs; calculating the completeness score of each historical script based on the script word count, the accuracy score of each historical script based on the text similarity between the original script and the final subtitles, the professionalism score of each historical script based on the number of medical terms appearing in the historical script, and the timeliness score of each historical script by performing attenuation calculation based on the difference between the script generation time and the current time; comprehensively calculating the completeness score, accuracy score, professionalism score, and timeliness score to obtain the comprehensive quality score of each historical script; filtering out historical scripts with a comprehensive quality score lower than a preset quality threshold, and extracting historical scripts with a comprehensive quality score ranking above the second preset rank from the remaining historical scripts to obtain the historical scripts after quality evaluation.

[0008] Optionally, a reference script is generated based on the evaluation results, including: inputting metadata and historical scripts after quality assessment from the evaluation results into a large language model; guiding the large language model to analyze the explanatory style features, language habit features, and terminology preference features in the input data through the first prompt word; and integrating video theme information from the metadata to generate a first candidate script; determining the target word count based on the duration of the medical science popularization video, and adjusting the word count of the first candidate script to the target word count to obtain a second candidate script; verifying the second candidate script, and determining the verified second candidate script as the reference script.

[0009] Optionally, extracting audio segments containing human voices from medical science videos includes: performing frame rate detection on the medical science video, and converting the medical science video to a standard frame rate if the frame rate is higher than a preset frame rate threshold, to obtain preprocessed video data; using a face detection algorithm to perform bidirectional segmentation detection on the video data to determine the effective time period containing faces; extracting initial audio segments from the video data based on the effective time period, and converting the initial audio segments according to a preset sampling rate and a preset encoding format to obtain audio segments containing human voices.

[0010] Optionally, after obtaining the audio segment, the method further includes: detecting the channel type of the audio segment; if the channel type of the audio segment is stereo, separating the left channel data and the right channel data to obtain channel separation data; performing human voice detection on the channel separation data to obtain human voice detection results; selecting a channel fusion strategy based on the human voice detection results; fusing the data of each channel according to the channel fusion strategy to generate processed audio data; uploading the processed audio data to cloud storage, and receiving the audio Uniform Resource Locator returned by the cloud storage for use by the speech recognition service.

[0011] Optionally, medical terms are extracted from the fused text, and a hot word list is constructed based on the medical terms. This includes: detecting the number of characters in the fused text and comparing the number of characters with a preset character count threshold; if the number of characters is less than the preset character count threshold, disease names are extracted from the fused text, and supplementary professional vocabulary data is retrieved from the medical knowledge base based on the disease names. The supplementary professional vocabulary data is added to the fused text to obtain the text to be processed; if the number of characters is equal to or greater than the preset character count threshold, the fused text is used as the text to be processed; the text to be processed is input into a large language model to guide the large language model to extract the first medical term and related terms through a second prompt word. The system collects weighted data corresponding to each first medical term; detects whether there are any long terms among the first medical terms that exceed a preset length threshold; if there are long terms, it splits the long terms into sub-terms that meet the length requirements, and identifies the sub-terms and the first medical terms other than the long terms as second medical terms; if there are no long terms, it uses the first medical terms as second medical terms; it sorts each second medical term in descending order according to the weighted data, and selects the third medical terms that rank before the third preset ranking; it calls the application programming interface of the speech recognition service to create a hot word list corresponding to the third medical term data.

[0012] Optionally, the initial subtitle text is optimized using a reference script to obtain target subtitle text that matches the medical science video. This includes: calculating the similarity between the initial subtitle text and the reference script, and comparing the similarity with a similarity threshold; if the similarity is greater than the similarity threshold, the initial subtitle text is aligned and corrected using a reference script-assisted optimization strategy; if the similarity is less than or equal to the similarity threshold, the initial subtitle text is corrected using an independent calibration optimization strategy to obtain intermediate subtitle text; the intermediate subtitle text is optimized using a large language model to obtain optimized intermediate subtitle text; the optimized intermediate subtitle text is time-aligned with the audio segment to generate subtitle data with timestamps; and the time axis of the subtitle data is offset and calibrated according to the start time of the effective time period to generate target subtitle text synchronized with the medical science video.

[0013] According to another aspect of the embodiments of this application, this application provides an electronic device, including a memory, a processor, a communication interface and a communication bus. The memory stores a computer program that can run on the processor. The memory and the processor communicate with each other through the communication bus and the communication interface. When the processor executes the computer program, it implements the steps of the above method.

[0014] According to another aspect of the embodiments of this application, this application also provides a computer-readable medium having processor-executable non-volatile program code that causes the processor to perform the above-described method.

[0015] Compared with related technologies, the technical solutions provided in this application have the following advantages: This application provides a method for generating subtitles for medical science videos, including: acquiring the medical science video and its corresponding metadata, and retrieving multiple historical scripts matching the metadata from a historical video database; performing quality assessments on each historical script from multiple dimensions, and generating a reference script based on the assessment results; extracting audio segments containing human voices from the medical science video, and fusing the metadata with the reference scripts to obtain fused text; extracting medical terms from the fused text, and constructing a hot word list based on the medical terms; generating initial subtitle text based on the audio segments and the hot word list, and optimizing the initial subtitle text using the reference scripts to obtain target subtitle text matching the medical science video. By retrieving multiple historical scripts matching the metadata from the historical video database, performing quality assessments on each historical script from multiple dimensions to generate a reference script, and fusing the metadata with the reference scripts to extract medical terms to construct a hot word list, the method enhances the ability to recognize professional terms during speech recognition, solving the problem of difficulty in generating accurate medical science video subtitles due to a lack of high-quality reference scripts. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the hardware environment for an optional subtitle generation method for medical science videos provided according to an embodiment of this application; Figure 2 This is a flowchart illustrating an optional method for generating subtitles for medical science videos according to an embodiment of this application; Figure 3 A flowchart illustrating another optional method for generating subtitles for medical science videos according to an embodiment of this application; Figure 4 This is a block diagram of an optional subtitle generation device for medical science videos according to an embodiment of this application; Figure 5 This is a schematic diagram of an optional electronic device structure provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module" and "part" may be used interchangeably.

[0021] Medical science videos serve as an important medium for disseminating health knowledge, and the quality of subtitles directly impacts viewers' comprehension of professional medical content. Currently, subtitle generation technology primarily relies on general speech recognition systems to convert audio into text; however, this approach has several limitations in practical applications. Firstly, the medical field contains numerous and obscure medical terms, and general speech recognition systems are not accurate enough in recognizing these terms, resulting in subtitles that lack professionalism and may even disseminate incorrect information. Secondly, the lack of an effective script reference mechanism makes it difficult to guarantee the quality of subtitle generation.

[0022] Therefore, the lack of high-quality reference scripts makes it difficult to generate accurate medical science video subtitles with professional terminology, which has become a technical problem that needs to be solved.

[0023] To address the problems mentioned in the background art, according to one aspect of the embodiments of this application, an embodiment of a method for generating subtitles for medical science popularization videos is provided.

[0024] Optionally, in the embodiments of this application, the above-described method for generating subtitles for medical science videos can be applied to, for example... Figure 1 The hardware environment shown consists of terminal 101 and server 103. Figure 1 As shown, server 103 is connected to terminal 101 via a network and can be used to provide services to the terminal or clients installed on the terminal. Database 105 can be set up on the server or independently of the server to provide data storage services for server 103. The network mentioned above includes, but is not limited to, wide area network, metropolitan area network or local area network. Terminal 101 includes, but is not limited to, PC, mobile phone, tablet computer, etc.

[0025] The subtitle generation method for medical science popularization videos in this application embodiment can be executed by server 103, or it can be jointly executed by server 103 and terminal 101, such as... Figure 2 As shown, it includes: Step 201: Obtain medical science popularization videos and corresponding metadata, and retrieve multiple historical scripts that match the metadata from the historical video database; Step 202: Evaluate the quality of each historical script from multiple dimensions and generate a reference script based on the evaluation results; Step 203: Extract audio segments containing human voices from medical science videos and fuse metadata with reference scripts to obtain fused text; Step 204: Extract medical terms from the fused text and construct a hot word list based on the medical terms; Step 205: Generate initial subtitle text based on audio clips and a hot word list, and optimize the initial subtitle text using a reference script to obtain target subtitle text that matches the medical science popularization video.

[0026] This application provides a method for generating subtitles for medical science videos, which can be applied to scenarios involving the automatic generation of video subtitles on online medical science platforms, hospital education systems, or short video platforms. When a user uploads a medical science video, professional and accurate subtitle text is automatically generated for that video.

[0027] The system acquires user-uploaded medical science videos and their corresponding metadata, including doctor identifiers, title text, content description text, and disease domain. Based on the metadata, it retrieves multiple historical scripts that match the metadata from the historical video database, which serve as the basic material for generating reference scripts.

[0028] The retrieved historical scripts were evaluated for quality based on four dimensions: completeness, accuracy, professionalism, and timeliness. Based on the evaluation results, the best historical scripts were selected to generate reference scripts.

[0029] It should be noted that if a user uploads a medical science video and provides a reliable external reference text, and the word count reaches the minimum limit (e.g., 50 words), the external reference text will be used directly as the reference script. The external reference text can be a script written in advance by a doctor or a subtitle script provided by the video production company.

[0030] Audio segments containing human voices are extracted from medical science videos, and invalid periods without dialogue, such as the opening and closing credits, are removed. Simultaneously, metadata is fused with a reference script to obtain the merged text.

[0031] Medical terms are extracted from the fused text, and a hot word list is constructed based on the extracted medical terms. The hot word list is used to enhance the recognition of professional terms during speech recognition.

[0032] Initial subtitle text is generated based on audio clips and a hot word list. Then, the initial subtitle text is optimized using a reference script. The correct terms and expressions in the reference script are compared and corrected with the initial subtitle text to finally obtain the target subtitle text that matches the content of the medical science popularization video.

[0033] This embodiment solves the problem of low coverage of manually written scripts by automatically generating reference scripts using historical video data, and improves the recognition accuracy of professional terms by constructing a hot word list of medical terms.

[0034] As an optional embodiment, multiple historical scripts matching the metadata are retrieved from the historical video database, including: extracting doctor identifiers, title text, content description text, and disease domain from the metadata; performing weighted searches in the historical video database based on the doctor identifier, title text, content description text, and disease domain respectively, and obtaining search results sorted by relevance; extracting each historical video ranked before the first preset ranking in the search results from the historical video database, and obtaining each historical script corresponding to each historical video.

[0035] The doctor identifier is used to locate historical video resources of the same presenting doctor, the title text and content description text are used for semantic similarity matching, and the disease domain is used to limit the scope of medical specialty to which the video belongs.

[0036] Based on the extracted doctor identifiers, title text, content description text, and disease domain information, a weighted search is performed in the historical video database. Specifically, precise matching is performed on the doctor identifiers, semantic similarity is calculated using deep learning text vector embedding for the title text and content description text, and precise matching is performed on the disease domain.

[0037] The search weights for doctor identification, title text, content description text, and disease domain information can be configured according to actual needs. For example, the doctor identification dimension has the highest weight to prioritize the retrieval of historical videos of the same doctor.

[0038] The search results from each dimension are merged to obtain search results sorted by relevance. Historical videos ranked before the first preset rank (e.g., 11) in the search results are extracted from the historical video database. For example, the 10 historical videos ranked before 11 are extracted, and the corresponding historical scripts for each video are obtained.

[0039] This embodiment uses multi-dimensional weighted retrieval to comprehensively mine historical video resources related to the current video from multiple perspectives such as doctors, title semantics, content semantics, and disease domains, thereby improving the relevance and coverage of the retrieval results.

[0040] As an optional implementation, the quality of each historical script is evaluated from multiple dimensions, including: obtaining the original script, final subtitles, script word count, and script generation time of the historical video to which each historical script belongs; calculating the completeness score of each historical script based on the script word count, the accuracy score of each historical script based on the text similarity between the original script and the final subtitles, the professionalism score of each historical script based on the number of medical terms appearing in the historical script, and the timeliness score of each historical script by attenuation calculation based on the difference between the script generation time and the current time; comprehensively calculating the completeness score, accuracy score, professionalism score, and timeliness score to obtain the comprehensive quality score of each historical script; filtering out historical scripts with a comprehensive quality score lower than a preset quality threshold, and extracting historical scripts with a comprehensive quality score ranking above the second preset rank from the remaining historical scripts to obtain the historical scripts after quality evaluation.

[0041] Retrieve the original script, final subtitles, script word count, and script generation time of each historical video to which the historical script belongs. The original script is the speech or other relevant materials prepared before the video recording, the final subtitles are the official subtitles generated after processing when the video is released, and the script generation time records the date the script was created.

[0042] The completeness score for each historical script is calculated based on its word count. Medical science video scripts typically have a reasonable word count range; too few words may indicate missing content or incomplete recording, while a reasonable word count indicates higher completeness. For example, the word count of a medical science video script is positively correlated with the video length. Using 4 to 5 words per second as a benchmark, the smaller the deviation of the script's word count from this benchmark, the higher the completeness score.

[0043] The accuracy score for each historical script is calculated based on the text similarity between the original script and the final subtitles. Methods such as edit distance (Levenshtein Distance) or cosine similarity can be used. A higher text similarity between the original script and the final subtitles indicates that the video content follows the original script well, and the script is more accurate. A lower similarity may indicate more improvisation or post-production modifications.

[0044] A professionalism score is calculated for each historical script based on the number of medical terms appearing in it. A higher number of medical terms indicates richer professional content, greater professionalism, and higher reference value.

[0045] The timeliness score of each historical script is calculated by performing a decay calculation based on the difference between the script's generation time and the current time. For example, by setting a time decay function, the closer the script's generation time is to the current time, the higher the timeliness score; the older the generation time, the more significant the score decay. Scripts generated a long time ago may have their reference value decrease over time, resulting in a correspondingly lower timeliness score.

[0046] The comprehensive quality score for each historical script is calculated by combining the completeness score, accuracy score, professionalism score, and timeliness score. The comprehensive calculation can use a weighted summation method, and the weights of each dimension can be configured according to actual needs.

[0047] Filter out historical scripts with an overall quality score below a preset quality threshold (e.g., 0.7). From the remaining historical scripts, extract those with an overall quality score ranking above the second preset rank to obtain the historical scripts after quality assessment. For example, if the preset quality threshold is 0.7, the second preset rank can be set to 6th place, meaning that after filtering out scripts with scores below 0.7, select the top 5 high-quality scripts with the highest overall scores.

[0048] First, scripts that fail to meet quality standards are eliminated by setting a preset quality threshold to ensure that all scripts participating in subsequent processing meet the basic quality requirements. Then, the highest-scoring scripts are selected from the qualified scripts for reference script generation. When the number of qualified scripts is insufficient, it is better to select fewer scripts than to introduce low-quality scripts, so as to avoid low-quality materials from misleading the subsequent generated results.

[0049] This embodiment conducts a multi-dimensional quantitative evaluation of historical scripts, selecting high-quality historical scripts from four perspectives: completeness, accuracy, professionalism, and timeliness. Low-quality scripts with incomplete content, low accuracy, poor professionalism, or insufficient timeliness are eliminated, providing a reliable material foundation for generating accurate and professional reference scripts in the future.

[0050] As an optional implementation, generating a reference script based on the evaluation results includes: inputting metadata and historical scripts after quality assessment from the evaluation results into a large language model; guiding the large language model to analyze the narration style features, language habit features, and terminology preference features in the input data through a first prompt word; and integrating video theme information from the metadata to generate a first candidate script; determining the target word count based on the duration of the medical science popularization video, and adjusting the word count of the first candidate script to the target word count to obtain a second candidate script; verifying the second candidate script, and determining the verified second candidate script as the reference script.

[0051] The metadata of medical science videos and the historical scripts after quality assessment are input into a Large Language Model (LLM). A Large Language Model is an artificial intelligence model trained on massive amounts of text data that can understand and generate natural language text.

[0052] The first prompt word guides the large language model to analyze the presentation style features, language habit features, and terminology preference features in the input data. Presentation style features represent the presenter's expression style, such as speaking speed, formality of word choice, and whether metaphors are used; language habit features represent the presenter's unique word choice preferences, such as common sentence structures and verbal tics; terminology preference features represent the professional vocabulary the presenter habitually uses when explaining specific diseases or treatment plans.

[0053] Furthermore, the large language model is instructed to integrate video topic information from metadata to generate a first candidate script that matches the current medical science popularization video.

[0054] The target word count is determined based on the length of the medical science video. For example, if the correlation between video length and word count is set to 4 to 5 words per second, then for a video length of 60 seconds, the target word count would be approximately 240 to 300 words.

[0055] Adjust the word count of the first candidate script to the target word count. If the word count of the first candidate script exceeds the target word count, shorten it; if the word count is less than the target word count, expand it to ensure that the generated script matches the video length.

[0056] The second candidate script is verified, including whether the script length meets the requirements, whether the use of technical terms is accurate, and whether the format is standardized. The second candidate script that passes the verification is determined as the reference script.

[0057] Once verified, the script can be automatically scored, and both the script and its score can be stored in a historical video database for retrieval in subsequent video subtitle generation tasks. By continuously accumulating newly generated scripts into the database, the historical script resources are constantly enriched.

[0058] This embodiment analyzes the narration style, language habits, and terminology preferences in high-quality historical scripts using a large language model, and integrates them with the current video theme to generate a personalized script. This ensures that the generated reference script retains the personal style of the narrator while also fitting the current video content.

[0059] As an optional embodiment, extracting audio segments containing human voices from medical science videos includes: performing frame rate detection on the medical science video, and converting the medical science video to a standard frame rate if the frame rate is higher than a preset frame rate threshold, to obtain preprocessed video data; using a face detection algorithm to perform bidirectional segmentation detection on the video data to determine the effective time period containing faces; extracting initial audio segments from the video data according to the effective time period, and converting the initial audio segments according to a preset sampling rate and a preset encoding format to obtain audio segments containing human voices.

[0060] Frame rate detection is performed on medical science videos. Frame rate refers to the number of image frames contained in a video per second, usually measured in frames per second (fps). When the frame rate of a medical science video exceeds a preset frame rate threshold (e.g., 30 frames per second), the video is converted to a standard frame rate. Frame rate conversion can be achieved through frame sampling or interpolation algorithms to ensure a uniform video frame rate.

[0061] A face detection algorithm is used to perform bidirectional segmentation detection on the preprocessed video data. The face detection algorithm is used to identify whether there are face regions in the video frame. Deep learning-based face detection models can be used, such as MTCNN (Multi-Task Cascaded Convolutional Networks) or SSD (SingleShot MultiBox Detector).

[0062] Bidirectional segmented detection involves detecting faces from both ends of the video. The beginning area is scanned from front to back, and the time of the first face detection is recorded as the start time. The end area is scanned from back to front, and the time of the last face detection is recorded as the end time. The effective time period containing faces is determined based on the start and end times. Medical science videos typically feature a doctor lecturing, with the beginning and end often consisting of animations, titles, or irrelevant footage. By using face detection to locate the doctor's on-screen moments, invalid content can be effectively eliminated.

[0063] The initial audio segment is extracted from the video data based on the valid time period. During extraction, the human voice audio corresponding to the valid time period is retained, while invalid audio at the beginning and end of the video is discarded. The initial audio segment is then converted according to a preset sampling rate and a preset encoding format. The sampling rate is the number of audio samples collected per second, and the preset sampling rate can be set to 16 kHz. The encoding format can be set to PCM (Pulse Code Modulation), which is a lossless audio format, facilitating subsequent speech recognition processing. The converted audio segment contains the human voice.

[0064] This embodiment ensures video format uniformity through frame rate conversion, provides stable input for face detection, and accurately locates the on-screen time of the main doctor through bidirectional face detection, effectively eliminating unspoken segments at the beginning and end of the video, avoiding the waste of recognition resources by invalid audio and the possible generation of erroneous text.

[0065] As an optional embodiment, after obtaining the audio segment, the method further includes: detecting the channel type of the audio segment; if the channel type of the audio segment is stereo, separating the left channel data and the right channel data to obtain channel separation data; performing human voice detection on the channel separation data to obtain human voice detection results; selecting a channel fusion strategy based on the human voice detection results; fusing the data of each channel according to the channel fusion strategy to generate processed audio data; uploading the processed audio data to cloud storage, and receiving the audio Uniform Resource Locator returned by the cloud storage for use by the speech recognition service.

[0066] Detects the channel type of an audio segment. Channel types include mono and stereo, with stereo containing two independent audio channels: left and right.

[0067] When the audio segment is in stereo, the left and right channel data are separated to obtain channel-separated data. During stereo recording, vocals may be unevenly distributed; for example, the voice of a presenting doctor might be recorded only in the left channel, while the right channel contains background music or ambient sound.

[0068] Human voice detection is performed on the channel-separated data to obtain the results. Voice detection employs VAD (Voice Activity Detection) technology, which analyzes parameters such as the spectral characteristics and energy changes of the audio signal to determine whether an audio segment contains human voices. VAD can distinguish human voices from non-human audio such as background music and environmental noise.

[0069] The channel fusion strategy is selected based on the voice detection results. There are three scenarios: when only one channel contains voice, the data from that channel is selected as the fusion result, discarding channels without voice to avoid background music or noise interference; when both channels contain voice, the left and right channel data are weighted and mixed, fusing the two channel data through a weighted average to avoid sound wave cancellation due to phase differences; when neither channel contains voice, the original channel data remains unchanged, which typically corresponds to background music or silent periods in a video clip.

[0070] The data from each channel is fused according to the selected channel fusion strategy to generate processed audio data. This processed audio data is then uploaded to a cloud storage service. Cloud storage is a distributed storage system deployed on a network to store large files and provide access links. The system receives an audio Uniform Resource Locator (URL) returned by the cloud storage. The URL serves as the access address for the audio segment, which is then used by the subsequent speech recognition service.

[0071] By uploading to cloud storage to obtain the URL, the audio processing module and the speech recognition service can be decoupled, facilitating asynchronous processing and resource reuse.

[0072] This embodiment determines the presence of human voices in each channel and adaptively selects a fusion strategy for three scenarios: one-sided human voice, two-sided human voice, and no human voice. This effectively solves the recognition problem caused by uneven channel distribution during stereo recording. Furthermore, it avoids phase cancellation through weighted mixing, ensuring the clarity of human voices.

[0073] As an optional embodiment, medical terms are extracted from the fused text, and a hot word list is constructed based on the medical terms. This includes: detecting the number of characters in the fused text and comparing the number of characters with a preset character count threshold; if the number of characters is less than the preset character count threshold, disease names are extracted from the fused text, and supplementary professional vocabulary data is retrieved from a medical knowledge base based on the disease names. The supplementary professional vocabulary data is added to the fused text to obtain the text to be processed; if the number of characters is equal to or greater than the preset character count threshold, the fused text is used as the text to be processed; the text to be processed is input into a large language model to guide the large language model to extract the first medical terms through a second prompt word. The system includes: weight data corresponding to each first medical term; detection of whether there are long terms among the first medical terms that exceed a preset length threshold; if long terms exist, splitting them into sub-terms that meet the length requirements, and identifying the sub-terms and the first medical terms other than the long terms as second medical terms; if no long terms exist, using the first medical terms as second medical terms; sorting each second medical term in descending order according to the weight data, and selecting the third medical terms that rank before the third preset ranking; and calling the application programming interface of the speech recognition service to create a hot word list corresponding to the third medical term data.

[0074] The system detects the number of words in the fused text and compares it to a preset word count threshold (e.g., 50 words). When the number of words in the fused text reaches or exceeds the preset threshold, it is directly used as the text to be processed. When the number of words in the fused text is less than the preset threshold, it indicates that the text content is insufficient and the professional terminology may be lacking. In this case, disease names are extracted from the fused text, and supplementary professional terminology data is retrieved from the medical knowledge base based on the disease names. The supplementary professional terminology is then added to the fused text to obtain the text to be processed. The medical knowledge base can be pre-built using a search engine and stores a large amount of medical literature, professional terminology, and disease-related knowledge.

[0075] The text to be processed is input into a large language model. A second prompt word guides the model to extract the first medical term and the weight data corresponding to each first medical term. The weight data reflects the importance of the medical term in the text; the higher the weight, the more critical the term.

[0076] This function detects whether there are any long terms in the primary medical terminology list that exceed a preset length threshold. The preset length threshold can be set to 10 characters. When a long term is found, it is split into sub-terms that meet the length requirement. For example, the jieba word segmentation tool can be used to segment the long term into several sub-terms of no more than 8 characters each. Through intelligent segmentation of long terms, excessively long professional terms are broken down into sub-terms that meet the length limit of the hot term list, avoiding the situation where terms are too long to be added to the hot term list. The sub-terms obtained from the splitting, along with the first medical term excluding the long term, are designated as the second medical term. If no long term exists, the first medical term is directly used as the second medical term.

[0077] After a long term is split into subterms, each subterm can inherit the weight of the original long term.

[0078] In another embodiment, to avoid weight inflation, the weight of the original long term can be evenly distributed among the sub-terms. For example, if the original long term has a weight of 0.9, it can be split into three sub-terms, each with a weight of 0.3. This approach is suitable for scenarios where the total number of hot words is limited and the importance of each term needs to be balanced.

[0079] The second medical terms are sorted in descending order based on weight data, with terms with higher weights appearing first. Third medical terms ranked before a predetermined third position (e.g., 291) are selected; for example, the top 290 terms by weight are chosen as candidate terms for constructing the hot term list.

[0080] By selecting core terms through weighted sorting to construct a hot word list, the number of terms in the hot word list is kept within a reasonable range, while covering the most important professional vocabulary, thereby effectively improving the recognition accuracy of medical professional terms in the speech recognition process.

[0081] The API (Application Programming Interface) of ASR (Automatic Speech Recognition) is invoked to create a hotword list corresponding to the third-party medical terminology data. The hotword list is a feature provided by the speech recognition service, used to add specified words to the recognition lexicon. During the recognition process, these words are prioritized for matching, thereby improving the accuracy of specialized terminology recognition.

[0082] This embodiment supplements professional vocabulary through multi-source information fusion, ensuring sufficient terminology sources even when the original fused text is insufficient in word count. It also extracts medical terms and assigns weights through a large language model, enabling a quantitative assessment of terminology importance.

[0083] As an optional embodiment, the initial subtitle text is optimized using a reference script to obtain target subtitle text that matches the medical science video. This includes: calculating the similarity between the initial subtitle text and the reference script, and comparing the similarity with a similarity threshold; if the similarity is greater than the similarity threshold, the initial subtitle text is aligned and corrected using a reference script-assisted optimization strategy; if the similarity is less than or equal to the similarity threshold, the initial subtitle text is corrected using an independent calibration optimization strategy to obtain intermediate subtitle text; a large language model is called to optimize the intermediate subtitle text to obtain optimized intermediate subtitle text; the optimized intermediate subtitle text is time-aligned with the audio segment to generate subtitle data with timestamps; and the time axis of the subtitle data is offset and calibrated according to the start time of the effective time period to generate target subtitle text synchronized with the medical science video.

[0084] Calculate the similarity between the initial subtitle text and the reference script, and compare the similarity to a similarity threshold. The similarity calculation can use an edit distance algorithm, which measures the similarity between two texts by calculating the minimum number of editing operations required to transform one string into another. A similarity threshold of 0.7 can be set to determine the degree of matching between the reference script and the initial subtitle text.

[0085] When the similarity score is greater than the similarity threshold, it indicates that the reference script matches the initial subtitle text well and has high reference value. In this case, a reference script-assisted optimization strategy is adopted, using the reference script to perform alignment and error correction on the initial subtitle text. For example, the correct terms in the reference script can replace the incorrect recognition results in the initial subtitle text, or the sentence breaks in the initial subtitle text can be adjusted according to the sentence structure of the reference script.

[0086] When the similarity is less than or equal to the similarity threshold, it indicates a low match between the reference script and the initial subtitle text. The reference scripts may originate from different sources or have significantly different content; direct use could be misleading. Therefore, an independent calibration and optimization strategy is adopted, independently correcting the initial subtitle text without relying on the reference script. For example, grammatical error correction and punctuation repair are performed based on a language model. The optimized text is then used to obtain the intermediate subtitle text.

[0087] The large language model is used to optimize the in-line subtitle text, including grammatical error correction, professional terminology verification, and punctuation standardization, resulting in optimized in-line subtitle text.

[0088] The optimized in-between subtitle text is time-aligned with the audio clip to generate subtitle data with timestamps. The time-alignment uses a forced alignment technique, matching the known text with the audio waveform word by word, generating a corresponding start and end time for each word to ensure precise time synchronization between the subtitles and the audio.

[0089] The timeline of the subtitle data is offset and calibrated based on the start time of the valid time period. Since the audio clip is cut from the valid time period, its timeline is offset relative to the original video. The timeline of the subtitle data is shifted as a whole, and the start time of the valid time period is added to generate target subtitle text that is completely synchronized with the timeline of the original medical science video.

[0090] This embodiment uses an adaptive selection optimization strategy based on similarity judgment. When the reference script and the initial subtitle have a high matching degree, the reference script is used for alignment and error correction. When the matching degree is low, independent calibration is used to avoid misleading, thus realizing the intelligent utilization of the reference script.

[0091] Figure 3 The flowchart of a method for generating subtitles for medical science videos provided in this application includes: receiving a medical video and its metadata, and determining whether an external script exists; if an external script exists, it is directly used as a reference; if no external script exists, an intelligent script generation process is initiated, generating a reference script through historical retrieval, quality assessment, and synthesis using a large language model; locating the effective time period of characters appearing in the video through face detection, performing channel fusion processing on the audio, extracting professional vocabulary to construct a hot word list, and generating initial subtitle text using speech recognition services enhanced by the hot word list; adaptively selecting an optimization strategy based on the similarity between the initial subtitle text and the reference script, optimizing the subtitle text, and outputting it after timeline alignment and post-processing to finally generate a subtitle URL.

[0092] This application provides a method for generating subtitles for medical science videos, including: acquiring the medical science video and its corresponding metadata, and retrieving multiple historical scripts matching the metadata from a historical video database; performing quality assessments on each historical script from multiple dimensions, and generating a reference script based on the assessment results; extracting audio segments containing human voices from the medical science video, and fusing the metadata with the reference scripts to obtain fused text; extracting medical terms from the fused text, and constructing a hot word list based on the medical terms; generating initial subtitle text based on the audio segments and the hot word list, and optimizing the initial subtitle text using the reference scripts to obtain target subtitle text matching the medical science video. By retrieving multiple historical scripts matching the metadata from the historical video database, performing quality assessments on each historical script from multiple dimensions to generate a reference script, and fusing the metadata with the reference scripts to extract medical terms to construct a hot word list, the method enhances the ability to recognize professional terms during speech recognition, solving the problem of difficulty in generating accurate medical science video subtitles due to a lack of high-quality reference scripts.

[0093] According to another aspect of the embodiments of this application, this application provides a subtitle generation device for medical science popularization videos, such as... Figure 4 As shown, it includes: The acquisition module 401 is used to acquire medical science popularization videos and corresponding metadata, and to retrieve multiple historical scripts that match the metadata from the historical video database; Evaluation module 402 is used to evaluate the quality of each historical script from multiple dimensions and generate reference scripts based on the evaluation results; The extraction module 403 is used to extract audio segments containing human voices from medical science videos and to fuse metadata with a reference script to obtain fused text. Module 404 is used to extract medical terms from the fused text and build a hot word list based on the medical terms; The generation module 405 is used to generate initial subtitle text based on audio segments and a hot word list, and to optimize the initial subtitle text using a reference script to obtain target subtitle text that matches the medical science popularization video.

[0094] It should be noted that the acquisition module 401 in this embodiment can be used to execute step 201 in this application embodiment, the evaluation module 402 in this embodiment can be used to execute step 202 in this application embodiment, the extraction module 403 in this embodiment can be used to execute step 203 in this application embodiment, the construction module 404 in this embodiment can be used to execute step 204 in this application embodiment, and the generation module 405 in this embodiment can be used to execute step 205 in this application embodiment.

[0095] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of a device, can operate in environments such as... Figure 1 The hardware environment shown.

[0096] According to another aspect of the embodiments of this application, this application provides an electronic device, such as... Figure 5 As shown, the system includes a memory 501, a processor 502, a communication interface 503, and a communication bus 504. The memory 501 stores a computer program that can run on the processor 502. The memory 501 and the processor 502 communicate through the communication interface 503 and the communication bus 504. When the processor 502 executes the computer program, it implements the steps of the above method.

[0097] The memory and processor in the aforementioned electronic devices communicate with each other via a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0098] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0099] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0100] According to another aspect of the embodiments of this application, a computer-readable medium having processor-executable non-volatile program code is also provided.

[0101] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0102] In specific implementation, the embodiments of this application can be referred to the above embodiments and have corresponding technical effects.

[0103] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0104] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0105] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0107] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0110] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0111] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for generating subtitles for medical science popularization videos, characterized in that, include: Acquire medical science videos and their corresponding metadata, and retrieve multiple historical scripts that match the metadata from the historical video database; The quality of each historical script is evaluated from multiple dimensions, and a reference script is generated based on the evaluation results. Audio segments containing human voices are extracted from the medical science popularization video, and the metadata is fused with the reference script to obtain fused text; Medical terms are extracted from the fused text, and a hot word list is constructed based on the medical terms; An initial subtitle text is generated based on the audio segment and the hot word list, and the initial subtitle text is optimized using the reference script to obtain a target subtitle text that matches the medical science video.

2. The method according to claim 1, characterized in that, The process of retrieving multiple historical scripts from the historical video database that match the metadata includes: Extract the doctor identifier, title text, content description text, and disease domain from the metadata; A weighted search is performed on the historical video database based on the doctor's identifier, the title text, the content description text, and the disease domain to obtain search results sorted by relevance. Extract each historical video ranked before the first preset ranking from the search results in the historical video database, and obtain each historical script corresponding to each historical video.

3. The method according to claim 2, characterized in that, The quality assessment of each historical script from multiple dimensions includes: Obtain the original script, final subtitles, script word count, and script generation time of each historical script to which it belongs; The completeness score of each historical script is calculated based on the number of words in the script, the accuracy score of each historical script is calculated based on the text similarity between the original script and the final subtitle, the professionalism score of each historical script is calculated based on the number of occurrences of the medical terms in the historical script, and the timeliness score of each historical script is obtained by attenuation calculation based on the difference between the script generation time and the current time. The comprehensive quality score of each historical script is obtained by comprehensively calculating the completeness score, the accuracy score, the professionalism score and the timeliness score. Filter out the historical scripts whose overall quality score is lower than a preset quality threshold, and extract the historical scripts whose overall quality score ranks before the second preset rank from the remaining historical scripts to obtain the historical scripts after quality assessment.

4. The method according to claim 1, characterized in that, The generation of reference scripts based on the evaluation results includes: The metadata and the historical scripts after quality assessment in the evaluation results are input into the large language model. The large language model is guided by the first prompt word to analyze the narration style features, language habit features and terminology preference features in the input data, and the video theme information in the metadata is integrated to generate the first candidate script. The target word count is determined based on the duration of the medical science video, and the word count of the first candidate script is adjusted to the target word count to obtain the second candidate script; The second candidate script is verified, and the second candidate script that passes the verification is determined as the reference script.

5. The method according to claim 1, characterized in that, The extraction of audio segments containing human voices from the medical science popularization video includes: The medical science popularization video is subjected to frame rate detection, and if the frame rate of the medical science popularization video is higher than a preset frame rate threshold, the medical science popularization video is converted into a standard frame rate to obtain preprocessed video data; A face detection algorithm is used to perform bidirectional segmentation detection on the video data to determine the effective time period containing a face; An initial audio segment is extracted from the video data according to the effective time period, and the initial audio segment is converted according to a preset sampling rate and a preset encoding format to obtain the audio segment containing human voice.

6. The method according to claim 5, characterized in that, After obtaining the audio segment, the method further includes: Detect the channel type of the audio segment; When the channel type of the audio segment is stereo, the left channel data and the right channel data are separated to obtain channel separation data; Human voice detection is performed on the vocal tract separation data to obtain human voice detection results; Select a vocal tract fusion strategy based on the human voice detection results; The data from each channel are fused according to the aforementioned channel fusion strategy to generate processed audio data. The processed audio data is uploaded to cloud storage, and the audio Uniform Resource Locator returned by the cloud storage is received for use by the speech recognition service.

7. The method according to claim 1, characterized in that, The step of extracting medical terms from the fused text and constructing a hot word list based on the medical terms includes: The number of characters in the fused text is detected, and the number of characters is compared with a preset character count threshold; If the number of characters is less than the preset character threshold, then the disease name is extracted from the fused text, and supplementary professional vocabulary data is retrieved from the medical knowledge base according to the disease name. The supplementary professional vocabulary data is added to the fused text to obtain the text to be processed. If the number of characters is equal to or greater than the preset character threshold, then the fused text is used as the text to be processed. The text to be processed is input into a large language model, so that the large language model can be guided by a second prompt word to extract the first medical term and the weight data corresponding to each of the first medical terms; Detect whether there are any long terms in the first medical terminology that exceed a preset length threshold; If the long term exists, the long term is split into sub-terms that meet the length requirement, and the sub-terms and the first medical term other than the long term are identified as the second medical term. If the long term does not exist, the first medical term is used as the second medical term. The second medical terms are sorted in descending order according to the weight data, and the third medical terms that rank before the third preset ranking are selected. The application programming interface of the speech recognition service is invoked to create the hot word list corresponding to the third medical terminology data.

8. The method according to claim 5, characterized in that, The step of optimizing the initial subtitle text using the reference script to obtain target subtitle text that matches the medical science video includes: Calculate the similarity between the initial subtitle text and the reference script, and compare the similarity with a similarity threshold; If the similarity is greater than the similarity threshold, a reference script-assisted optimization strategy is used to perform alignment and error correction on the initial subtitle text. If the similarity is less than or equal to the similarity threshold, an independent calibration optimization strategy is used to correct the initial subtitle text to obtain the intermediate subtitle text. The large language model is called to optimize the intermediate subtitle text, resulting in optimized intermediate subtitle text; The optimized intermediate subtitle text is aligned with the audio segment along the timeline to generate subtitle data with timestamps. The timeline of the subtitle data is offset and calibrated according to the start time of the effective time period to generate the target subtitle text synchronized with the medical science popularization video.

9. An electronic device comprising a memory, a processor, a communication interface, and a communication bus, wherein the memory stores a computer program executable on the processor, and the memory and the processor communicate via the communication bus and the communication interface, characterized in that... When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the method of any one of claims 1 to 8.

Images