Medical popular science video production method and system based on deep learning

Abstract

The application relates to the technical field of medical video generation, and discloses a medical popular science video production method and system based on deep learning, which comprises the following steps: acquiring a theme text, and constructing a hierarchical evidence chain from a knowledge graph; extracting entity relations to input a large model to generate a script containing explanation words, a shooting script and a voiceover; decomposing the shooting script into images and video instructions to input a PR-GAN generator to generate animation frames; inputting the animation frames into a discriminator to obtain an adversarial loss; inputting the animation frames and target frames into a VGG network to extract features, calculate Euclidean distances and obtain a perception loss; extracting real and generated features in an intermediate layer of the discriminator to calculate L1 distances and obtain a feature matching loss; weighting the three losses to update the generator and the discriminator; using the updated generator to generate animation frames, synthesizing a preliminary video with the explanation voiceover, and outputting a final video after fact verification. The application generates an accurate and real popular science video and realizes full-automatic production.

Description

Technical Field

[0001] This invention relates to the field of medical video generation technology, and in particular to a method and system for producing medical science videos based on deep learning. Background Technology

[0002] In recent years, breakthroughs in generative artificial intelligence (AI) technology have provided new technological pathways for video content creation. In the field of medical science popularization, research has explored generating science scripts using large language models and visual materials using diffusion models and text-to-image models. Meanwhile, related patent applications have been filed concerning AIGC-based 3D science animation generation, medical site video generation, and medical teaching video generation.

[0003] However, existing technologies still have significant shortcomings in the automatic generation of medical science videos. On the one hand, general video generation models often produce images that do not conform to anatomical common sense or contain factual errors when processing medical content. This is because existing models lack medical knowledge constraint mechanisms, making it difficult to guarantee the accuracy of anatomical structures and the correctness of medical logic in the generated visual content. On the other hand, existing solutions mostly rely on semi-automated processes of manual script writing and material selection, resulting in relatively low production efficiency. Furthermore, existing GAN models typically only use adversarial loss for optimization in medical image generation, leading to gaps in texture detail and semantic accuracy compared to real medical images, and problems such as mode collapse are prone to occur during training.

[0004] To address the above problems, this invention provides a method and system for producing medical science videos based on deep learning. Summary of the Invention

[0005] One objective of this invention is to propose a method and system for producing medical science videos based on deep learning. This invention utilizes a medical knowledge graph and a multi-level sentence-level evidence chain to construct a generative model that can execute science script generation and video frame synthesis under the constraint of fact consistency. By introducing perceptual loss and feature matching loss, the invention strengthens the dual alignment of generated video frames in the high-level semantic space and the discriminator feature space, and has the advantages of high fidelity of anatomical structure, high accuracy of generated content, and fully automated production process.

[0006] The method for producing medical science videos based on deep learning according to embodiments of the present invention includes the following steps: Obtain medical science popularization texts, retrieve medical knowledge data from medical knowledge graphs, and construct multi-level sentence-level evidence chains; Key medical entities and entity relationships are extracted from multi-level sentence-level evidence chains and input into a large language model to generate a popular science script. The popular science script includes narration, storyboard description and voice-over text. The storyboard description is broken down into an image generation instruction set and a video generation instruction set, which are then input into the PR-GAN generator to generate a medical animation frame sequence. The medical animation frame sequence is input into the discriminator to determine whether it is real or fake, and the adversarial loss is calculated. The medical animation frame sequence and the target medical video frame sequence are respectively input into the pre-trained VGG network, the first feature map and the second feature map of the specified layer are extracted, and the Euclidean distance between them is calculated as the perceptual loss. Extract the feature maps of real samples and generated samples from the intermediate layer of the discriminator, and calculate the L1 distance between them as the feature matching loss. The total loss is obtained by weighted summing of the adversarial loss, perceptual loss and feature matching loss, and the network parameters of the PR-GAN generator and discriminator are updated based on the total loss. The updated PR-GAN generator is used to generate a sequence of medical animation frames. The audio and video are then synthesized in sync with the narration and voice-over text to output a preliminary popular science video. Perform factual consistency checks on the initial science popularization videos, and output the final science popularization videos after the checks pass.

[0007] Optionally, constructing a multi-level sentence-level chain of evidence specifically includes: Retrieve several knowledge entries from the medical knowledge graph that match the medical science popularization text, break down each knowledge entry into several knowledge sentences, assign a source identifier to each knowledge sentence, and organize all knowledge sentences according to the medical topic hierarchy to form a three-layer structure. The first layer is the disease classification node, the second layer is the four branch nodes of symptoms, etiology, diagnosis and treatment, and the third layer is the set of knowledge sentences corresponding to each branch node.

[0008] Optionally, the extraction of key medical entities and entity relationships from multi-level sentence-level evidence chains specifically involves: Named entity recognition is performed on each knowledge sentence in the multi-level sentence-level evidence chain to label medical entities and their types. Medical entity types include disease names, anatomical locations, examination methods, drug names, and treatment methods. All medical entities appearing in the knowledge sentences are paired, and each pair is matched with a predefined entity relationship type, which includes causal relationships, positional relationships, treatment relationships, and examination relationships. The successfully matched medical entity combinations and their corresponding entity relationship types are output as a set of triples.

[0009] Optionally, the step of decomposing the storyboard description into an image generation instruction set and a video generation instruction set specifically includes the following steps: Analyze the storyboard description and identify the first sentence containing static scene descriptions and the second sentence containing dynamic process descriptions; Static image elements are extracted from the first clause. These static image elements include anatomical structure names, organ morphology descriptions, color features, and spatial location information. The static image elements are then formatted into image generation instructions according to the image generation template. All image generation instructions form an image generation instruction set. Extract dynamic process elements from the second clause. The dynamic process elements include the initial state description, the final state description, the motion trajectory description, and the time evolution information. Format the dynamic process elements into video generation instructions according to the video generation template. All video generation instructions form a video generation instruction set.

[0010] Optionally, the input PR-GAN generator for generating a medical animation frame sequence specifically includes the following steps: Input the image generation instruction set into the PR-GAN generator; The PR-GAN generator downsamples the input image step by step through several downsampling layers to obtain feature maps of different scales output by each downsampling layer. The feature map output from the deepest downsampling layer is input into several residual modules. The residual modules sequentially extract and enhance residual features from the feature map, and output a residual enhanced feature map. The residual enhancement feature map is input into several upsampling layers and upsampled step by step to restore the spatial size of the input image, and a single frame medical animation image is output. The process of generating a single frame of medical animation image from the input image is executed repeatedly until all the medical animation images corresponding to the segment descriptions are generated. Based on the time evolution information and motion trajectory information in the video generation instruction set, all generated medical animation images are sorted and frame interpolated to output a medical animation frame sequence.

[0011] Optionally, the specific steps for inputting the medical animation frame sequence into the discriminator for authenticity determination and calculating the adversarial loss are as follows: Each frame of the target medical video frame sequence is used as a real sample and input into the discriminator frame by frame. The discriminator extracts features from the input image through a backbone network composed of several convolutional layers, batch normalization layers and LeakyReLU activation functions stacked alternately. The last convolutional layer of the backbone network outputs a feature map. The feature map is flattened into a one-dimensional feature vector and then input into the fully connected layer of the discriminator. After being mapped by the Sigmoid activation function, the fully connected layer outputs the first probability value that the real sample is true. Each frame of the medical animation frame sequence is used as a generated sample and input into the discriminator frame by frame. The discriminator extracts features from the input image through a backbone network composed of multiple convolutional layers, batch normalization layers and LeakyReLU activation functions stacked alternately. The last convolutional layer of the backbone network outputs a feature map. The feature map is flattened into a one-dimensional feature vector and then input into the fully connected layer of the discriminator. After being mapped by the Sigmoid activation function, the fully connected layer outputs a second probability value that the generated sample is true. Substitute the first probability value into the first log loss function to calculate the true sample discrimination loss; Substituting the second probability value into the second log loss function, the generated sample discrimination loss is calculated. The adversarial loss is obtained by calculating the discriminant loss for all real samples and the discriminant loss for all generated samples separately.

[0012] Optionally, inputting the medical animation frame sequence and the target medical video frame sequence into the pre-trained VGG network respectively includes the following steps: Each frame of the medical animation frame sequence is used as a sample image to be generated and fed into the pre-trained VGG network frame by frame. The pre-trained VGG network contains several sequentially arranged convolutional blocks. Each convolutional block consists of several stacked convolutional layers and pooling layers. Each convolutional layer is configured with a convolutional kernel of a set size, a set stride, and a set padding. The pooling layer uses max pooling with a set window size and a set stride. Each convolutional layer is followed by a ReLU activation function for non-linear mapping. The generated image samples pass through several convolutional blocks sequentially starting from the first convolutional block. The first feature map is output from the last convolutional layer of the specified convolutional block after the middle position in the sorting. The first feature map contains the texture features, edge features and semantic features of the generated image sample. Each frame of the target medical video frame sequence is used as a target image sample and input into the same pre-trained VGG network frame by frame. The target image sample passes through several convolutional blocks in sequence starting from the first convolutional block. The second feature map is output from the last convolutional layer of the specified convolutional block after the middle position in the sorting. The second feature map contains the texture features, edge features and semantic features of the target image sample. Flatten the first feature map into a one-dimensional vector to obtain the first feature vector. Flatten the second feature map into a one-dimensional vector to obtain the second feature vector. Calculate the square of the difference between the elements of the first feature vector and the second feature vector at the same dimension position. The Euclidean distance is calculated by squared values ​​of all dimensional positions, and this Euclidean distance is used as the perceptual loss.

[0013] Optionally, updating the network parameters of the PR-GAN generator and discriminator based on the total loss specifically includes the following steps: The network parameters of the PR-GAN generator are frozen, and the backpropagation gradient of the adversarial loss part of the total loss is passed to the discriminator. The discriminator updates its network parameters based on the backpropagation gradient, so that the probability value of the discriminator outputting real samples approaches 1 and the probability value of the discriminator outputting generated samples approaches 0. Unfreeze the network parameters of the PR-GAN generator, freeze the network parameters of the discriminator, pass the backpropagation gradient of the total loss to the PR-GAN generator, and update the network parameters of the PR-GAN generator based on the backpropagation gradient. Alternately update the discriminator network parameters and the PR-GAN generator network parameters until the network parameters of the PR-GAN generator and discriminator converge.

[0014] Optionally, the factual consistency verification of the preliminary science popularization video specifically includes the following steps: Extract the narration text, subtitle text, and on-screen text tags from the initial science popularization video and merge them into the text to be verified. The text to be verified is split into several knowledge sentences to be verified; Each knowledge sentence to be verified is semantically matched with each knowledge sentence in the multi-level sentence-level evidence chain, the semantic similarity is calculated, the knowledge sentence of the evidence chain with the highest semantic similarity is selected as the matching knowledge sentence, and the semantic similarity is used as the matching score. When the matching score is greater than the set threshold, the knowledge sentence to be verified passes the verification; when the matching score is less than or equal to the set threshold, the knowledge sentence to be verified fails the verification. If the number of unverified knowledge sentences exceeds the set tolerance value, a regeneration process is triggered; if the number of unverified knowledge sentences is less than or equal to the set tolerance value, the final science video is output.

[0015] The medical science popularization video production system based on deep learning according to an embodiment of the present invention includes the following modules: The knowledge acquisition module is used to acquire medical science popularization texts, retrieve medical knowledge data from the medical knowledge graph, and construct multi-level sentence-level evidence chains; The script generation module is used to extract key medical entities and entity relationships from multi-level sentence-level evidence chains, input them into a large language model to generate popular science scripts containing narration, scene descriptions and voice-over text; The image generation module is used to decompose the storyboard description into an image generation instruction set, input it into the PR-GAN generator, generate a single frame of medical animation image, and execute it in a loop until all the medical animation images corresponding to the storyboard description are generated. The video synthesis module is used to arrange all medical animation images into a sequence of medical animation frames, combine them with narration and voice-over text to synthesize audio and video synchronously, and output a preliminary popular science video. The adversarial loss calculation module is used to input the target medical video frame sequence and medical animation frame sequence into the discriminator, and calculate the adversarial loss based on the first probability value and the second probability value output by the discriminator. The perceptual loss calculation module is used to input the medical animation frame sequence and the target medical video frame sequence into the pre-trained VGG network, respectively, extract the first feature map and the second feature map output by the specified convolutional block, and calculate the Euclidean distance between them as the perceptual loss. The feature matching loss calculation module is used to extract the real sample feature map and the generated sample feature map output by the intermediate convolutional layer of the discriminator, and calculate the L1 distance between the two as the feature matching loss. The parameter update module is used to obtain the total loss by weighted summing of adversarial loss, perceptual loss and feature matching loss, and to alternately update the network parameters of the discriminator and PR-GAN generator based on the total loss. The fact consistency verification module is used to split the text content in the initial science popularization video into multiple knowledge sentences to be verified. Each knowledge sentence to be verified is semantically matched with each knowledge sentence in the multi-level sentence-level evidence chain. If the matching score is greater than a set threshold, the verification is passed. If the number of knowledge sentences to be verified that fail the verification is greater than a set fault tolerance value, the regeneration process is triggered. If the number of knowledge sentences to be verified that fail the verification is less than or equal to the set fault tolerance value, the final science popularization video is output.

[0016] The beneficial effects of this invention are: This invention constructs a multi-level sentence-level evidence chain to constrain the popular science scripts generated by the large language model within the scope of authoritative medical knowledge, thus solving the problem that the accuracy of medical content generated by general models cannot be guaranteed, and ensuring that every piece of popular science information is verifiable and traceable.

[0017] This invention introduces the perceptual loss of a pre-trained VGG network during the training process of the PR-GAN generator, constraining the generated medical animation frames to align with real medical images in the high-level semantic feature space, making the generated anatomical structure morphology and texture details closer to real medical images, and improving the visual realism of popular science videos.

[0018] This invention introduces feature matching loss in the intermediate layer of the discriminator during the training process of the PR-GAN generator, which aligns the feature distribution of the generated samples with the feature distribution of the real samples, effectively suppressing the pattern collapse problem in GAN training and improving the diversity and coherence of the generated medical animation frame sequences.

[0019] This invention weighted and fused adversarial loss, perceptual loss, and feature matching loss into a total loss, constraining the generator's optimization direction from three levels: pixel level, semantic level, and feature distribution level, thus taking into account the realism of the generated image, the accuracy of the anatomical structure, and the training stability.

[0020] This invention integrates medical knowledge acquisition, popular science script generation, medical animation generation, video synthesis, and factual consistency verification into an end-to-end fully automated process, eliminating the need for manual script writing and material selection, and significantly reducing the production threshold and cost of medical popular science videos. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of the deep learning-based medical science popularization video production method proposed in this invention; Figure 2 This is a flowchart illustrating the perceptual loss calculation process of the deep learning-based medical science popularization video production method proposed in this invention. Figure 3 This is a flowchart illustrating the feature matching loss calculation process for the deep learning-based medical science popularization video production method proposed in this invention. Figure 4 This is a module structure diagram of the deep learning-based medical science popularization video production system proposed in this invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0023] refer to Figures 1-4 The method for producing medical science videos based on deep learning includes the following steps: Obtain medical science popularization texts, retrieve medical knowledge data from medical knowledge graphs, and construct multi-level sentence-level evidence chains; Key medical entities and entity relationships are extracted from multi-level sentence-level evidence chains and input into a large language model to generate a popular science script. The popular science script includes narration, storyboard description and voice-over text. The storyboard description is broken down into an image generation instruction set and a video generation instruction set, which are then input into the PR-GAN generator to generate a medical animation frame sequence. The medical animation frame sequence is input into the discriminator to determine whether it is real or fake, and the adversarial loss is calculated. The medical animation frame sequence and the target medical video frame sequence are respectively input into the pre-trained VGG network, the first feature map and the second feature map of the specified layer are extracted, and the Euclidean distance between them is calculated as the perceptual loss. Extract the feature maps of real samples and generated samples from the intermediate layer of the discriminator, and calculate the L1 distance between them as the feature matching loss. The total loss is obtained by weighted summing of the adversarial loss, perceptual loss and feature matching loss, and the network parameters of the PR-GAN generator and discriminator are updated based on the total loss. The updated PR-GAN generator is used to generate a sequence of medical animation frames. The audio and video are then synthesized in sync with the narration and voice-over text to output a preliminary popular science video. Perform factual consistency checks on the initial science popularization videos, and output the final science popularization videos after the checks pass.

[0024] In this embodiment, constructing a multi-level sentence-level evidence chain specifically includes: Retrieve several knowledge entries from the medical knowledge graph that match the medical science popularization text, break down each knowledge entry into several knowledge sentences, assign a source identifier to each knowledge sentence, and organize all knowledge sentences according to the medical topic hierarchy to form a three-layer structure. The first layer is the disease classification node, the second layer is the four branch nodes of symptoms, etiology, diagnosis and treatment, and the third layer is the set of knowledge sentences corresponding to each branch node.

[0025] In this embodiment, extracting key medical entities and entity relationships from a multi-level sentence-level evidence chain specifically involves: Named entity recognition is performed on each knowledge sentence in the multi-level sentence-level evidence chain to label medical entities and their types. Medical entity types include disease names, anatomical locations, examination methods, drug names, and treatment methods. All medical entities appearing in the knowledge sentences are paired, and each pair is matched with a predefined entity relationship type, which includes causal relationships, positional relationships, treatment relationships, and examination relationships. The successfully matched medical entity combinations and their corresponding entity relationship types are output as a set of triples.

[0026] Specifically, performing named entity recognition on each knowledge sentence in a multi-level sentence-level evidence chain includes the following steps: Each character in each knowledge sentence is assigned a label, which is used to mark the character's position and entity type in the medical entity; Each knowledge sentence after being labeled is input into the named entity recognition model, which includes an embedding layer, a sequence encoding layer, and a label output layer. The embedding layer converts each character into a numerical vector; The sequence coding layer uses a bidirectional long short-term memory network to calculate the numerical vector of each character in both the forward and reverse directions. The results of the calculation in both directions are concatenated to output the comprehensive feature vector of each character. The label output layer uses a conditional random field to output the optimal label for each character based on the comprehensive feature vector of each character; Based on the optimal label of each character, consecutive characters with the same label type are combined into a complete medical entity, and the entity type to which the medical entity belongs is determined.

[0027] In this embodiment, decomposing the storyboard description into an image generation instruction set and a video generation instruction set specifically includes the following steps: Analyze the storyboard description and identify the first sentence containing static scene descriptions and the second sentence containing dynamic process descriptions; Static image elements are extracted from the first clause. These static image elements include anatomical structure names, organ morphology descriptions, color features, and spatial location information. The static image elements are then formatted into image generation instructions according to the image generation template. All image generation instructions form an image generation instruction set. Extract dynamic process elements from the second clause. The dynamic process elements include the initial state description, the final state description, the motion trajectory description, and the time evolution information. Format the dynamic process elements into video generation instructions according to the video generation template. All video generation instructions form a video generation instruction set.

[0028] The process of formatting static image elements into image generation instructions according to an image generation template includes the following steps: Write the names of anatomical structures in static image elements into the main field of the template, write the organ morphology descriptions into the detail field of the template, write the color features into the color scheme field of the template, and write the spatial position information into the layout field of the template. Image generation templates that match the names of anatomical structures are retrieved from the template library. The image generation templates include subject placeholders, detail placeholders, color placeholders, and layout placeholders. Enter the anatomical structure name into the main placeholder, the organ morphology description into the detail placeholder, the color feature into the color scheme placeholder, and the spatial location information into the layout placeholder, and combine them to generate the image generation command.

[0029] The video generation instruction set, which comprises all video generation instructions, specifically includes: All video generation instructions are arranged in the order of appearance of the second clause in the storyboard description, and each video generation instruction carries the corresponding storyboard identifier and timestamp information; All the arranged video generation instructions are packaged into a video generation instruction set, which includes an identifier for the total number of instructions and the start and end frame numbers of each video generation instruction.

[0030] In this embodiment, the process of inputting a PR-GAN generator to generate a medical animation frame sequence includes the following steps: Input the image generation instruction set into the PR-GAN generator; The PR-GAN generator downsamples the input image step by step through several downsampling layers to obtain feature maps of different scales output by each downsampling layer. The feature map output from the deepest downsampling layer is input into several residual modules. The residual modules sequentially extract and enhance residual features from the feature map, and output a residual enhanced feature map. The residual enhancement feature map is input into several upsampling layers and upsampled step by step to restore the spatial size of the input image, and a single frame medical animation image is output. The process of generating a single frame of medical animation image from the input image is executed repeatedly until all the medical animation images corresponding to the segment descriptions are generated. Based on the time evolution information and motion trajectory information in the video generation instruction set, all generated medical animation images are sorted and frame interpolated to output a medical animation frame sequence.

[0031] The PR-GAN generator performs successive downsampling of the input image through several downsampling layers, specifically including the following steps: The input image is passed to the first downsampling layer. The first downsampling layer performs convolution operation on the input image using the set convolution kernel size and stride to extract shallow features of the input image and obtain the first scale feature map. The first-scale feature map is passed to the second downsampling layer. The second downsampling layer performs convolution operation on the first-scale feature map using the set convolution kernel size and stride to obtain the second-scale feature map. The second-scale feature map is passed to each subsequent downsampling layer. Each downsampling layer performs a convolution operation on the feature map output by the previous layer to obtain feature maps of different scales in sequence. The spatial resolution of each scale feature map decreases layer by layer, and the number of channels increases layer by layer. After each downsampling layer completes the convolution operation, the result of the convolution operation is batch normalized and the activation function is mapped before being output to the next downsampling layer.

[0032] The process of inputting the residual enhancement feature map into several upsampling layers for sequential upsampling includes the following steps: The residual enhanced feature map is passed to the first upsampling layer. The first upsampling layer performs deconvolution operation on the residual enhanced feature map using the set deconvolution kernel size and stride to obtain the first scale upsampling feature map. The first upsampled feature map is concatenated with the feature map output by the downsampled layer at the corresponding scale to obtain the first concatenated feature map. The first stitched feature map is passed to the second upsampling layer. The second upsampling layer performs deconvolution operation on the first stitched feature map using the set deconvolution kernel size and stride to obtain the second scale upsampling feature map. The upsampled feature map at the second scale is concatenated with the feature map output by the downsampled layer at the corresponding scale to obtain the second concatenated feature map. The second concatenated feature map is passed to each subsequent upsampling layer. Each upsampling layer performs a deconvolution operation on the concatenated feature map output by the previous layer, concatenates it with the feature map output by the downsampling layer at the corresponding scale, and then outputs it to the next upsampling layer. After the last upsampling layer completes the deconvolution and stitching operations, it performs a convolution operation on the stitched result to map the number of channels in the feature map to the number of channels in the input image, restoring the spatial size of the input image.

[0033] In this embodiment, the medical animation frame sequence is input into the discriminator for authenticity determination, and the specific steps for calculating the adversarial loss are as follows: Each frame of the target medical video frame sequence is used as a real sample and input into the discriminator frame by frame. The discriminator extracts features from the input image through a backbone network composed of several convolutional layers, batch normalization layers and LeakyReLU activation functions stacked alternately. The last convolutional layer of the backbone network outputs a feature map. The feature map is flattened into a one-dimensional feature vector and then input into the fully connected layer of the discriminator. After being mapped by the Sigmoid activation function, the fully connected layer outputs the first probability value that the real sample is true. Each frame of the medical animation frame sequence is used as a generated sample and input into the discriminator frame by frame. The discriminator extracts features from the input image through a backbone network composed of multiple convolutional layers, batch normalization layers and LeakyReLU activation functions stacked alternately. The last convolutional layer of the backbone network outputs a feature map. The feature map is flattened into a one-dimensional feature vector and then input into the fully connected layer of the discriminator. After being mapped by the Sigmoid activation function, the fully connected layer outputs a second probability value that the generated sample is true. Substitute the first probability value into the first log loss function, which is defined as the negative of the natural logarithm of the first probability value, to calculate the true sample discrimination loss. Substitute the second probability value into the second log loss function, which is defined as the negative of the natural logarithm of the difference between the second probability value and the second log loss value, to calculate the generated sample discrimination loss. The adversarial loss is obtained by averaging the loss of all real samples and the loss of all generated samples.

[0034] The backbone network performs feature extraction on the input image, specifically including the following steps: The input image is passed to the first convolutional layer. The first convolutional layer performs convolution operations on the input image using the set convolutional kernel size, stride and padding method. The convolution operation result is normalized by the batch normalization layer and then input into the LeakyReLU activation function for non-linear mapping, and outputs the first convolutional feature map. The first convolutional feature map is passed to the first pooling layer. The first pooling layer performs max pooling on the first convolutional feature map using a set window size and stride, and outputs the first pooled feature map. The first pooling feature map is passed to each subsequent convolutional and pooling layer. Each convolutional layer performs convolution, batch normalization and LeakyReLU activation in sequence, and each pooling layer performs max pooling. The pooling feature maps output by each pooling layer are used as input to the next convolutional layer. After alternating processing by all convolutional layers and pooling layers, the last convolutional layer of the backbone network outputs the final feature map.

[0035] The process of the fully connected layer outputting the first probability value of a true sample after being mapped by the Sigmoid activation function includes the following steps: The one-dimensional feature vector is input into the fully connected layer. The fully connected layer multiplies each element in the one-dimensional feature vector by the corresponding weight parameter and sums them. Then, the bias parameter is added to obtain the first linear weighted sum. The first linearly weighted sum is input into the Sigmoid activation function, which maps the first linearly weighted sum to a value between 0 and 1, and this value is used as the first probability value that the real sample is true.

[0036] In this embodiment, inputting the medical animation frame sequence and the target medical video frame sequence into the pre-trained VGG network specifically includes the following steps: Each frame of the medical animation frame sequence is used as a sample image to be generated and fed into the pre-trained VGG network frame by frame. The pre-trained VGG network contains several sequentially arranged convolutional blocks. Each convolutional block consists of several stacked convolutional layers and pooling layers. Each convolutional layer is configured with a convolutional kernel of a set size, a set stride, and a set padding. The pooling layer uses max pooling with a set window size and a set stride. Each convolutional layer is followed by a ReLU activation function for non-linear mapping. The generated image samples pass through several convolutional blocks sequentially starting from the first convolutional block. The first feature map is output from the last convolutional layer of the specified convolutional block after the middle position in the sorting. The first feature map contains the texture features, edge features and semantic features of the generated image sample. Each frame of the target medical video frame sequence is used as a target image sample and input into the same pre-trained VGG network frame by frame. The target image sample passes through several convolutional blocks in sequence starting from the first convolutional block. The second feature map is output from the last convolutional layer of the specified convolutional block after the middle position in the sorting. The second feature map contains the texture features, edge features and semantic features of the target image sample. Flatten the first feature map into a one-dimensional vector to obtain the first feature vector. Flatten the second feature map into a one-dimensional vector to obtain the second feature vector. Calculate the square of the difference between the elements of the first feature vector and the second feature vector at the same dimension position. The squared values ​​of all dimensional positions are summed to obtain the sum of squares. The square root of the sum of squares is then taken to obtain the Euclidean distance, which is used as the perceptual loss.

[0037] The frame-by-frame input to the pre-trained VGG network specifically includes the following steps: Each frame in the medical animation frame sequence is extracted sequentially according to time order, with one frame extracted at a time as the current input image. Adjust the current input image to the input size required by the pre-trained VGG network; The resized current input image is input into the first convolutional block of the pre-trained VGG network, and after processing by the first convolutional block, it is output to the next convolutional block. After the current input image passes through all convolutional blocks in sequence, the feature map output by the specified convolutional block is extracted as the feature map corresponding to the frame image. Continue to extract the next frame image from the medical animation frame sequence and repeat the above steps until all frame images in the medical animation frame sequence have been processed. The target medical video frame sequence is input into the same pre-trained VGG network frame by frame using the same steps to obtain the feature map corresponding to each frame of the target image.

[0038] The process of flattening the first feature map into a one-dimensional vector includes the following steps: Obtain the number of channels in the channel dimension, the number of pixels in width in the horizontal direction, and the number of pixels in height in the vertical direction of the first feature map; The element values ​​of each position in the first feature map are read sequentially according to the channel dimension, vertical direction and horizontal direction, and all element values ​​are arranged into a one-dimensional sequence. The one-dimensional sequence is output as the first feature vector.

[0039] The process of extracting the feature maps of real and generated samples from the intermediate layer of the discriminator and calculating the L1 distance between them as the feature matching loss includes the following steps: The target medical video frame sequence is used as the real sample input to the discriminator. After the discriminator processes the data through multiple convolutional layers and pooling layers, it outputs the real sample feature map at the designated intermediate convolutional layer of the discriminator. The medical animation frame sequence is used as the input of the discriminator to generate samples. After the discriminator processes the samples through multiple convolutional and pooling layers, the sample feature map is output at the specified intermediate convolutional layer of the discriminator. The absolute difference between the elements at the same spatial location of the real sample feature map and the generated sample feature map is obtained by subtracting the elements at the same spatial location from the absolute value. The absolute differences at all spatial locations are summed to obtain the L1 distance, which is then used as the feature matching loss.

[0040] The discriminator contains multiple intermediate convolutional layers. The feature matching loss can be calculated by selecting the feature map output from one of the intermediate convolutional layers, or by selecting the feature maps output from multiple intermediate convolutional layers, calculating the L1 distance separately, and then summing them by weight.

[0041] In this embodiment, updating the network parameters of the PR-GAN generator and discriminator based on the total loss specifically includes the following steps: The network parameters of the PR-GAN generator are frozen, and the backpropagation gradient of the adversarial loss part of the total loss is passed to the discriminator. The discriminator updates its network parameters based on the backpropagation gradient, so that the probability value of the discriminator outputting real samples approaches 1 and the probability value of the discriminator outputting generated samples approaches 0. Unfreeze the network parameters of the PR-GAN generator, freeze the network parameters of the discriminator, pass the backpropagation gradient of the total loss to the PR-GAN generator, and update the network parameters of the PR-GAN generator based on the backpropagation gradient. Alternately update the discriminator network parameters and the PR-GAN generator network parameters until the network parameters of the PR-GAN generator and discriminator converge.

[0042] Freezing the network parameters of the PR-GAN generator specifically includes the following steps: Iterate through all network layers of the PR-GAN generator, and for each network layer, set the gradient update flag of all parameters in that network layer to the off state; During backpropagation, the gradient calculation stops when it reaches the network layer where the gradient update flag is turned off, and the parameter values ​​of that network layer remain unchanged.

[0043] The backpropagation process is as follows: Starting from the total loss, the gradient values ​​of each network layer parameter are calculated layer by layer in the reverse order of the PR-GAN generator and discriminator network structures. The gradient value of each network layer parameter is the partial derivative of the total loss with respect to that network layer parameter. The gradient values ​​of each network layer parameter are combined with the current values ​​of that network layer parameter, and the network layer parameter is updated according to the set learning rate.

[0044] The specific steps for unfreezing the network parameters of the PR-GAN generator are as follows: Iterate through all network layers of the PR-GAN generator, and for each network layer, set the gradient update flag of all parameters in that network layer to the open state; The backpropagation gradient of the total loss is passed to the PR-GAN generator, and the PR-GAN generator updates all network layer parameters of the PR-GAN generator based on the backpropagation gradient. After the PR-GAN generator is updated, the network parameter values ​​of the PR-GAN generator change, while the network parameter values ​​of the discriminator remain unchanged.

[0045] The alternating updating of the discriminator network parameters and the PR-GAN generator network parameters specifically includes the following steps: In one training epoch, the network parameters of the PR-GAN generator are first frozen, and the backpropagation gradient of the adversarial loss part of the total loss is passed to the discriminator. The discriminator updates its network parameters according to the backpropagation gradient, so that the probability value of the discriminator outputting real samples approaches 1 and the probability value of the discriminator outputting generated samples approaches 0. Then, unfreeze the network parameters of the PR-GAN generator, freeze the network parameters of the discriminator, and pass the backpropagation gradient of the total loss to the PR-GAN generator. The PR-GAN generator updates its network parameters based on the backpropagation gradient, thereby reducing the total loss. The process of updating the discriminator network parameters and the process of updating the PR-GAN generator network parameters are executed alternately and repeatedly. In each alternation process, the parameters of the other network are updated based on the state of the other network before the current alternation process, until the network parameters of the PR-GAN generator and the discriminator converge.

[0046] In this embodiment, the factual consistency verification of the preliminary science popularization video specifically includes the following steps: Extract the narration text, subtitle text, and on-screen text tags from the initial science popularization video and merge them into the text to be verified. The text to be verified is split into several knowledge sentences to be verified; Each knowledge sentence to be verified is semantically matched with each knowledge sentence in the multi-level sentence-level evidence chain, the semantic similarity is calculated, the knowledge sentence of the evidence chain with the highest semantic similarity is selected as the matching knowledge sentence, and the semantic similarity is used as the matching score. The knowledge sentence to be verified passes the verification when the matching score is greater than the set threshold, and fails the verification when the matching score is less than or equal to the set threshold. If the number of unverified knowledge sentences exceeds the set tolerance value, a regeneration process is triggered; if the number of unverified knowledge sentences is less than or equal to the set tolerance value, the final science video is output.

[0047] The semantic matching of each knowledge sentence to be verified with each knowledge sentence in the multi-level sentence-level evidence chain specifically includes the following steps: The knowledge sentence to be verified is input into the semantic encoding model, and the semantic encoding model maps the knowledge sentence to be verified into a first semantic vector. Each knowledge sentence in the evidence chain is input into the same semantic encoding model, and the semantic encoding model maps each knowledge sentence in the evidence chain to a second semantic vector. Calculate the cosine similarity between the first semantic vector and each second semantic vector, and use the cosine similarity as the semantic similarity between the knowledge sentence to be verified and the knowledge sentence of the evidence chain.

[0048] The medical science popularization video production system based on deep learning according to an embodiment of the present invention includes the following modules: The knowledge acquisition module is used to acquire medical science popularization texts, retrieve medical knowledge data from the medical knowledge graph, and construct multi-level sentence-level evidence chains; The script generation module is used to extract key medical entities and entity relationships from multi-level sentence-level evidence chains, input them into a large language model to generate popular science scripts containing narration, scene descriptions and voice-over text; The image generation module is used to decompose the storyboard description into an image generation instruction set, input it into the PR-GAN generator, generate a single frame of medical animation image, and execute it in a loop until all the medical animation images corresponding to the storyboard description are generated. The video synthesis module is used to arrange all medical animation images into a sequence of medical animation frames, combine them with narration and voice-over text to synthesize audio and video synchronously, and output a preliminary popular science video. The adversarial loss calculation module is used to input the target medical video frame sequence and medical animation frame sequence into the discriminator, and calculate the adversarial loss based on the first probability value and the second probability value output by the discriminator. The perceptual loss calculation module is used to input the medical animation frame sequence and the target medical video frame sequence into the pre-trained VGG network, respectively, extract the first feature map and the second feature map output by the specified convolutional block, and calculate the Euclidean distance between them as the perceptual loss. The feature matching loss calculation module is used to extract the real sample feature map and the generated sample feature map output by the intermediate convolutional layer of the discriminator, and calculate the L1 distance between the two as the feature matching loss. The parameter update module is used to obtain the total loss by weighted summing of adversarial loss, perceptual loss and feature matching loss, and to alternately update the network parameters of the discriminator and PR-GAN generator based on the total loss. The fact consistency verification module is used to split the text content in the initial science popularization video into multiple knowledge sentences to be verified. Each knowledge sentence to be verified is semantically matched with each knowledge sentence in the multi-level sentence-level evidence chain. If the matching score is greater than a set threshold, the verification is passed. If the number of knowledge sentences to be verified that fail the verification is greater than a set fault tolerance value, the regeneration process is triggered. If the number of knowledge sentences to be verified that fail the verification is less than or equal to the set fault tolerance value, the final science popularization video is output.

[0049] Example 1: In the field of medical science popularization video production, the traditional method relies on doctors writing scripts, filming teams collecting materials, and post-production editors compositing the video. A typical 3-minute science popularization video takes 5 to 7 working days to complete. After receiving the task of producing a coronary heart disease prevention science popularization video, the science popularization department of a medical institution adopted the method of this invention.

[0050] The operator input "coronary heart disease prevention" as the medical science popularization topic text into the system. The system first retrieves medical knowledge data related to coronary heart disease prevention from the medical knowledge graph, constructing a multi-level sentence-level evidence chain. This evidence chain contains 342 knowledge sentences extracted from guidelines and literature, each with a source identifier. The system extracts key medical entities and entity relationships from the evidence chain, such as 87 triples including "coronary heart disease - statins - treatment relationship" and "high cholesterol - atherosclerosis - causal relationship," which are then input into a large language model to generate a science popularization script.

[0051] After the script is generated, the system automatically decomposes the storyboard descriptions within it. For the storyboard of "the process of coronary vascular plaque formation," the system extracts static image elements, including the names of coronary artery anatomical structures, plaque morphology descriptions, and color features, generating 6 image generation instructions; it also extracts dynamic process elements, including the temporal evolution information of plaques from lipid streaks to fibrous plaques, generating 4 video generation instructions. Both types of instructions are input into the PR-GAN generator, resulting in a medical animation frame sequence containing 240 frames.

[0052] The system trains the PR-GAN generator using a dataset containing 12,000 paired medical images. During training, the generated medical animation frame sequence is input into the discriminator for real / fake detection, and adversarial loss is calculated. Simultaneously, the medical animation frame sequence and the target medical video frame sequence are input into a pre-trained VGG network containing five convolutional blocks. In the experiment, the feature map output from the last convolutional layer of the fourth convolutional block is selected, with a size of 14×14×512, and Euclidean distance is calculated as the perceptual loss. The feature maps of the real and generated samples output from the third convolutional layer of the discriminator are extracted, with a size of 28×28×256, and L1 distance is calculated as the feature matching loss. The adversarial loss, perceptual loss, and feature matching loss are weighted and summed according to a ratio of 1:10:5 to obtain the total loss. The network parameters of the discriminator and generator are alternately updated based on the total loss. After 150 rounds of training, the medical animation frame sequence output by the generator reaches a usable visual quality standard.

[0053] After training, the updated PR-GAN generator regenerated all medical animation frame sequences. The 240 frames were arranged chronologically and synthesized with the narration and voice-over text for audio-visual synchronization, outputting a preliminary science popularization video. The system then performed factual consistency checks on the preliminary video, extracting 156 knowledge sentences from the narration, subtitles, and on-screen text tags. These were semantically matched against 342 knowledge sentences in the evidence chain. 149 knowledge sentences with a matching score greater than the set threshold of 0.75 passed the check, while 7 failed. The number of failures was less than the set tolerance value of 10. After successful verification, the final science popularization video was output. The entire process, from topic input to final video output, took 47 minutes.

[0054] To verify the effectiveness of this invention, a comparative experiment was conducted on the task of producing popular science videos covering four diseases: coronary heart disease, hypertension, diabetes, and stroke. Five videos were produced for each disease, for a total of 20 videos. The comparison was between traditional manual production methods and a benchmark GAN model using only adversarial loss. The evaluation was conducted from three dimensions: production efficiency, content accuracy, and visual quality. The results are shown in Table 1.

[0055] Table 1: Comparison Results of Different Production Methods in Medical Science Popularization Video Production Tasks

[0056] Table 1 presents comparative data on the three production methods in medical science popularization video production tasks. Analyzing production efficiency, the traditional manual method takes an average of 42.5 hours to produce a single video, not including the time cost of pre-production communication and post-production modifications. Using only the baseline GAN model with adversarial loss, which eliminates the need for manual material acquisition and editing, the time is reduced to 2.8 hours. The method of this invention further compresses this time to 0.78 hours, meaning it only takes 47 minutes from inputting the topic to outputting the final video. This is mainly because this invention integrates medical knowledge acquisition, script generation, video synthesis, and fact-checking into an end-to-end process, avoiding the need for manual script writing and repeated modifications.

[0057] From the perspective of content accuracy, the traditional manual method achieved a 98.6% pass rate for factual consistency verification, with only 0.2% of frames containing anatomical structure errors and only 0.3 errors per thousand words in medical concept descriptions. This result was ensured by medical experts personally writing the script and reviewing each frame of the footage. The benchmark GAN model's verification pass rate dropped significantly to 71.3%, meaning that nearly one-third of the generated videos contained factual issues, with 8.7% of frames containing anatomical structure errors—meaning that nearly 9 out of every 100 frames contained problems such as incorrect vascular connections, abnormal ventricular morphology, or valve position misalignment. Medical experts scored only 6.8 points. The method of this invention achieved a 96.8% pass rate, with 0.5% of frames containing anatomical structure errors and only 0.4 errors per thousand words in medical concept descriptions. Medical experts scored 9.4 points, only 0.3 points lower than the traditional manual method's 9.7 points, indicating that the method of this invention has approached the level of human verification in terms of content accuracy. This is attributed to the factual constraints imposed on script generation by multi-level sentence-level evidence chains and the alignment constraints imposed on the semantic features of anatomical structures by perceptual loss.

[0058] From a visual quality perspective, the method of this invention improves the PSNR from 28.4dB to 33.7dB (5.3dB) compared to the baseline model; SSIM improves from 0.821 to 0.913 (0.092), indicating that the generated images are closer to real medical images in terms of pixel-level accuracy and structural similarity; LPIPS decreases from 0.187 to 0.096, a reduction of nearly half, indicating a significant improvement in the perceived realism of the images. The video frame coherence improves from 76.4% to 97.2%, essentially eliminating inter-frame flicker and morphological abrupt changes. This is attributed to the constraint of the feature distribution in the discriminator's intermediate layer by the feature matching loss, enabling the generator to maintain consistency in feature distribution during continuous frame generation.

[0059] Based on the above data, the method of the present invention improves production efficiency by more than 50 times while maintaining accuracy close to that of traditional manual content production. At the same time, the visual quality of the generated video is significantly better than that of the benchmark model using only adversarial loss, verifying the feasibility and effectiveness of the present invention in practical applications.

[0060] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for producing medical science popularization videos based on deep learning, characterized in that, Includes the following steps: Obtain medical science popularization texts, retrieve medical knowledge data from medical knowledge graphs, and construct multi-level sentence-level evidence chains; Key medical entities and entity relationships are extracted from multi-level sentence-level evidence chains and input into a large language model to generate a popular science script. The popular science script includes narration, storyboard description and voice-over text. The storyboard description is broken down into an image generation instruction set and a video generation instruction set, which are then input into the PR-GAN generator to generate a medical animation frame sequence. The medical animation frame sequence is input into the discriminator to determine whether it is real or fake, and the adversarial loss is calculated. The medical animation frame sequence and the target medical video frame sequence are respectively input into the pre-trained VGG network, the first feature map and the second feature map of the specified layer are extracted, and the Euclidean distance between them is calculated as the perceptual loss. Extract the feature maps of real samples and generated samples from the intermediate layer of the discriminator, and calculate the L1 distance between them as the feature matching loss. The total loss is obtained by weighted summing of the adversarial loss, perceptual loss and feature matching loss, and the network parameters of the PR-GAN generator and discriminator are updated based on the total loss. The updated PR-GAN generator is used to generate a sequence of medical animation frames. The audio and video are then synthesized in sync with the narration and voice-over text to output a preliminary popular science video. Perform factual consistency checks on the initial science popularization videos, and output the final science popularization videos after the checks pass.

2. The method for producing medical science popularization videos based on deep learning according to claim 1, characterized in that, The construction of a multi-level sentence-level evidence chain specifically includes: Retrieve several knowledge entries from the medical knowledge graph that match the medical science popularization text, break down each knowledge entry into several knowledge sentences, assign a source identifier to each knowledge sentence, and organize all knowledge sentences according to the medical topic hierarchy to form a three-layer structure. The first layer is the disease classification node, the second layer is the four branch nodes of symptoms, etiology, diagnosis and treatment, and the third layer is the set of knowledge sentences corresponding to each branch node.

3. The method for producing medical science popularization videos based on deep learning according to claim 1, characterized in that, The extraction of key medical entities and entity relationships from multi-level sentence-level evidence chains specifically involves: Named entity recognition is performed on each knowledge sentence in the multi-level sentence-level evidence chain to label medical entities and entity types. Medical entity types include disease name, anatomical location, examination method, drug name, and treatment method. All medical entities appearing in all knowledge sentences are paired together. For each combination, match a predefined entity relationship type, including causal relationship, positional relationship, treatment relationship, and examination relationship. Output the successfully matched medical entity combinations and their corresponding entity relationship types as a set of triples.

4. The method for producing medical science popularization videos based on deep learning according to claim 1, characterized in that, The step of decomposing the storyboard description into an image generation instruction set and a video generation instruction set specifically includes the following steps: Analyze the storyboard description and identify the first sentence containing static scene descriptions and the second sentence containing dynamic process descriptions; Static image elements are extracted from the first clause. These static image elements include anatomical structure names, organ morphology descriptions, color features, and spatial location information. The static image elements are then formatted into image generation instructions according to the image generation template. All image generation instructions form an image generation instruction set. Extract dynamic process elements from the second clause. The dynamic process elements include the initial state description, the final state description, the motion trajectory description, and the time evolution information. Format the dynamic process elements into video generation instructions according to the video generation template. All video generation instructions form a video generation instruction set.

5. The method for producing medical science popularization videos based on deep learning according to claim 1, characterized in that, The process of inputting the PR-GAN generator to generate a sequence of medical animation frames specifically includes the following steps: Input the image generation instruction set into the PR-GAN generator; The PR-GAN generator downsamples the input image step by step through several downsampling layers to obtain feature maps of different scales output by each downsampling layer. The feature map output from the deepest downsampling layer is input into several residual modules. The residual modules sequentially extract and enhance residual features from the feature map, and output a residual enhanced feature map. The residual enhancement feature map is input into several upsampling layers and upsampled step by step to restore the spatial size of the input image, and a single frame medical animation image is output. The process of generating a single frame of medical animation image from the input image is executed repeatedly until all the medical animation images corresponding to the segment descriptions are generated. Based on the time evolution information and motion trajectory information in the video generation instruction set, all generated medical animation images are sorted and frame interpolated to output a medical animation frame sequence.

6. The method for producing medical science popularization videos based on deep learning according to claim 1, characterized in that, The specific steps for inputting the medical animation frame sequence into the discriminator for authenticity determination and calculating the adversarial loss are as follows: Each frame of the target medical video frame sequence is used as a real sample and input into the discriminator frame by frame. The discriminator extracts features from the input image through a backbone network composed of several convolutional layers, batch normalization layers and LeakyReLU activation functions stacked alternately. The last convolutional layer of the backbone network outputs a feature map. The feature map is flattened into a one-dimensional feature vector and then input into the fully connected layer of the discriminator. After being mapped by the Sigmoid activation function, the fully connected layer outputs the first probability value that the real sample is true. Each frame of the medical animation frame sequence is used as a generated sample and input into the discriminator frame by frame. The discriminator extracts features from the input image through a backbone network composed of multiple convolutional layers, batch normalization layers and LeakyReLU activation functions stacked alternately. The last convolutional layer of the backbone network outputs a feature map. The feature map is flattened into a one-dimensional feature vector and then input into the fully connected layer of the discriminator. After being mapped by the Sigmoid activation function, the fully connected layer outputs a second probability value that the generated sample is true. Substitute the first probability value into the first log loss function to calculate the true sample discrimination loss; Substitute the second probability value into the second log loss function to calculate the generated sample discrimination loss; The adversarial loss is obtained by calculating the discriminant loss for all real samples and the discriminant loss for all generated samples separately.

7. The method for producing medical science popularization videos based on deep learning according to claim 1, characterized in that, The step of inputting the medical animation frame sequence and the target medical video frame sequence into the pre-trained VGG network includes the following steps: Each frame of the medical animation frame sequence is used as a sample image to be generated and fed into the pre-trained VGG network frame by frame. The pre-trained VGG network contains several sequentially arranged convolutional blocks. Each convolutional block consists of several stacked convolutional layers and pooling layers. Each convolutional layer is configured with a convolutional kernel of a set size, a set stride, and a set padding. The pooling layer uses max pooling with a set window size and a set stride. Each convolutional layer is followed by a ReLU activation function for non-linear mapping. The generated image samples pass through several convolutional blocks sequentially starting from the first convolutional block. The first feature map is output from the last convolutional layer of the specified convolutional block after the middle position in the sorting. The first feature map contains the texture features, edge features and semantic features of the generated image sample. Each frame of the target medical video frame sequence is used as a target image sample and input into the same pre-trained VGG network frame by frame. The target image sample passes through several convolutional blocks sequentially starting from the first convolutional block. The second feature map is output from the last convolutional layer of the specified convolutional block after the middle position in the sorting. The second feature map contains the texture features, edge features and semantic features of the target image sample. Flatten the first feature map into a one-dimensional vector to obtain the first feature vector. Flatten the second feature map into a one-dimensional vector to obtain the second feature vector. Calculate the square of the difference between the elements of the first feature vector and the second feature vector at the same dimension position. The Euclidean distance is calculated by squared values ​​of all dimensional positions, and this Euclidean distance is used as the perceptual loss.

8. The method for producing medical science popularization videos based on deep learning according to claim 1, characterized in that, The process of updating the network parameters of the PR-GAN generator and discriminator based on the total loss specifically includes the following steps: The network parameters of the PR-GAN generator are frozen, and the backpropagation gradient of the adversarial loss part of the total loss is passed to the discriminator. The discriminator updates its network parameters based on the backpropagation gradient, so that the probability value of the discriminator outputting real samples approaches 1 and the probability value of the discriminator outputting generated samples approaches 0. Unfreeze the network parameters of the PR-GAN generator, freeze the network parameters of the discriminator, pass the backpropagation gradient of the total loss to the PR-GAN generator, and update the network parameters of the PR-GAN generator based on the backpropagation gradient. Alternately update the discriminator network parameters and the PR-GAN generator network parameters until the network parameters of the PR-GAN generator and discriminator converge.

9. The method for producing medical science popularization videos based on deep learning according to claim 1, characterized in that, The specific steps involved in performing factual consistency verification on the preliminary science popularization video are as follows: Extract the narration text, subtitle text, and on-screen text tags from the initial science popularization video and merge them into the text to be verified. The text to be verified is split into several knowledge sentences to be verified; Each knowledge sentence to be verified is semantically matched with each knowledge sentence in the multi-level sentence-level evidence chain, the semantic similarity is calculated, the knowledge sentence of the evidence chain with the highest semantic similarity is selected as the matching knowledge sentence, and the semantic similarity is used as the matching score. When the matching score is greater than the set threshold, the knowledge sentence to be verified passes the verification; when the matching score is less than or equal to the set threshold, the knowledge sentence to be verified fails the verification. If the number of unverified knowledge sentences exceeds the set tolerance value, a regeneration process is triggered; if the number of unverified knowledge sentences is less than or equal to the set tolerance value, the final science video is output.

10. A deep learning-based medical science popularization video production system, applied to any one of the deep learning-based medical science popularization video production methods according to claims 1 to 9, characterized in that, Includes the following modules: The knowledge acquisition module is used to acquire medical science popularization texts, retrieve medical knowledge data from the medical knowledge graph, and construct multi-level sentence-level evidence chains; The script generation module is used to extract key medical entities and entity relationships from multi-level sentence-level evidence chains, input them into a large language model to generate popular science scripts containing narration, storyboard descriptions and narration text; The image generation module is used to decompose the storyboard description into an image generation instruction set, input it into the PR-GAN generator, generate a single frame of medical animation image, and execute it in a loop until all the medical animation images corresponding to the storyboard description are generated. The video synthesis module is used to arrange all medical animation images into a sequence of medical animation frames, combine them with narration and voice-over text to synthesize audio and video synchronously, and output a preliminary popular science video. The adversarial loss calculation module is used to input the target medical video frame sequence and medical animation frame sequence into the discriminator, and calculate the adversarial loss based on the first probability value and the second probability value output by the discriminator. The perceptual loss calculation module is used to input the medical animation frame sequence and the target medical video frame sequence into the pre-trained VGG network, respectively, extract the first feature map and the second feature map output by the specified convolutional block, and calculate the Euclidean distance between them as the perceptual loss. The feature matching loss calculation module is used to extract the real sample feature map and the generated sample feature map output by the intermediate convolutional layer of the discriminator, and calculate the L1 distance between the two as the feature matching loss. The parameter update module is used to obtain the total loss by weighted summing of adversarial loss, perceptual loss and feature matching loss, and to alternately update the network parameters of the discriminator and PR-GAN generator based on the total loss. The fact consistency verification module is used to split the text content in the initial science popularization video into multiple knowledge sentences to be verified. Each knowledge sentence to be verified is semantically matched with each knowledge sentence in the multi-level sentence-level evidence chain. If the matching score is greater than a set threshold, the verification is passed. If the number of knowledge sentences to be verified that fail the verification is greater than a set fault tolerance value, the regeneration process is triggered. If the number of knowledge sentences to be verified that fail the verification is less than or equal to the set fault tolerance value, the final science popularization video is output.

Images