Generation method and system applied to traditional village travel video scene description text

Abstract

The invention provides a method and a system for generating a traditional village travel video scene description text. The method comprises the following steps: collecting and preprocessing a village video; establishing a multi-target detection model, identifying and positioning targets in the preprocessed village video, obtaining video key information, and constructing structured video content representation based on the video key information; based on a village characteristic knowledge graph, constructing a village knowledge base, and performing semantic association on the video content representation and the village knowledge base to obtain structured data with characteristic annotations; and constructing a sequence-to-sequence text generation model based on an attention mechanism, and inputting the structured data with the characteristic labels into the sequence-to-sequence text generation model to obtain a description text applied to a traditional village text travel video scene. The method can ensure that the generated description text is accurate and vivid, can attract the attention of the user, and improves the user experience.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a method and system for generating description text for a traditional village cultural tourism video scene. Background Art

[0002] The generation of description texts for traditional village cultural tourism video scenes faces many technical challenges. First, the content of the video is complex and diverse, containing a large number of objects, characters, scenes, activities and other elements, and it is necessary to accurately identify and understand the key information. Secondly, different villages have unique regional characteristics, cultural customs and historical stories. How to mine and extract the themes and highlights that best reflect the characteristics of the village based on the video content is a major difficulty. Furthermore, the description text must be concise, vivid and interesting, and must also conform to the reading tastes and appreciation habits of the target audience, which places high demands on the language style and semantic coherence of text generation. At the same time, the length of village video materials is often tens of minutes or even hours. The massive amount of video data brings tremendous pressure to the analysis and processing, and it is necessary to design efficient algorithms to balance computing resources and generation quality. In addition, the shooting methods and picture quality of different villages vary, and environmental factors such as lighting and sound at the video collection site also have a great impact on information extraction. In summary, the generation of text descriptions for traditional village cultural and tourism video scenes requires comprehensive consideration of cutting-edge technologies in multiple artificial intelligence fields such as computer vision, natural language processing, and knowledge mining, and overcoming a series of technical difficulties such as semantic understanding, feature extraction, language generation, and massive data processing, in order to ultimately achieve automatic, high-quality, and personalized text generation. Summary of the invention

[0003] In order to solve the problems existing in the prior art, the present invention provides a method and system for generating description text for traditional village cultural tourism video scenes.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A method for generating a description text of a scene in a traditional village cultural tourism video comprises the following steps:

[0006] Collect and pre-process village videos;

[0007] Establish a multi-target detection model to identify and locate targets in the preprocessed village video, obtain key video information, and construct a structured video content representation based on the key video information;

[0008] Based on the village characteristic knowledge graph, a village knowledge base is constructed, and the video content representation is semantically associated with the village knowledge base to obtain structured data with characteristic annotations;

[0009] A sequence-to-sequence text generation model based on the attention mechanism is constructed, and structured data with characteristic annotations are input into the sequence-to-sequence text generation model to obtain descriptive text applied to traditional village cultural tourism video scenes.

[0010] Preferably, the method for preprocessing the village video includes:

[0011] Acquire a video frame sequence of the village video, and adjust the brightness and contrast of each frame image to obtain an enhanced image frame sequence;

[0012] Using wavelet transform to denoise the enhanced image frame sequence to obtain a denoised image frame sequence;

[0013] Based on motion estimation and motion compensation technology, new interpolation frames are inserted between adjacent frames of the denoised image frame sequence to obtain a video frame sequence after interpolation.

[0014] A video super-resolution algorithm based on deep learning is used to process the video frame sequence after frame complementation to obtain a video sequence that meets a preset resolution, thereby completing the preprocessing of the village video.

[0015] Preferably, the network structure of the multi-target detection model includes a feature extraction network CSPDarkNet53, a path aggregation network PANet module and a spatial pyramid pooling SPP module.

[0016] Preferably, the method for obtaining structured data with characteristic annotations includes:

[0017] Using entity linking technology, semantically associating the video content representation with entities in the village knowledge base; wherein the video content representation includes scene, character and activity information;

[0018] Extracting relationships and mapping attributes between the video content representation and entities in the village database;

[0019] Based on the results of the semantic association, the relationship extraction and the attribute mapping, characteristic annotations are established to obtain structured data with characteristic annotations.

[0020] Preferably, the network structure of the sequence-to-sequence text generation model based on the attention mechanism includes an encoder, a decoder and an attention mechanism module; wherein,

[0021] The encoder uses a gated recurrent unit to process the input sequence word by word to obtain a hidden state sequence as an intermediate representation;

[0022] The decoder uses a recurrent neural network to generate an output sequence word by word using the received hidden state sequence as an initial state, wherein the generation of each word is based on the output of the previous word and the current hidden state;

[0023] In the attention module, the query vector corresponds to the hidden state of the decoder, and the key vector and value vector correspond to the hidden state of the encoder.

[0024] Preferably, the method for obtaining the description text applied to the traditional village cultural tourism video scene includes:

[0025] Obtain village feature information from structured data with feature annotations and construct village feature vectors;

[0026] Obtain prior knowledge of the target audience’s reading preferences and construct an audience preference vector;

[0027] Concatenating the village feature vector with the audience preference vector to obtain a fusion vector;

[0028] The fusion vector is encoded and decoded using a sequence-to-sequence text generation model to obtain a description text applied to a traditional village cultural tourism video scene.

[0029] The present invention also provides a system for generating a description text of a scene of a traditional village cultural tourism video, which is used to implement the method, and includes:

[0030] Video acquisition module, used to acquire and pre-process village videos;

[0031] A multi-target detection module is used to establish a multi-target detection model, identify and locate targets in the pre-processed village video, obtain key video information, and construct a structured video content representation based on the key video information;

[0032] A feature annotation module is used to construct a village knowledge base based on the village feature knowledge graph, and semantically associate the video content representation with the village knowledge base to obtain structured data with feature annotations;

[0033] The description text generation module is used to build a sequence-to-sequence text generation model based on the attention mechanism, input the structured data with characteristic annotations into the sequence-to-sequence text generation model, and obtain the description text applied to the traditional village cultural tourism video scene.

[0034] Preferably, the video acquisition module includes:

[0035] An image enhancement unit, used to obtain a video frame sequence of the village video, and adjust the brightness and contrast of each frame of the image to obtain an enhanced image frame sequence;

[0036] An image denoising unit, used for denoising the enhanced image frame sequence by using wavelet transform to obtain a denoised image frame sequence;

[0037] A frame interpolation unit, used for inserting new interpolation frames between adjacent frames of the denoised image frame sequence based on motion estimation and motion compensation technology to obtain a video frame sequence after interpolation;

[0038] The resolution processing unit is used to process the video frame sequence after frame complementation by using a video super-resolution algorithm based on deep learning, obtain a video sequence that meets a preset resolution, and complete the preprocessing of the village video.

[0039] Compared with the prior art, the beneficial effects of the present invention are: 1. Improving cultural inheritance and protection. By semantically associating village videos with village-specific knowledge graphs, the generated description text can more accurately convey the cultural characteristics and historical background of the village, which helps to enhance the public's cognition and interest in traditional culture, thereby promoting cultural inheritance and protection. 2. Promoting the development of the cultural tourism industry. The generated description text can provide rich content materials for the cultural tourism industry, help create tourism products with local characteristics, attract tourists to experience, and promote the development of the local economy. 3. Enhance user experience. The structured video content representation and the text generation model based on the attention mechanism can ensure that the generated description text is both accurate and vivid, can attract the user's attention and enhance the user experience, and enhance the user's interest and participation in the village cultural tourism video. 4. Improving the efficiency and accuracy of information processing. Through the combination of the multi-target detection model and the sequence-to-sequence text generation model, the automatic processing and information extraction of village videos are realized, which greatly improves the efficiency and accuracy of information processing and reduces the cost and time of manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 The present invention is a flowchart of a method for generating text descriptions of traditional village cultural tourism video scenes according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Embodiment 1

[0045] like Figure 1 As shown, a method for generating a description text of a scene in a traditional village cultural tourism video includes the following steps:

[0046] S1: Collecting and preprocessing village videos; In a further implementation, the method for preprocessing village videos includes:

[0047] S11: obtaining a video frame sequence of the village video, and adjusting the brightness and contrast of each frame image to obtain an enhanced image frame sequence;

[0048] S12: De-noising the enhanced image frame sequence by using wavelet transform to obtain a de-noised image frame sequence;

[0049] S13: Based on motion estimation and motion compensation technology, insert new interpolation frames between adjacent frames of the denoised image frame sequence to obtain a video frame sequence after interpolation; specifically, use the optical flow method of OpenCV for motion estimation. Calculate the motion vector between adjacent frames based on the optical flow field. Use the motion vector to perform motion compensation and interpolate to generate new frames. Specifically, use optical flow interpolation to generate interpolation frames.

[0050] S14: Use a video super-resolution algorithm based on deep learning to process the video frame sequence after frame complementation, obtain a video sequence that meets the preset resolution, and complete the preprocessing of the village video. In this embodiment, the video super-resolution algorithm based on deep learning includes a video frame degradation model and an upsampling layer. The video frame degradation model is established based on an image degradation function and various degradation factors (noise, downsampling factor, etc.); the upsampling layer uses sub-pixel convolution.

[0051] Video super-resolution algorithms can improve the spatial resolution of videos and increase the details of images without enlarging them. Super-resolution algorithms based on deep learning can learn the mapping relationship between low-resolution images and high-resolution images. For example, a video with a resolution of 720p can be upgraded to 1080p or even 4K resolution through super-resolution algorithms. This makes objects in the video clearer and the edges sharper, improving the viewing experience. Taking a village video as an example, the super-resolution algorithm can make details such as the tiles of the house and the texture of the trees more clearly visible.

[0052] S2: Establish a multi-target detection model to identify and locate targets in the preprocessed village video, obtain key video information, and construct a structured video content representation based on the key video information;

[0053] A further implementation method is that the network structure of the multi-target detection model includes a feature extraction network CSPDarkNet53, a path aggregation network PANet module and a spatial pyramid pooling SPP module.

[0054] Specifically, this embodiment adopts a lightweight improved model TD-YOLOv4 that integrates residual self-attention. The model is mainly composed of a feature extraction network CSPDarkNet53, a path aggregation network PANet module, and a spatial pyramid pooling SPP module. The feature extraction network CSPDarkNet53 integrates a residual self-attention module to enhance the ability to obtain global information while extracting local features. The PANet module introduces deep separable convolution, which simplifies the network while strengthening feature fusion, which is conducive to the extraction of diversified features; the SPP module uses maximum pooling of different scales to process features to increase the feature receptive field and enrich the feature map expression capability.

[0055] In this embodiment, the residual self-attention module first performs a downsampling operation on the input feature map (an image in a video sequence that meets a preset resolution) to reduce the size of the feature map. Subsequently, the module divides the input path into two branches: a main branch and a short connection branch. In the main branch, the feature map is processed by multiple residual structures consisting of 1×1 convolutions and self-attention modules. These structures are responsible for processing and aggregating contextual information in the input feature map to generate a main branch output that incorporates global information, in order to improve the expressiveness of the features. Finally, the output of the main branch is passed through a 1×1 convolution that eliminates the possible overlapping effects that may occur when features are fused, and then spliced ​​with the feature map of the short connection branch to obtain the final output feature.

[0056] The depthwise separable convolution performs a deep convolution on the input feature map with M channels and the corresponding M convolution kernels, and then uses N separable convolutions to perform point-by-point convolution on the feature map obtained by the deep convolution. After using the depthwise separable convolution, the amount of calculation and the number of parameters are greatly reduced, which effectively improves the computing speed of the feature extraction network and saves computing costs.

[0057] A spatial pyramid pooling (SPP) layer is added before the path aggregation network PANet. The SPP structure uses the maximum pooling of four different scales to process the input feature map, and finally connects the pooled features in a concat manner, ensuring that input images of different sizes are output at the same size, thereby retaining more feature information of the image, thereby improving the detection accuracy of the network when there are different sizes of the same data set.

[0058] Specifically, a multi-target detection model is used to obtain each frame of the preprocessed video. According to each frame of the image, the deep learning target detection model is used for reasoning to obtain the target detection result, including the object category, confidence and bounding box coordinates. If the confidence is lower than the preset threshold, the detection result is discarded. According to the target detection result, the object category information appearing in the video is extracted. The position of the object in the video frame is located through the bounding box coordinate information. According to the identified object category, the human target is judged. If the human category is detected, the feature information related to the person is extracted. In a frame of rural video image, the model detects a dog with a confidence of 0.95 and a bounding box coordinate of (100, 50, 200, 150). This confidence is higher than the preset threshold, so this result is retained. If a "human" category is detected at the same time, the confidence is only 0.3, which is lower than the threshold, and it is discarded. Through target detection, the object categories appearing in the video can be extracted, such as "human", "dog", "house", "tree", etc. Combined with the bounding box coordinates, it is known that the dog is located in the upper left area of ​​the image. If the “person” category is detected, further person features can be extracted, such as whether the person is wearing red clothes or whether the person is standing.

[0059] In particular, in order to construct a more comprehensive structured video content representation, this embodiment also uses a scene recognition model to perform scene recognition on the preprocessed village video. According to the scene category information and the detected object category information, a video content representation is constructed. Among them, a large amount of village video data is collected to ensure that the data covers different scene categories, such as natural landscapes, village buildings, cultural activities, etc. The video data is labeled, and one or more scene category labels are assigned to each frame or each video segment. The video data is cleaned to remove repeated, blurred or irrelevant video frames. The video data is divided into continuous frames for subsequent feature extraction and model training. The frames are normalized, denoised, etc. to improve the accuracy of feature extraction. SIFT (Scale-Invariant Feature Transform) is used to extract image features in video frames, such as color, texture, shape, etc. Motion features in the video, such as optical flow, trajectory, etc., are extracted to capture dynamic information in the video. A convolutional neural network (CNN) is used to extract high-level features of video frames.

[0060] S3: Based on the village-specific knowledge graph, a village knowledge base is constructed, and the video content representation is semantically associated with the village knowledge base to obtain structured data with characteristic annotations; the village-specific knowledge graph includes village-related entities, relationships, and attribute information.

[0061] A further implementation method is that the method for obtaining structured data with characteristic annotations includes:

[0062] Using entity linking technology, the video content representation is semantically associated with entities in the village knowledge base; the video content representation includes scene, character and activity information;

[0063] Extract relations and map attributes between video content representation and entities in the village database;

[0064] Based on the results of semantic association, relationship extraction and attribute mapping, feature annotations are established to obtain structured data with feature annotations.

[0065] S4: Construct a sequence-to-sequence text generation model based on the attention mechanism, input the structured data with characteristic annotations into the sequence-to-sequence text generation model, and obtain the descriptive text applied to the traditional village cultural tourism video scene.

[0066] A further implementation method is that the network structure of the sequence-to-sequence text generation model based on the attention mechanism includes an encoder, a decoder and an attention mechanism module; wherein,

[0067] The encoder uses a gated recurrent unit to process the input sequence word by word and obtain a hidden state sequence as an intermediate representation; specifically, the encoder part uses a gated recurrent unit (GRU) as its basic structure. GRU is a special recurrent neural network (RNN) variant that controls the flow of information by introducing update gates and reset gates, thereby alleviating the long-term dependency problem of RNN to a certain extent.

[0068] The decoder uses a recurrent neural network to generate an output sequence word by word, taking the received hidden state sequence as the initial state. The generation of each word is based on the output of the previous word and the current hidden state. To achieve this, the decoder contains an output layer that uses a softmax function to calculate the probability distribution of each word in the vocabulary. Then, based on this probability distribution, the decoder selects the word with the highest probability as the output of the current word.

[0069] In the attention module, the query vector corresponds to the hidden state of the decoder, and the key vector and value vector correspond to the hidden state of the encoder. Specifically, in order to calculate the attention weight, the model calculates the similarity score between the query vector and each key vector. These scores are then normalized to a probability distribution (using a softmax function) to obtain the weight of each value vector. In this embodiment, the hidden state of the encoder is weighted and summed according to the attention weight. The output of the attention mechanism is added to the current hidden state of the decoder to obtain an enhanced hidden state. The enhanced hidden state is used to calculate the probability distribution of each word in the vocabulary through the output layer. The word with the highest probability is selected as the output of the current word according to the probability distribution.

[0070] A further implementation method is that the method for obtaining a description text applied to a traditional village cultural tourism video scene includes:

[0071] Obtain village feature information from structured data with feature annotations and construct village feature vectors;

[0072] Obtain prior knowledge of the target audience’s reading preferences and construct an audience preference vector;

[0073] The village feature vector is concatenated with the audience preference vector to obtain a fusion vector;

[0074] The fused vector is encoded and decoded using a sequence-to-sequence text generation model to obtain descriptive text applied to traditional village cultural tourism video scenes.

[0075] In this embodiment, structured data with village characteristics annotations are input into a sequence-to-sequence text generation model with an attention mechanism, and prior knowledge of the target audience's reading preferences is incorporated to automatically generate descriptive text that highlights the village's characteristic themes and highlights, and optimize the text's language style and semantic coherence.

[0076] Get village characteristic information from structured data, that is, extract geographical location (such as "Lijiang, Yunnan"), historical culture (such as "Naxi Dongba culture"), folk characteristics (such as "March Street") and other information from the video structured data with village characteristic annotations generated in the previous step. Then, quantify this information and construct a village feature vector. Specifically, a vector can be used to represent different characteristics of a village. Each dimension of the vector represents a feature, and its value represents the prominence of the feature. For example, the prominence of "Naxi Dongba culture" can be calculated based on factors such as the length of the footage about Dongba culture in the video and the frequency of commentary. Get the prior knowledge of the target audience's reading preferences, such as analyzing the user's historical browsing records, likes, comments and other behavioral data to understand the user's interest in different types of villages and different cultural characteristics. Then, quantify this preference information and construct an audience preference vector. Specifically, if the user often browses videos about traditional handicrafts, the value of the dimension "handicrafts" in the preference vector will be higher. Concatenate the village feature vector and the audience preference vector to obtain a fusion vector. Specifically, the feature vector representing the village "Lijiang, Yunnan" and the preference vector representing a certain user are concatenated together to form a new vector. This allows the village features and user preference information to be integrated into one vector so that the subsequent model can better understand the user's needs and generate a description text that is more in line with the user's interests. The fused vector is input into the pre-trained sequence-to-sequence attention mechanism text generation model. The sequence-to-sequence model is suitable for text generation tasks and can convert the input vector sequence into the output text sequence. The attention mechanism allows the model to focus on the part of the input vector that is most relevant to the current generated word when generating text, thereby improving the quality of the generated text. For example, when the model generates the word "Dongba culture", the attention mechanism allows the model to pay more attention to the part of the input vector that is related to "Dongba culture", such as "Naxi", "hieroglyphics", etc. The text generation model decodes the fused vector and generates a description text.

[0077] The decoding process is the process of converting the fusion vector into a text sequence. The model will generate words in the description text one by one based on the information contained in the fusion vector. Specifically, if the fusion vector contains information such as "Lijiang, Yunnan", "Naxi people", and "Dongba culture", the model may generate a description text such as "Lijiang is an ancient city full of Naxi customs, where precious Dongba culture is preserved". The generated description text is optimized for language style and semantic coherence.

[0078] Grammatical error correction and semantic disambiguation techniques can be used to polish the generated text to make it more fluent and easy to understand. For example, change "Lijiang is an ancient city full of Naxi customs" to "Lijiang, an ancient city full of Naxi customs". This can make the text more concise and attractive. If the perplexity of the text is higher than the preset threshold, the text is regenerated. Perplexity is an indicator to measure the quality of the text generation model. The higher the perplexity, the more unnatural and difficult to understand the text generated by the model. Setting a preset threshold can ensure that the quality of the generated text meets a certain standard. For example, if the perplexity of the generated text "Lijiang is a city of both Naxi and Dongba" is higher than the preset threshold, the text needs to be regenerated. Finally, a descriptive text that highlights the characteristics and highlights of the village is obtained. For example, for the village "Lijiang, Yunnan", the final description text may be: "Lijiang, a thousand-year-old city guarded by the Jade Dragon Snow Mountain, where the Naxi people have lived for generations and inherited the unique Dongba culture. Strolling on the bluestone road in the ancient city, you can feel the charm of ancient civilization, listen to the melodious melody of Naxi ancient music, and experience the unique folk customs. Come to Lijiang and start an unforgettable cultural journey." Such a description text highlights Lijiang's geographical location (Jade Dragon Snow Mountain), historical culture (Naxi people, Dongba culture) and folk characteristics, which can attract the attention of the target audience.

[0079] In this embodiment, an optimization process of a sequence-to-sequence text generation model based on an attention mechanism is also included, as follows:

[0080] The user feedback data on the generated text is collected through the user interaction interface, and it is used as a reward signal for reinforcement learning. The parameters of the text generation model are dynamically adjusted so that it can continuously learn and adapt to user preferences, and generate village feature description text that is closer to user needs. According to the village-related information and the pre-established text feature vector, the reward model is trained using a supervised learning algorithm to obtain a predicted reward value. According to the reward value, user feedback and village-related information, the reinforcement learning environment is constructed. The text generation model is trained using a reinforcement learning algorithm. The text generation model is based on the reward value output by the reward model, and the parameters of the text generation model are adjusted to obtain the optimized text generation model.

[0081] Among them, the training goal of the reward model is to predict the quality of the text. It takes village-related information and pre-established text feature vectors as input and uses a supervised learning algorithm for training. For example, some high-quality village description texts can be collected and their quality scores can be manually annotated (for example, 1-5 points). Then, these texts and their corresponding scores are used as training data to train a regression model. Village-related information includes the village's geographical location, population, historical culture, characteristic industries, etc. The text feature vector includes text length, sentiment tendency, keywords, etc. Through training, the reward model can learn which features are associated with high-quality text, thereby predicting the quality of newly generated text.

[0082] The construction of the reinforcement learning environment requires reward values, user feedback, and village-related information. The reward value is predicted by the reward model. User feedback can be user ratings, comments, or other forms of feedback on the generated text. The village-related information is the same as the input to the reward model. Together, this information constitutes the state space of the reinforcement learning environment. The goal of reinforcement learning is to train a text generation model that can generate text that gets the highest reward based on the current state.

[0083] In this embodiment, it also includes continuously collecting new village video data, and using incremental learning to update the video recognition and feature mining models. At the same time, the active learning algorithm is used to identify video clips with low confidence, and user feedback is obtained through the crowdsourcing platform, which is used as a new training sample to iteratively optimize the model and continuously improve the system's village video understanding and description generation capabilities.

[0084] Specifically, an incremental learning algorithm based on feature playback is used to update the video recognition model to obtain an updated video recognition model. The recognition model includes a multi-target detection model and a scene recognition model. The multi-target detection model is used to recognize objects in the video, such as houses, trees, roads, etc. The scene recognition model is used to recognize scenes in the video, such as farmland, rivers, village squares, etc.

[0085] Embodiment 2

[0086] The present invention also provides a system for generating a description text of a scene in a traditional village cultural tourism video, which is used to implement a method, including:

[0087] Video acquisition module, used to acquire and pre-process village videos;

[0088] The multi-target detection module is used to establish a multi-target detection model, identify and locate targets in the pre-processed village video, obtain key video information, and construct a structured video content representation based on the key video information;

[0089] The feature annotation module is used to build a village knowledge base based on the village feature knowledge graph, and semantically associate the video content representation with the village knowledge base to obtain structured data with feature annotations;

[0090] The description text generation module is used to build a sequence-to-sequence text generation model based on the attention mechanism, input structured data with characteristic annotations into the sequence-to-sequence text generation model, and obtain the description text applied to the traditional village cultural tourism video scene.

[0091] A further implementation is that the video acquisition module includes:

[0092] An image enhancement unit, used to obtain a video frame sequence of the village video, and adjust the brightness and contrast of each frame of the image to obtain an enhanced image frame sequence;

[0093] An image denoising unit, used for denoising the enhanced image frame sequence by using wavelet transform to obtain a denoised image frame sequence;

[0094] A frame interpolation unit, used for inserting new interpolation frames between adjacent frames of the denoised image frame sequence based on motion estimation and motion compensation technology to obtain a video frame sequence after interpolation;

[0095] The resolution processing unit is used to process the video frame sequence after frame complementation by using a video super-resolution algorithm based on deep learning, obtain a video sequence that meets a preset resolution, and complete the preprocessing of the village video.

[0096] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for generating a description text of a scene in a traditional village cultural tourism video, characterized in that: The following steps are involved: Collect and pre-process village videos; Establish a multi-target detection model to identify and locate targets in the preprocessed village video, obtain key video information, and construct a structured video content representation based on the key video information; Based on the village characteristic knowledge graph, a village knowledge base is constructed, and the video content representation is semantically associated with the village knowledge base to obtain structured data with characteristic annotations; A sequence-to-sequence text generation model based on the attention mechanism is constructed, and structured data with characteristic annotations are input into the sequence-to-sequence text generation model to obtain descriptive text applied to traditional village cultural tourism video scenes.

2. The method for generating scene description text for traditional village cultural tourism video according to claim 1 is characterized in that: The method for preprocessing the village video includes: Acquire a video frame sequence of the village video, and adjust the brightness and contrast of each frame image to obtain an enhanced image frame sequence; Using wavelet transform to denoise the enhanced image frame sequence to obtain a denoised image frame sequence; Based on motion estimation and motion compensation technology, new interpolation frames are inserted between adjacent frames of the denoised image frame sequence to obtain a video frame sequence after interpolation. A video super-resolution algorithm based on deep learning is used to process the video frame sequence after frame complementation to obtain a video sequence that meets a preset resolution, thereby completing the preprocessing of the village video.

3. The method for generating scene description text for traditional village cultural tourism video according to claim 1 is characterized in that: The network structure of the multi-target detection model includes a feature extraction network CSPDarkNet53, a path aggregation network PANet module and a spatial pyramid pooling SPP module.

4. The method for generating scene description text for traditional village cultural tourism video according to claim 1 is characterized in that: Methods for obtaining structured data with featured annotations include: Using entity linking technology, semantically associating the video content representation with entities in the village knowledge base; wherein the video content representation includes scene, character and activity information; Extracting relationships and mapping attributes between the video content representation and entities in the village database; Based on the results of the semantic association, the relationship extraction and the attribute mapping, characteristic annotations are established to obtain structured data with characteristic annotations.

5. The method for generating scene description text for traditional village cultural tourism video according to claim 1, characterized in that: The network structure of the sequence-to-sequence text generation model based on the attention mechanism includes an encoder, a decoder and an attention mechanism module; wherein, The encoder uses a gated recurrent unit to process the input sequence word by word to obtain a hidden state sequence as an intermediate representation; The decoder uses a recurrent neural network to generate an output sequence word by word using the received hidden state sequence as an initial state, wherein the generation of each word is based on the output of the previous word and the current hidden state; In the attention module, the query vector corresponds to the hidden state of the decoder, and the key vector and value vector correspond to the hidden state of the encoder.

6. The method for generating scene description text for traditional village cultural tourism video according to claim 5, characterized in that: The method for obtaining the description text applied to the traditional village cultural tourism video scene includes: Obtain village feature information from structured data with feature annotations and construct village feature vectors; Obtain prior knowledge of the target audience’s reading preferences and construct an audience preference vector; Concatenating the village feature vector with the audience preference vector to obtain a fusion vector; The fusion vector is encoded and decoded using a sequence-to-sequence text generation model to obtain a description text applied to a traditional village cultural tourism video scene.

7. A system for generating description text of a traditional village cultural tourism video scene, used to implement the method described in any one of claims 1 to 6, characterized in that: include: Video acquisition module, used to acquire and pre-process village videos; A multi-target detection module is used to establish a multi-target detection model, identify and locate targets in the pre-processed village video, obtain key video information, and construct a structured video content representation based on the key video information; A feature annotation module is used to construct a village knowledge base based on the village feature knowledge graph, and semantically associate the video content representation with the village knowledge base to obtain structured data with feature annotations; The description text generation module is used to build a sequence-to-sequence text generation model based on the attention mechanism, input the structured data with characteristic annotations into the sequence-to-sequence text generation model, and obtain the description text applied to the traditional village cultural tourism video scene.

8. The system for generating scene description text for traditional village cultural tourism video according to claim 7 is characterized in that: The video acquisition module comprises: An image enhancement unit, used to obtain a video frame sequence of the village video, and adjust the brightness and contrast of each frame of the image to obtain an enhanced image frame sequence; An image denoising unit, used for denoising the enhanced image frame sequence by using wavelet transform to obtain a denoised image frame sequence; A frame interpolation unit, used for inserting new interpolation frames between adjacent frames of the denoised image frame sequence based on motion estimation and motion compensation technology to obtain a video frame sequence after interpolation; The resolution processing unit is used to process the video frame sequence after frame complementation by using a video super-resolution algorithm based on deep learning, obtain a video sequence that meets a preset resolution, and complete the preprocessing of the village video.

Images