Video image description text generation method and device and storage medium

Abstract

The invention relates to the technical field of image processing, in particular to a video image description text generation method and device and a storage medium, and the method comprises the steps: carrying out the frame extraction processing and cutting processing of a to-be-described video, and obtaining at least one frame of to-be-described image and at least one to-be-described sub-video; performing face detection processing on the to-be-described image and the to-be-described sub-video to obtain a face detection result; performing picture description processing on the to-be-described image to obtain a first image description text, and performing video description processing on the to-be-described sub-video to obtain a first video description text; for the target to-be-described image, determining the to-be-described sub-video containing the target to-be-described image as a target to-be-described sub-video; and according to a face detection result, fusing the first video description text of the target to-be-described sub-video into the first image description text of the target to-be-described image to obtain an image description text of the target to-be-described image. According to the embodiment of the invention, the accuracy of generating the video image description text can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, and storage medium for generating video image description text. Background Art

[0002] Text descriptions of video images help users quickly access key video information and improve their understanding. Currently, video image descriptions are typically generated by first performing content recognition on individual video frames within a video, then directly aggregating the recognition results of the individual frames into a corresponding video description. This method of generating video image descriptions fails to consider the correlation between the frames of the complete video, resulting in poor coherence and inaccuracy in the generated video image descriptions. Summary of the Invention

[0003] In order to solve the above technical problems, the present disclosure provides a method, device and storage medium for generating video image description text, which can improve the accuracy of generating video image description text.

[0004] In a first aspect, the present disclosure provides a method for generating video image description text, comprising: performing frame extraction and segmentation processing on a video to be described to obtain at least one frame of an image to be described and at least one sub-video to be described; wherein one sub-video to be described includes at least one frame of an image to be described; performing face detection processing on both the image to be described and the sub-video to be described to obtain a face detection result; performing picture description processing on the image to be described to obtain a first image description text for each frame of the image to be described, and performing video description processing on the sub-video to be described to obtain a first video description text for each sub-video to be described; for a target image to be described in at least one frame of the image to be described, determining the sub-video to be described containing the target image to be described as the target sub-video to be described; the target image to be described is any image to be described in at least one frame of the image to be described; and according to the face detection results of the target image to be described and the target sub-video to be described, fusing the first video description text of the target sub-video to be described into the first image description text of the target image to be described to obtain the image description text of the target image to be described.

[0005] In some embodiments, the face detection result includes a person identifier; the first image description text includes at least one image description subtext corresponding to the person identifier, and the first video description text includes at least one video description subtext corresponding to the person identifier; based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is merged into the first image description text of the target image to be described to obtain the image description text of the target image to be described, including: determining the person identifier that exists in the person detection results of both the target sub-video to be described and the target image to be described as a common identifier; determining the video description subtext corresponding to the common identifier in the first video description text of the target sub-video to be described as the common video description text, and determining the image description subtext corresponding to the common identifier in the first image description text of the target image to be described as the common image description text; merging the common video description text into the common image description text to obtain a fused image description text corresponding to the common identifier; and jointly determining the fused image description text and the image description subtext in the target image to be described that is not fused with the common video description text as the image description text of the target image to be described.

[0006] In some embodiments, image description processing is performed on the image to be described to obtain the first image description text of each frame of the image to be described, and video description processing is performed on the sub-video to be described to obtain the first video description text of each sub-video to be described, including: image description processing is performed on the image to be described according to the face detection result to obtain the first image description text of each frame of the image to be described; the first image description text includes at least one image description sub-text corresponding to a character identifier; video description processing is performed on the sub-video to be described according to the face detection result to obtain the first video description text of each sub-video to be described; the first video description text includes at least one video description sub-text corresponding to a character identifier.

[0007] In some embodiments, the shared video description text is fused into the shared image description text to obtain a fused image description text corresponding to the shared identifier, including: deduplicating and splicing the shared video description text and the shared image description text to obtain a fused image description text corresponding to the shared identifier; or, performing keyword extraction on the shared video description text to obtain a first keyword, and performing keyword extraction on the shared image description text to obtain a second keyword; splicing the first keyword and the second keyword, and performing semantic expansion on the spliced ​​first keyword and second keyword to obtain a fused image description text corresponding to the shared identifier; or, using a preset semantic fusion model to fuse the shared video description text and the shared image description text to obtain a fused image description text corresponding to the shared identifier.

[0008] In some embodiments, based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is merged into the first image description text of the target image to be described to obtain the image description text of the target image to be described, including: constructing prompt words based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described, and using the prompt words to generate the image description text of the target image to be described through a preset generation model.

[0009] In some embodiments, when the face detection result includes person identifiers and person positions, the first image description text, the first video description text, and the image description text of the target image to be described also include the person position corresponding to each person identifier.

[0010] In some embodiments, the video to be described is subjected to frame extraction and segmentation processing, including: performing frame extraction processing on the video to be described according to a first time interval to obtain at least one frame of image to be described; performing segmentation processing on the video to be described according to a second time interval to obtain at least one sub-video to be described; the second time interval is greater than the first time interval.

[0011] In a second aspect, the present disclosure provides a device for generating video image description text, comprising: a video processing module, configured to perform frame extraction and segmentation processing on a video to be described to obtain at least one frame of an image to be described and at least one sub-video to be described; wherein a sub-video to be described includes at least one frame of an image to be described; an image processing module, configured to perform face detection processing on both the image to be described and the sub-video to be described to obtain a face detection result; perform picture description processing on the image to be described to obtain a first image description text for each frame of the image to be described, and perform video description processing on the sub-video to be described to obtain a first video description text for each sub-video to be described; a video determination module, configured to determine, for a target image to be described in at least one frame of an image to be described, a sub-video to be described containing a target image to be described as a target sub-video to be described; the target image to be described is any image to be described in at least one frame of an image to be described; and a text generation module, configured to fuse the first video description text of the target sub-video to be described into the first image description text of the target image to be described based on the face detection results of the target image to be described and the target sub-video to be described to obtain an image description text of the target image to be described.

[0012] In some embodiments, the face detection result includes a person identifier; the first image description text includes at least one image description sub-text corresponding to the person identifier, and the first video description text includes at least one video description sub-text corresponding to the person identifier; the text generation module is specifically used to: determine the person identifier that exists in the person detection results of the target sub-video to be described and the target image to be described as a common identifier; determine the video description sub-text corresponding to the common identifier in the first video description text of the target sub-video to be described as the common video description text, and determine the image description sub-text corresponding to the common identifier in the first image description text of the target image to be described as the common image description text; fuse the common video description text into the common image description text to obtain a fused image description text corresponding to the common identifier; and jointly determine the fused image description text and the image description sub-text in the target image to be described that is not fused with the common video description text as the image description text of the target image to be described.

[0013] In some embodiments, the image processing module is specifically used to: perform image description processing on the image to be described according to the face detection result, and obtain the first image description text of each frame of the image to be described; the first image description text includes at least one image description sub-text corresponding to a character identifier; perform video description processing on the sub-video to be described according to the face detection result, and obtain the first video description text of each sub-video to be described; the first video description text includes at least one video description sub-text corresponding to a character identifier.

[0014] In some embodiments, the text generation module is specifically used to: deduplicate and splice the shared video description text and the shared image description text to obtain a fused image description text corresponding to the shared identifier; or, extract keywords from the shared video description text to obtain a first keyword, and extract keywords from the shared image description text to obtain a second keyword; splice the first keyword and the second keyword, and perform semantic expansion on the spliced ​​first keyword and second keyword to obtain a fused image description text corresponding to the shared identifier; or, use a preset semantic fusion model to fuse the shared video description text and the shared image description text to obtain a fused image description text corresponding to the shared identifier.

[0015] In some embodiments, the text generation module is specifically used to: construct prompt words based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described, and use the prompt words to generate image description text of the target image to be described through a preset generation model.

[0016] In some embodiments, when the face detection result includes person identifiers and person positions, the first image description text, the first video description text, and the image description text of the target image to be described also include the person position corresponding to each person identifier.

[0017] In some embodiments, the video processing module is specifically used to: extract frames of the video to be described according to a first time interval to obtain at least one frame of image to be described; cut the video to be described according to a second time interval to obtain at least one sub-video to be described; the second time interval is greater than the first time interval.

[0018] In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, a method for generating a video image description text as in any embodiment of the first aspect is implemented.

[0019] In a fourth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, a method for generating video image description text as in any embodiment of the first aspect is implemented.

[0020] In a fifth aspect, the present disclosure provides a computer program product, comprising: when the computer program product is run on a computer, enabling the computer to implement the method for generating video image description text as in any embodiment of the first aspect.

[0021] The technical solution provided by the present disclosure has the following advantages compared with the existing technology:

[0022] First, the video to be described is subjected to frame extraction and segmentation processing to obtain at least one frame of image to be described and at least one sub-video to be described. A sub-video to be described includes at least one frame of image to be described. Then, face detection processing is performed on both the image to be described and the sub-video to be described to obtain face detection results. The image to be described is subjected to image description processing to obtain the first image description text for each frame of image to be described, and the sub-video to be described is subjected to video description processing to obtain the first video description text for each sub-video to be described. Then, for the target image to be described in the at least one frame of image to be described, the sub-video to be described containing the target image to be described is determined as the target sub-video to be described. The target image to be described is any image to be described in the at least one frame of image to be described. Finally, based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is merged into the first image description text of the target image to be described to obtain the image description text of the target image to be described. In this way, on the one hand, when generating the image description text of the target image to be described, the face detection results can be used to connect the target sub-video to be described and the individual characters in the target image to be described, and finally form the image description text of the target image to be described, thereby improving the coherence between the image description text of the target image to be described and the image description texts of other images to be described, thereby improving the accuracy of the generated video image description text. On the other hand, when generating the image description text of the target image to be described, the corresponding spatiotemporal information of the target image to be described in the target sub-video to be described is also taken into account, further enriching the content of the image description text, thereby improving the accuracy of the generated video image description text. In addition, the image description text combined with the face detection results can accurately distinguish different people in the target image to be described, avoiding identifying different people as the same person, thereby further improving the accuracy of the image description text. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 This is a schematic diagram of one of the application scenarios of the method for generating video image description text provided by an embodiment of the present disclosure;

[0026] Figure 2This is a flowchart of a method for generating a video image description text according to an embodiment of the present disclosure;

[0027] Figure 3 A second schematic diagram of an application scenario of the method for generating video image description text provided by an embodiment of the present disclosure;

[0028] Figure 4 The second flowchart of the method for generating a video image description text provided by the embodiment of the present disclosure;

[0029] Figure 5 The third flowchart of the method for generating a video image description text provided by the embodiment of the present disclosure;

[0030] Figure 6 A fourth flowchart of a method for generating a video image description text provided in an embodiment of the present disclosure;

[0031] Figure 7 A schematic diagram of the structure of a device for generating video image description text provided by an embodiment of the present disclosure;

[0032] Figure 8 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0035] Figure 1 Schematic diagram of the scene architecture of the method for generating video image description text provided by the embodiment of the present disclosure. Figure 1 As shown, the scenario architecture provided by the embodiment of the present disclosure includes: a server 100 and an electronic device 200.

[0036] The electronic device 200 provided in the embodiment of the present disclosure may have various implementation forms, for example, it may be a mobile phone, a personal computer (PC), a smart TV, a laser projection device, a TV, a monitor, a wearable device, etc.

[0037] In some embodiments, upon receiving the instruction to generate the video image description text, the electronic device 200 may communicate data with the server 100. The electronic device 200 may be allowed to communicate with the server 100 via a local area network (LAN) or a wireless local area network (WLAN).

[0038] The server 100 can be a server that provides various services, such as providing support for the video to be described obtained by the electronic device 200. The server can perform frame extraction, segmentation, and other processing on the received video to be described, and feed the processing results back to the electronic device 200. The server 100 can be a server cluster or multiple server clusters, and can include one or more types of servers.

[0039] It should be noted that the method for generating video image description text provided in the embodiment of the present disclosure can be executed by the server 100, or by the electronic device 200, or by both the server 100 and the electronic device 200, and the present disclosure does not limit this.

[0040] The method for generating the video image description text provided by the embodiment of the present disclosure can be executed by a device for generating the video image description text, and the device for generating the video image description text can be hardware or software. When the device for generating the video image description text is hardware, it can be various electronic devices with a function of generating the video image description text, including but not limited to mobile phones, computers, computers, tablet computers, televisions, smart TVs, laser projection equipment, monitors, etc. When the device for generating the video image description text is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules, or it can be implemented as a single software or software module. It is not specifically limited here.

[0041] Figure 2 A flowchart of a method for generating a video image description text according to an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the method for generating video image description text may include the following steps:

[0042] S11 , performing frame extraction and segmentation processing on the video to be described to obtain at least one frame of image to be described and at least one sub-video to be described.

[0043] The video to be described may be any type of video, such as cartoons, TV dramas, documentaries, news, variety shows, etc. A sub-video to be described includes at least one frame of image to be described.

[0044] First, the video to be described is subjected to frame extraction to obtain at least one frame of the image to be described. Specifically, frame extraction can be performed in a fixed interval manner, i.e., the video to be described is subjected to frame extraction at a first time interval to obtain at least one frame of the image to be described. Alternatively, frame extraction can be performed in a proportional manner, i.e., the number of frames is reduced according to a preset frame rate, for example, a video with a frame rate of 30 frames per second (fps) is reduced to 15 fps to obtain at least one frame of the image to be described. Alternatively, frame extraction can be performed in a time-balanced manner, i.e., video frames are evenly extracted at different time periods to obtain at least one frame of the image to be described.

[0045] Next, the video to be described is segmented to obtain at least one sub-video to be described. Specifically, the video to be described can be segmented at fixed time intervals, i.e., the video to be described is segmented at second time intervals to obtain at least one sub-video to be described. Alternatively, the video to be described can be segmented at fixed frame counts, i.e., the video to be described is segmented at a preset frame count, for example, every 300 frames to obtain at least one sub-video to be described.

[0046] The first time interval and the second time interval are both preset, and the second time interval is greater than the first time interval. For example, the first time interval is 1 second (s), and the second time interval is 10 seconds. In this way, during the frame extraction and segmentation processing, the time interval for segmenting the video is greater than the time interval for extracting the frames of the video, so as to ensure that each sub-video to be described includes at least one frame of the image to be described, thereby avoiding the situation where the time interval for extracting the frames is too long, which affects the generation result of the image description text, or even leads to useless work and waste of computing resources.

[0047] S12: Perform face detection processing on both the image to be described and the sub-video to be described to obtain a face detection result.

[0048] Specifically, a target detection algorithm can be used to perform face detection processing on both the image to be described and the sub-video to be described to obtain a face detection result. For example, Faster Region-based Convolutional Neural Network (Faster R-CNN), YOLO (You Only Look Once), Single Shot MultiBox Detector (SSD), RetinaNet, DETR (Detection Transformer), etc. Among them, the target detection algorithm can be a single detection algorithm or a combination of multiple detection algorithms. A pre-trained face detection model can also be used to perform face detection processing on the image to be described and the sub-video to be described to obtain a face detection result.

[0049] In some embodiments, the face detection results include a person identifier and a person position. The person identifier is an identifier that uniquely identifies the person, such as the person's name or serial number. The person position can be the coordinates of the upper left and lower right corners of the face detection frame. The person's position information must be accurate to milliseconds.

[0050] S13 , performing picture description processing on the image to be described to obtain a first image description text for each frame of the image to be described.

[0051] Specifically, a preset tool can be used to perform image description processing on the image to be described to obtain the first image description text for each frame of the image to be described. For example, Minicpmv 2.6, Contrastive Language-Image Pretraining (CLIP), Bootstrapping Language-Image Pretraining (BLIP), etc. It is also possible to use a method based on traditional image processing to perform image description processing on the image to be described, that is, to use edge detection, texture analysis, color histogram and other technologies to extract the features of the image to be described, and then combine rules or templates to generate the first image description text for each frame of the image to be described. It is also possible to use a method based on a visual-language model to perform image description processing on the image to be described, that is, to use a pre-trained visual-language model to jointly learn the representation of image and text to generate the first image description text of the image to be described.

[0052] S14: Perform video description processing on the sub-video to be described to obtain a first video description text for each sub-video to be described.

[0053] Specifically, a preset tool can be used to perform video description processing on the sub-video to be described, and the first video description text of each sub-video to be described can be obtained. For example, the Spatio-Temporal Large Language Model (St-llm), Flamingo, etc. A method based on a visual-language pre-training model can also be used to perform video description processing on the sub-video to be described, and the first video description text of each sub-video to be described can be obtained. A method based on multimodal fusion can also be used to perform video description processing on the sub-video to be described, that is, combining multiple modal information such as vision, audio, speech, and text to perform video description processing on the sub-video to be described. A multimodal pre-training model such as UniVL can synthesize different signals to generate a more complete video description, thereby obtaining the first video description text of each sub-video to be described.

[0054] S15 . For a target image to be described in at least one frame of images to be described, determine a sub-video to be described that includes the target image to be described as the target sub-video to be described.

[0055] The target image to be described is any image to be described in at least one frame of images to be described.

[0056] Specifically, for any image to be described, the image to be described is determined as a target image to be described, and the sub-video to be described including the image to be described is determined as a target sub-video to be described.

[0057] For example, Figure 3 A schematic diagram of a scenario of a method for generating a video image description text provided by an embodiment of the present disclosure. Figure 3 As shown, the duration of the video V to be described is 20 seconds, the first preset time interval is 1 second, and the second preset time interval is 10 seconds. Then, 20 frames of images to be described are extracted from the video to be described. Figure 3 The images 1-20 to be described in the video to be described are divided into two sub-videos to be described, namely Figure 3 The sub-video A to be described and the sub-video B to be described in .

[0058] In this way, when any one of the images to be described 1 to 10 is determined as the target image to be described, the sub-video to be described A is determined as the target sub-video to be described; when any one of the images to be described 11 to 20 is determined as the target image to be described, the sub-video to be described B is determined as the target sub-video to be described.

[0059] S16. According to the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is merged into the first image description text of the target image to be described to obtain the image description text of the target image to be described.

[0060] Specifically, according to the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is merged into the first image description text of the target image to be described to obtain the image description text of the target image to be described. The method can be to directly splice the first video description text of the target sub-video to be described with the first image description text of the target image to be described based on the face detection results to obtain the image description text of the image to be described; of course, the splicing can also include deduplication, reduction, supplementation and other operations, and the present disclosure does not limit the specific splicing method.

[0061] In some embodiments, based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is integrated into the first image description text of the target image to be described to obtain the image description text of the target image to be described. Alternatively, prompt words are first constructed based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described. The prompt words are then used to generate the image description text of the target image to be described using a preset generation model. The prompt words are constructed based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described. The prompt words can be generated based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described, the first image description text of the target image to be described, and a preset template. The preset template can be a pre-set template for generating prompt words. The preset generation model can be a large language generation model or a pre-trained generation model. In this way, a preset generative model can be used to fuse the first video description text of the target sub-video to be described with the first image description text of the target image to be described based on the face detection results to generate an image description text for the target image to be described. This image description text, combined with the face detection results, can accurately distinguish different people in the target image to be described, avoiding misidentification of different people as the same person, thereby improving the accuracy of the image description text.

[0062] In some embodiments, when the face detection results include person identifiers and person positions, the first image description text, the first video description text, and the image description text of the target image to be described also include the person positions corresponding to each person identifier. This means that when performing face detection processing on the image to be described and the sub-video to be described, performing image description processing on the image to be described, and performing video description processing on the sub-video to be described, the person position of each person also needs to be identified. In this way, when generating the image description text, the person position is also taken into account to avoid failing to identify closer or smaller people, thereby further improving the accuracy of the generated image description text.

[0063] In the above scheme, first, the video to be described is subjected to frame extraction and segmentation processing to obtain at least one frame of image to be described and at least one sub-video to be described. A sub-video to be described includes at least one frame of image to be described. Then, face detection processing is performed on both the image to be described and the sub-video to be described to obtain face detection results. The image to be described is subjected to image description processing to obtain the first image description text for each frame of image to be described, and the sub-video to be described is subjected to video description processing to obtain the first video description text for each sub-video to be described. Then, for the target image to be described in the at least one frame of image to be described, the sub-video to be described containing the target image to be described is determined as the target sub-video to be described. The target image to be described is any image to be described in the at least one frame of image to be described. Finally, based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is merged into the first image description text of the target image to be described to obtain the image description text of the target image to be described. In this way, on the one hand, when generating the image description text of the target image to be described, the face detection results can be used to connect the target sub-video to be described and the individual characters in the target image to be described, and finally form the image description text of the target image to be described, thereby improving the coherence between the image description text of the target image to be described and the image description texts of other images to be described, thereby improving the accuracy of the generated video image description text. On the other hand, when generating the image description text of the target image to be described, the corresponding spatiotemporal information of the target image to be described in the target sub-video to be described is also taken into account, further enriching the content of the image description text, thereby improving the accuracy of the generated video image description text. In addition, the image description text combined with the face detection results can accurately distinguish different people in the target image to be described, avoiding identifying different people as the same person, thereby further improving the accuracy of the image description text.

[0064] In some embodiments, the face detection result includes a person identifier; the first image description text includes at least one image description subtext corresponding to the person identifier, and the first video description text includes at least one video description subtext corresponding to the person identifier. Figure 4 As shown, according to the face detection results of the target image to be described and the target sub-video to be described, the method of fusing the first video description text of the target sub-video to be described into the first image description text of the target image to be described to obtain the image description text of the target image to be described may include the following steps:

[0065] S161 : Determine a person identifier that exists in both the person detection results of the target sub-video to be described and the target image to be described as a shared identifier.

[0066] For example, if the character detection results of the target sub-video to be described include characters A, B, C, and D, and the character detection results of the target image to be described include characters B and C, the character identifiers of characters B and C are determined as common identifiers.

[0067] S162: Determine the video description subtext corresponding to the shared identifier in the first video description text of the target sub-video to be described as the shared video description text.

[0068] For example, when the shared identifiers include characters B and C, and the first video description text of the target sub-video to be described includes video description sub-texts corresponding to the character identifiers of characters A, B, C, and D, the video description sub-texts corresponding to the character identifiers of characters B and C are determined as the shared video description text.

[0069] S163: Determine the image description subtext corresponding to the shared identifier in the first image description text of the target image to be described as the shared image description text.

[0070] Specifically, since the target sub-video to be described itself includes the target image to be described, the character identifiers that can be identified in the target image to be described will most likely fall within the character identifiers of the target sub-video to be described, that is, all character identifiers included in the first image description text of the target image to be described will basically be determined as common identifiers and participate in the fusion, further ensuring the accuracy of the final generated image description text.

[0071] S164: Fusing the shared video description text into the shared image description text to obtain a fused image description text corresponding to the shared identifier.

[0072] Specifically, the shared video description text can be fused with the shared image description text to obtain a fused image description text corresponding to the shared identifier. The shared video description text and the shared image description text can be deduplicated and concatenated to obtain the fused image description text corresponding to the shared identifier. Alternatively, keyword extraction can be performed on the shared video description text to obtain a first keyword, and keyword extraction can be performed on the shared image description text to obtain a second keyword; the first keyword and the second keyword can then be concatenated, and semantic expansion can be performed on the concatenated first and second keywords to obtain the fused image description text corresponding to the shared identifier. Alternatively, the shared video description text and the shared image description text can be fused using a preset semantic fusion model to obtain a fused image description text corresponding to the shared identifier. The preset semantic fusion model can be a pre-trained fusion model or a large language model. In this way, the shared video description text can be fused with the shared image description text using different fusion methods to obtain a fused image description text, ensuring the integrity and contextual consistency of the generated fused image description text, enriching the semantic information of the fused image description text, and thereby improving the accuracy of the generated image description text for the target image to be described.

[0073] S165 , jointly determining the fused image description text and the image description subtext of the target image to be described that is not fused with the shared video description text as the image description text of the target image to be described.

[0074] That is, after the common video description text and the common image description text are fused, the fused image description text and the unfused image description sub-text are determined together as the image description text of the target image to be described.

[0075] In the above scheme, the first video description text with the same character identifier can be fused into the first image description text according to the character identifier in the face detection result, and then jointly determined as the final image description text together with the part of the first image description text that is not fused with the first video description text. That is, when fusing the first video description text and the first image description text, only the video description sub-text and the image description sub-text of the same character are fused according to the character identifier, instead of integrating all the contents in the first video description text into the first image description text. In this way, while ensuring the accuracy of the image description text of the target image to be described, computing resources are also saved.

[0076] In some embodiments, as Figure 5 As shown, the method of performing picture description processing on the image to be described to obtain the first image description text of each frame of the image to be described, and performing video description processing on the sub-video to be described to obtain the first video description text of each sub-video to be described may include the following steps:

[0077] S131 , performing image description processing on the image to be described according to the face detection result, to obtain a first image description text for each frame of the image to be described.

[0078] The first image description text includes at least one image description sub-text corresponding to a character identifier.

[0079] Specifically, when performing picture description processing on the image to be described, it is performed in combination with the face detection result to ensure that the generated first image description text includes the image description subtext corresponding to each character.

[0080] S141 , performing video description processing on the sub-video to be described according to the face detection result, to obtain a first video description text for each sub-video to be described.

[0081] The first video description text includes at least one video description subtext corresponding to a character identifier.

[0082] Similarly, when performing video description processing on the sub-video to be described, the face detection result is combined to ensure that the generated first video description text includes the image description sub-text corresponding to each character.

[0083] In the above scheme, the image to be described and the sub-video to be described can be processed separately according to the face detection results to obtain the character identification in the image to be described and the sub-video to be described, and the corresponding relationship between each image description sub-text and each video description sub-text, which ensures that the video description sub-text and image description sub-text of the same character can be fused according to the character identification, indirectly improving the accuracy of generating the image description text of the target image to be described.

[0084] In some embodiments, as Figure 6 As shown, a method of generating prompt words based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described, the first image description text of the target image to be described, and a preset template, and then using the prompt words to generate the image description text of the target image to be described through a preset generation model may include the following steps:

[0085] S166. Determine a preset template according to the face detection result.

[0086] The preset template is a preset template for generating prompt words.

[0087] Specifically, the method for determining the preset template based on the face detection results is related to the pre-set association relationship of the preset template. For example, if the association relationship of the preset template is an association relationship between the preset template and a keyword, the method for determining the preset template based on the face detection results is to match the preset template with the keyword in the face detection result. If the preset association relationship is an association relationship between the preset template and a regular expression, the method for determining the preset template based on the face detection result is to use a regular expression to match the face detection result with the preset template. For another example, when the preset template is a general template, after determining the face detection result, the general template is directly determined as the preset template.

[0088] In some embodiments, the face detection result includes at least a person identifier. The method for determining the preset template based on the face detection result can be to determine whether the face detection result includes the person identifier, and if the face detection result includes the person identifier, determine the preset template as the first preset template; if the face detection result does not include the person identifier, determine the preset template as the second preset template.

[0089] Among them, the method of determining whether the face detection result includes the person identification can be to determine whether the face detection result includes the person identification through keyword matching. For example, when a pre-set person identification is matched in the face detection result, it is determined that the face detection result includes the person identification. When the pre-set person identification is not matched, it is determined that the face detection result does not include the person identification.

[0090] The first preset template at least includes the required output: a first image description text, a first video description text, a character identifier, and a fusion result of the character's position.

[0091] The second preset template at least requires output: the result of merging the first image description text and the first video description text. For example, the second preset template is as follows:

[0092] I want you to act as an intelligent image captioner. You should generate a descriptive, coherent and logical description of the image based on the given descriptions from different people for the same image.

[0093] Overall Video Caption: {the first video description text of the sub-video to be described including the image to be described}.

[0094] Overall Image Caption: {the first image description text of the image to be described}.

[0095] Please generate a description of the plot of the image to be described based on the overall description of the video to be described, the first image description of the image to be described, and the first video description of the sub-video to be described that includes the image to be described. Please do not output the overall description of the video to be described, nor the character and movie title information in the first video description. Do not include dialogue in the description. Output must be strictly in JSON format, with the key "seconds" representing the second in the video to be described. Output of other content is prohibited.

[0096] In the above solution, different preset templates for generating prompt words can be automatically called according to different input information, thereby improving the flexibility of generating video image description text.

[0097] S167, generating prompt words from the face detection result, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described according to a preset template, and using the prompt words to generate an image description text of the target image to be described through a preset generation model.

[0098] First, prompt words are generated from the face detection result, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described according to a preset template.

[0099] For example, when the preset template is the first preset template, the face detection result includes the person identification and the person position; the person position is the coordinate information corresponding to the upper left corner and the lower right corner of the face detection frame, and the coordinate information of the person position retains the thousandths, the generated prompt words are as follows:

[0100] I want you to act as an intelligent image captioner. You should generate a descriptive, coherent and logical description of the image based on the given descriptions from different people for the same image. The position is represented by normalized top-left and bottom-right coordinate.

[0101] Overall Video Caption: {the first video description text of the sub-video to be described including the image to be described}.

[0102] Overall Image Caption: {the first image description text of the image to be described}.

[0103] Face detection results (the person's position is reflected by the face detection frame's thousandths coordinate information, [x1, y1] represents the upper left corner, and [x2, y2] represents the lower right corner): {person identifier and person position}.

[0104] Please generate a description of the plot of the video to be described based on the overall description of the video to be described, the face detection results, the first image description of the image to be described, and the first video description of the sub-video to be described that includes the image to be described. Please do not output the overall description of the video to be described, nor the characters and movie title information in the first video description. Please match the face detection bounding boxes in the face detection results with the human body detection bounding boxes in the first video description of the sub-video to be described that includes the image to be described. Do not include coordinates in the results. For people with face detection results, replace them with person identifiers, and all person identifiers in the face detection results must be output. The description should highlight the relative position of the characters (left and right). Do not include dialogue in the description. The output must be strictly in JSON format, with the key "second" representing the second in the video to be described. No other content is allowed.

[0105] Afterwards, the prompt words are fed into a preset generation model to generate an image description text of the target image to be described. For example, the generated image description text of the target image to be described is as follows: at the nth second, plot description n.

[0106] In the above scheme, the input information can be generated into prompt words according to the preset template, and the prompt words can be used to generate the image description text of the target image to be described through the preset generation model, providing a method for integrating the generation of the image description text of the image to be described to ensure the accuracy of the generated video image description text.

[0107] In some embodiments, after generating image description text for each image to be described, the method for generating video image description text further includes generating target video description text for a target video segment in the video to be described based on the image description text for each frame of the image to be described and the face detection results for each frame of the image to be described.

[0108] Specifically, the method of generating the target video description text is similar to the method of generating the image description text of the target image to be described based on the face detection result, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described in step S16, that is, the image description texts of all the images to be described included in the target video clip can be directly simply spliced ​​to obtain the target video description text of the target video clip; it is also possible to first generate prompt words based on the image description text of the image to be described, the face detection result of the image to be described, and the third preset template, and then use the prompt words to generate the target video description text of the target video clip through the preset generation model; it is also possible to first string together the characters and positions in the target video clip based on the face detection result of the image to be described in the target video clip, and then combine the image description text of the image to be described in the target video clip and the preset generation model to generate the target video description text of the target video clip.

[0109] For example, in Figure 3 In the scene shown, the duration of the video V to be described is 20s, the first preset time interval is 1s, the second preset time interval is 10s, and the target video segment is a video segment with a total duration of 3s from the 3rd to the 5th second of the video to be described. Then, the characters and positions in the target video segment are strung together based on the face detection results of the image to be described 3, the face detection results of the image to be described 4, and the face detection results of the image to be described 5; then, the image description text of the image to be described 3, the image description text of the image to be described 4, the image description text of the image to be described 5 and the first preset generation model are combined to generate the target video description text from the 3rd to the 5th second of the video to be described.

[0110] In this approach, the target video description text for the target video segment in the video being described can be generated using the character identifiers and image descriptions of each frame of the image being described. This ensures the accuracy and consistency of the generated target video description text while improving the scalability of the generated video image description text.

[0111] The embodiment of the present disclosure can divide the functional modules of the device for generating the video image description text according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present disclosure is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0112] like Figure 7, which is a structural diagram of a device for generating a video image description text according to an embodiment of the present disclosure, the device for generating a video image description text includes a video processing module 51 , an image processing module 52 , a video determination module 53 , and a text generation module 54 .

[0113] The video processing module 51 is used to perform frame extraction and segmentation processing on the video to be described to obtain at least one frame of image to be described and at least one sub-video to be described; wherein, one sub-video to be described includes at least one frame of image to be described; the image processing module 52 is used to perform face detection processing on both the image to be described and the sub-video to be described to obtain a face detection result; perform picture description processing on the image to be described to obtain a first image description text for each frame of image to be described, and perform video description processing on the sub-video to be described to obtain a first video description text for each sub-video to be described; the video determination module 53 is used to determine the sub-video to be described containing the target image to be described as the target sub-video to be described for the target image to be described in the at least one frame of image to be described; the target image to be described is any image to be described in the at least one frame of image to be described; the text generation module 54 is used to merge the first video description text of the target sub-video to be described into the first image description text of the target image to be described based on the face detection results of the target image to be described and the target sub-video to be described to obtain the image description text of the target image to be described.

[0114] In the above scheme, first, the video to be described is subjected to frame extraction and segmentation processing to obtain at least one frame of image to be described and at least one sub-video to be described. A sub-video to be described includes at least one frame of image to be described. Then, face detection processing is performed on both the image to be described and the sub-video to be described to obtain face detection results. The image to be described is subjected to image description processing to obtain the first image description text for each frame of image to be described, and the sub-video to be described is subjected to video description processing to obtain the first video description text for each sub-video to be described. Then, for the target image to be described in the at least one frame of image to be described, the sub-video to be described containing the target image to be described is determined as the target sub-video to be described. The target image to be described is any image to be described in the at least one frame of image to be described. Finally, based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is merged into the first image description text of the target image to be described to obtain the image description text of the target image to be described. In this way, on the one hand, when generating the image description text of the target image to be described, the face detection results can be used to connect the target sub-video to be described and the individual characters in the target image to be described, and finally form the image description text of the target image to be described, thereby improving the coherence between the image description text of the target image to be described and the image description texts of other images to be described, thereby improving the accuracy of the generated video image description text. On the other hand, when generating the image description text of the target image to be described, the corresponding spatiotemporal information of the target image to be described in the target sub-video to be described is also taken into account, further enriching the content of the image description text, thereby improving the accuracy of the generated video image description text. In addition, the image description text combined with the face detection results can accurately distinguish different people in the target image to be described, avoiding identifying different people as the same person, thereby further improving the accuracy of the image description text.

[0115] In some embodiments, the face detection result includes a person identifier; the first image description text includes at least one image description subtext corresponding to the person identifier, and the first video description text includes at least one video description subtext corresponding to the person identifier; the text generation module 54 is specifically used to: determine the person identifier that exists in the person detection results of the target sub-video to be described and the target image to be described as a common identifier; determine the video description subtext corresponding to the common identifier in the first video description text of the target sub-video to be described as the common video description text, and determine the image description subtext corresponding to the common identifier in the first image description text of the target image to be described as the common image description text; fuse the common video description text into the common image description text to obtain a fused image description text corresponding to the common identifier; and jointly determine the fused image description text and the image description subtext in the target image to be described that is not fused with the common video description text as the image description text of the target image to be described.

[0116] In some embodiments, the image processing module 52 is specifically used to: perform picture description processing on the image to be described according to the face detection result, and obtain a first image description text for each frame of the image to be described; the first image description text includes at least one image description sub-text corresponding to a character identifier; perform video description processing on the sub-video to be described according to the face detection result, and obtain a first video description text for each sub-video to be described; the first video description text includes at least one video description sub-text corresponding to a character identifier.

[0117] In some embodiments, the text generation module 54 is specifically used to: deduplicate and splice the shared video description text and the shared image description text to obtain a fused image description text corresponding to the shared identifier; or, extract keywords from the shared video description text to obtain a first keyword, and extract keywords from the shared image description text to obtain a second keyword; splice the first keyword and the second keyword, and perform semantic expansion on the spliced ​​first keyword and second keyword to obtain a fused image description text corresponding to the shared identifier; or, use a preset semantic fusion model to fuse the shared video description text and the shared image description text to obtain a fused image description text corresponding to the shared identifier.

[0118] In some embodiments, the text generation module 54 is specifically used to: construct prompt words based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described, and use the prompt words to generate image description text of the target image to be described through a preset generation model.

[0119] In some embodiments, when the face detection result includes person identifiers and person positions, the first image description text, the first video description text, and the image description text of the target image to be described also include the person position corresponding to each person identifier.

[0120] In some embodiments, the video processing module 51 is specifically used to: extract frames of the video to be described according to a first time interval to obtain at least one frame of image to be described; cut the video to be described according to a second time interval to obtain at least one sub-video to be described; the second time interval is greater than the first time interval.

[0121] The device for generating video image description text provided in this embodiment can execute the method for generating video image description text provided in the above method embodiment. Its implementation principle and technical effects are similar to those of the above method and will not be repeated here.

[0122] Figure 8 An electronic device according to an exemplary embodiment may include a processor 902 configured to execute application code to implement the method for generating a video image description text in the present disclosure.

[0123] The processor 902 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the disclosed solution.

[0124] like Figure 8 As shown, the electronic device may further include a memory 903. The memory 903 is used to store application code for executing the solution of the present disclosure, and the execution is controlled by the processor 902.

[0125] The memory 903 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 903 may exist independently and be connected to the processor 902 via the bus 904. The memory 903 may also be integrated with the processor 902.

[0126] like Figure 8 As shown, the electronic device may further include a communication interface 901, wherein the communication interface 901, the processor 902, and the memory 903 may be coupled to each other, for example, via a bus 904. The communication interface 901 is used to exchange information with other devices, for example, to support information exchange between the electronic device and other devices.

[0127] It should be pointed out that Figure 8 The device structure shown in the figure does not constitute a limitation on the electronic device, except Figure 8 In addition to the components shown, the electronic device may include more or fewer components than shown, or may combine certain components or arrange the components differently. Furthermore, the electronic device provided in this embodiment can execute the method for generating video image description text provided in the above method embodiment. Its implementation principles and technical effects are similar to those of the above method and will not be further described here.

[0128] The embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the various processes of the method for generating video image description text in the above method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0129] The computer-readable storage medium may be a ROM, RAM, a magnetic disk, or an optical disk.

[0130] An embodiment of the present disclosure provides a computer program product, which stores a computer program. When the computer program is executed by a processor, each process of the method for generating video image description text in the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0131] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0132] In this disclosure, memory may include non-permanent memory in a computer-readable medium, RAM and / or non-volatile memory, such as ROM or flash RAM. Memory is an example of a computer-readable medium.

[0133] In the present disclosure, computer-readable media includes permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data and carrier waves.

[0134] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0135] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a video image description text, characterized in that: include: Performing frame extraction and segmentation processing on the video to be described to obtain at least one frame of image to be described and at least one sub-video to be described; wherein a sub-video to be described includes at least one frame of image to be described; Performing face detection processing on both the image to be described and the sub-video to be described to obtain a face detection result; performing picture description processing on the image to be described to obtain a first image description text for each frame of the image to be described, and performing video description processing on the sub-video to be described to obtain a first video description text for each sub-video to be described; For a target image to be described in the at least one frame of images to be described, determining a sub-video to be described containing the target image to be described as a target sub-video to be described; the target image to be described is any image to be described in the at least one frame of images to be described; According to the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described is merged into the first image description text of the target image to be described to obtain the image description text of the target image to be described.

2. The generation method according to claim 1, characterized in that The face detection result includes a person identifier; the first image description text includes at least one image description subtext corresponding to the person identifier, and the first video description text includes at least one video description subtext corresponding to the person identifier; and the step of fusing the first video description text of the target sub-video to be described with the first image description text of the target image to be described based on the face detection results of the target image to be described and the target sub-video to be described to obtain the image description text of the target image to be described includes: Determine the person identifiers that exist in both the person detection results of the target sub-video to be described and the target image to be described as common identifiers; Determining the video description subtext corresponding to the shared identifier in the first video description text of the target sub-video to be described as the shared video description text, and determining the image description subtext corresponding to the shared identifier in the first image description text of the target image to be described as the shared image description text; Merging the shared video description text into the shared image description text to obtain a fused image description text corresponding to the shared identifier; The fused image description text and the image description subtext of the target image to be described that is not fused with the shared video description text are jointly determined as the image description text of the target image to be described.

3. The generation method according to claim 2, characterized in that The performing picture description processing on the image to be described to obtain a first image description text for each frame of the image to be described, and performing video description processing on the sub-video to be described to obtain a first video description text for each sub-video to be described, includes: Performing image description processing on the image to be described according to the face detection result to obtain a first image description text for each frame of the image to be described; the first image description text includes at least one image description subtext corresponding to a character identifier; Video description processing is performed on the sub-video to be described according to the face detection result to obtain a first video description text for each sub-video to be described; the first video description text includes a video description sub-text corresponding to at least one character identifier.

4. The generation method according to claim 2, characterized in that The step of fusing the shared video description text to the shared image description text to obtain a fused image description text corresponding to the shared identifier includes: After deduplication processing and splicing processing are performed on the shared video description text and the shared image description text, a fused image description text corresponding to the shared identifier is obtained; or, Performing keyword extraction on the shared video description text to obtain a first keyword, and performing keyword extraction on the shared image description text to obtain a second keyword; splicing the first keyword and the second keyword, and performing semantic expansion processing on the spliced ​​first keyword and the second keyword to obtain a fused image description text corresponding to the shared identifier; or, The common video description text and the common image description text are fused using a preset semantic fusion model to obtain a fused image description text corresponding to the common identifier.

5. The generation method according to claim 1, characterized in that The step of fusing the first video description text of the target sub-video to be described into the first image description text of the target image to be described based on the face detection results of the target image to be described and the target sub-video to be described to obtain the image description text of the target image to be described includes: Prompt words are constructed based on the face detection results of the target image to be described and the target sub-video to be described, the first video description text of the target sub-video to be described, and the first image description text of the target image to be described, and the prompt words are used to generate image description text of the target image to be described through a preset generation model.

6. The generation method according to any one of claims 1 to 5, characterized in that: When the face detection result includes person identifiers and person positions, the first image description text, the first video description text, and the image description text of the target image to be described also include the person position corresponding to each person identifier.

7. The generation method according to any one of claims 1 to 5, characterized in that: The frame extraction and segmentation processing of the video to be described includes: Performing frame extraction processing on the video to be described according to a first time interval to obtain at least one frame of image to be described; The video to be described is segmented according to a second time interval to obtain at least one sub-video to be described; the second time interval is greater than the first time interval.

8. A device for generating a video image description text, characterized in that: include: A video processing module is used to extract frames and cut the video to be described to obtain at least one frame of image to be described and at least one sub-video to be described; wherein a sub-video to be described includes at least one frame of image to be described; An image processing module is configured to perform face detection processing on both the image to be described and the sub-video to be described to obtain a face detection result; perform picture description processing on the image to be described to obtain a first image description text for each frame of the image to be described; and perform video description processing on the sub-video to be described to obtain a first video description text for each sub-video to be described; a video determination module, configured to determine, for a target image to be described in the at least one frame of images to be described, a sub-video to be described containing the target image to be described as a target sub-video to be described; the target image to be described is any image to be described in the at least one frame of images to be described; The text generation module is used to fuse the first video description text of the target sub-video to be described into the first image description text of the target image to be described based on the face detection results of the target image to be described and the target sub-video to be described, so as to obtain the image description text of the target image to be described.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method for generating a video image description text according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating video image description text according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to implement the method for generating a video image description text according to any one of claims 1 to 7.

Images