Method and system for automatically generating vehicle introduction video

Abstract

The invention discloses a method and a system for automatically generating a vehicle introduction video. The method comprises the following steps: acquiring a plurality of images of a vehicle; generating vehicle description text information corresponding to the image content based on the image and the category label thereof; performing sentence segmentation processing on the vehicle description text information to obtain a plurality of commentary short sentences; determining an optimal matching image for each commentary short sentence in sequence to form an image-text pair sequence; converting the explanation short sentences into voice audios, and obtaining start timestamp and end timestamp information of the voice audios corresponding to each explanation short sentence, so as to adjust the duration of the image corresponding to the explanation short sentence and determine the duration of the subtitle display result; and according to the commentary short sentence and the length of the text content of the commentary short sentence, generating at least one subtitle line through line breaking processing to obtain a subtitle display result. And synthesizing the dynamic subtitles, the voice audio and the matched image to obtain a vehicle introduction video containing the synchronous subtitles and dubbing. Through the method, the vehicle introduction video can be automatically generated.

Description

Technical Field

[0001] This application relates to the field of video generation technology, and in particular to a method and system for automatically generating vehicle introduction videos. Background Technology

[0002] With the increasing prevalence of digital marketing and online transactions, the visual presentation of vehicles (especially used cars) has become a key factor influencing consumer decisions. Creating professional, engaging, and personalized video introductions for each vehicle can significantly improve information delivery efficiency and user viewing experience. However, faced with massive amounts of vehicle data, traditional and mainstream video production methods face significant challenges in terms of efficiency, cost, quality, and scalability.

[0003] Currently, the industry mainly relies on the following two types of technical solutions for vehicle video content production:

[0004] The first approach is a completely manual production solution. A professional team (including a host, cameraman, and editor) films each car on location, provides on-site narration, and then performs complex post-production editing, dubbing, background music, and subtitle creation. However, this method is entirely manual, resulting in low production efficiency and high costs. It requires a significant investment of manpower and time, has a long production cycle, and struggles to meet the video production needs of the large number of new used car listings added daily by used car platforms.

[0005] The second approach is template-based video generation. Using online video editing tools or software, users manually upload vehicle images to a preset video template. Users then manually write the narration script, record audio themselves, or use a TTS tool to convert the text into an audio file. Finally, users manually import the generated video and audio files into editing software for alignment, editing, and compositing. This method still relies entirely on manual work in professional software for the core "audio-visual-subtitle" alignment and compositing process. While some steps are assisted by technology, it essentially remains within the paradigm of manual editing, and the bottleneck in production efficiency remains significant. Furthermore, existing TTS tools directly convert long texts, lacking pauses and rhythm that conform to natural speaking habits, resulting in unnatural audio in the generated videos. Summary of the Invention

[0006] Based on this, and in response to the aforementioned technical problems, an automatic method for generating vehicle introduction videos is provided to solve the problem that existing technologies rely on manual operation to generate vehicle introduction videos, resulting in unnatural audio.

[0007] Firstly, a method for automatically generating vehicle introduction videos, the method comprising:

[0008] Acquire multiple images of the vehicle, each image carrying a category label identifying the type of part it was photographed;

[0009] Based on the image and its category labels, a multimodal AI text generation model is invoked to generate vehicle description text information corresponding to the image content;

[0010] The vehicle description text information is segmented into sentences to obtain multiple explanatory sentences; the best matching image is determined for each explanatory sentence in turn to form a sequence of image-text pairs;

[0011] The narration phrases are converted into audio, and the start and end timestamps of the audio corresponding to each narration phrase are obtained; the duration of the image corresponding to the narration phrase is adjusted according to the start and end timestamps of the audio corresponding to the narration phrase.

[0012] Based on the explanatory short sentence, at least one subtitle line is generated through line breaking based on the length of its text content to obtain the subtitle display result; and based on the start and end timestamp information of the audio of the explanatory short sentence, the duration of the subtitle display result is generated to generate dynamic subtitles;

[0013] The dynamic subtitles, the audio, and the matching image are combined to obtain a vehicle introduction video containing synchronized subtitles and voice-over.

[0014] Optionally, in the above scheme, the step of generating vehicle description text information corresponding to the image content by calling a multimodal AI text generation model based on the image and its category labels includes:

[0015] Based on the category label of the image, match the corresponding descriptive dimension guide from the preset guide word library;

[0016] The image and the descriptive dimension guide are input into the multimodal AI text generation model so that the multimodal AI text generation model can recognize the visual content of the image and, in combination with the descriptive dimension defined by the guide, generate vehicle description text information with industry professionalism.

[0017] Optionally, in the above scheme, determining the best matching image for each explanatory phrase to form a sequence of image-text pairs includes:

[0018] Based on the text group to which the current explanatory phrase belongs, images with consistent category labels are selected from the multiple images to form a candidate image set;

[0019] The pre-trained multimodal semantic matching model is invoked to project the current explanatory phrase and the candidate image set into the same high-dimensional cross-modal semantic space.

[0020] Calculate the cosine similarity between the feature vector of the explanatory phrase and the feature vector of each candidate image;

[0021] The image with the highest similarity is selected as the initial matching image for the current explanatory phrase; if the initial matching image is the same as the matching image of the previous explanatory phrase, the image with the second highest similarity is selected as the final matching image to form a sequence of image-text pairs.

[0022] Optionally, in the above scheme, the multimodal semantic matching model is constructed using a dual-tower contrastive language-image pre-training architecture, including:

[0023] Candidate images and explanatory phrases are projected into the same high-dimensional cross-modal semantic space using a visual encoder and a text encoder, respectively.

[0024] A contrastive learning strategy is adopted to train the visual encoder and text encoder on a large-scale image-text pair dataset to maximize the cosine similarity of feature vectors of related image-text pairs in the semantic space, thereby achieving semantic alignment between images and text.

[0025] Optionally, in the above scheme, the vehicle description text information can be segmented into sentences to obtain multiple explanatory short sentences, including:

[0026] Step S51: Using regular expressions, the vehicle description text information is initially segmented based on punctuation marks in the text to obtain text fragments, and the punctuation marks are retained as delimiters for each fragment;

[0027] Step S52: Initialize an empty cache region;

[0028] Step S53: Append each text fragment sequentially to the end of the cache to form a temporary sentence, and perform the following judgment after each append:

[0029] If the length of the temporary sentence exceeds the preset minimum length threshold for the first time, then continue to add subsequent segments until the next punctuation-type segmentation point is encountered, resulting in a new temporary sentence. If the length of the new temporary sentence exceeds the preset maximum length threshold, then revert to all content before the most recent addition and output it as an explanatory short sentence. Otherwise, directly output the temporary sentence as an explanatory short sentence and then clear the cache.

[0030] Step S54: Repeat step S53 until all segments have been processed.

[0031] In the above scheme, optionally, based on the explanatory short sentence and the length of its text content, at least one subtitle line is generated through line breaking to obtain the subtitle display result, including:

[0032] Determine whether the number of characters in the explanatory phrase exceeds a preset single-line display threshold. If not, treat the explanatory phrase as a single subtitle line; if so, divide the explanatory phrase into multiple sub-text segments, each sub-text segment corresponding to a subtitle line.

[0033] Optionally, in the above scheme, before synthesizing the dynamic subtitles, the audio, and the matching image, the following steps are also included:

[0034] The display effects for the dynamic subtitles are set, including: font size, font style, and color;

[0035] Configure background music for the voice audio and mix it with the audio after adjusting its volume to be lower than that of the voice audio.

[0036] Randomly selected transition effects are inserted between the images.

[0037] Secondly, an automatic generation system for vehicle introduction videos, the system comprising:

[0038] Image acquisition module: used to acquire multiple images of the vehicle, the images carrying category labels that identify the type of the captured component;

[0039] Text generation module: used to generate vehicle description text information corresponding to the image content by calling a multimodal AI text generation model based on the image and its category labels;

[0040] The image-text matching module is used to segment the vehicle description text information into sentences to obtain multiple explanatory sentences; and to determine the best matching image for each explanatory sentence in turn to form an image-text pair sequence.

[0041] The audio-visual adjustment module is used to convert the narration phrases into audio, and obtain the start and end timestamp information of the audio corresponding to each narration phrase; and adjust the duration of the image corresponding to the narration phrase according to the start and end timestamp information of the audio corresponding to the narration phrase.

[0042] The subtitle generation module is used to generate at least one subtitle line based on the length of the explanatory phrase and its text content through line breaking to obtain the subtitle display result; and to generate the duration of the subtitle display result based on the start and end timestamp information of the audio of the explanatory phrase, thereby generating dynamic subtitles.

[0043] Vehicle introduction video output module: used to synthesize the dynamic subtitles, the audio, and the matching image to obtain a vehicle introduction video containing synchronized subtitles and voice-over.

[0044] Thirdly, a computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the automatic generation method steps for the vehicle introduction video described in the first aspect.

[0045] Fourthly, a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the automatic generation method for vehicle introduction videos described in the first aspect.

[0046] This application has at least the following beneficial effects:

[0047] This application utilizes a multimodal AI text generation model to directly generate descriptive text from vehicle images with category labels, achieving end-to-end automated production from raw images to complete videos. This eliminates the need for manual script writing, significantly reducing production barriers and labor costs. By reverse-engineering the optimal matching images for narration phrases to form image-text pairs, it ensures precise alignment between visual presentation and voice narration, enhancing narrative coherence and the viewer experience. By adjusting image duration using audio timestamp information, it achieves precise audio-visual synchronization, eliminating the asynchronicity often found in traditional methods. Furthermore, by intelligently breaking lines based on text length and controlling subtitle duration using audio timestamps, it optimizes subtitle readability and visual presentation. The entire process requires no manual intervention, eliminating subjective differences and ensuring consistent style and stable, controllable quality for batch-generated videos. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a method for automatically generating vehicle introduction videos according to an embodiment of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] In one embodiment, such as Figure 1 As shown, an automatic generation method for vehicle introduction videos is provided, including:

[0051] Step S1: Acquire multiple images of the vehicle, each image carrying a category label identifying the type of part it was photographed.

[0052] In step S1, the captured images are pre-categorized into groups such as summary (vehicle overview), appearance (exterior details), inner (interior details), and engine (engine compartment) when uploaded, and therefore carry category tags for the type of captured parts.

[0053] Step S2: Based on the image and its category label, call the multimodal AI text generation model to generate vehicle description text information corresponding to the image content.

[0054] In step S2, for each set of images, the system invokes a multimodal AI image-to-text model. Based on the specific image content (such as identifying car body gloss, seat material, engine compartment layout, etc.), the model automatically generates corresponding descriptive text. Since the text is generated independently based on images of a specific category, the generated text data naturally corresponds to the four structured fields mentioned above, directly completing the text structuring process without requiring complex text semantic analysis and decomposition.

[0055] Step S3: Segment the vehicle description text information to obtain multiple explanatory sentences; determine the best matching image for each explanatory sentence in turn to form a sequence of image-text pairs.

[0056] In step S3, the intelligent sentence segmentation module is invoked to process each part of the text. This algorithm employs a composite strategy combining punctuation and length thresholds. Simultaneously, the vehicle's image library is categorized, containing at least one "cover image" representing the vehicle as a whole, and a "picture list" containing multiple detailed images of the exterior, interior, engine compartment, and other areas.

[0057] The system directly processes the structured data generated during the vehicle data collection phase. Because operators strictly follow pre-defined procedures for taking and uploading images during vehicle inspection or photography (e.g., left front 45-degree angle, center console panoramic view, engine compartment, etc.), the image data input into the system naturally carries location tag information. The system reads these pre-defined metadata tags or storage paths, directly mapping the image set to the corresponding logical groups; it automatically locks a standard angle image representing the overall vehicle view (e.g., left front 45-degree angle) as the cover image, and directly categorizes the remaining images into the corresponding image lists for exterior, interior, engine, etc., based on the tags. This eliminates the computational overhead of image recognition and classification, ensuring classification accuracy.

[0058] The first sentence of the summary section is forcibly linked to the vehicle's "cover image" to ensure that the video opens with a full display of the vehicle.

[0059] Then, the image-text matching module is invoked to send all the remaining short texts and the "image list" to an external text-image semantic matching API for image-text matching.

[0060] Step S4: Convert the narration phrases into audio, and obtain the start and end timestamps of the audio corresponding to each narration phrase; adjust the duration of the image corresponding to the narration phrase based on the start and end timestamps of the audio corresponding to the narration phrase.

[0061] In step S4, the matched image-text pairs can be distributed to four independent video generation threads based on the four parts: "summary," "appearance," "inner," and "engine." These four threads work in parallel, which, ideally, can reduce the generation time of the main video portion by approximately 75%.

[0062] In each thread, for each image-text pair, the following process is executed:

[0063] TTS Processing and Timestamp Acquisition: Call the TTS service to convert short text sentences into WAV audio files and return a timestamp metadata containing each word and its "start_time" and "end_time" accurate to milliseconds.

[0064] The matching image is loaded using the "ImageClip" function from the "moviepy" library. To enhance visual dynamism, a time-dependent scaling transformation is applied, creating a slow, zooming effect as the image is played.

[0065] Step S5: Based on the explanatory phrase and the length of its text content, generate at least one subtitle line through line breaking to obtain the subtitle display result; and generate the duration of the subtitle display result based on the start and end timestamp information of the audio of the explanatory phrase, thus generating dynamic subtitles.

[0066] In step S5, the system utilizes the fine-grained timestamp metadata (Word / Character-level Timestamps) returned by the TTS interface to achieve millisecond-level synchronization between subtitles, audio, and video. The specific calculation logic is as follows:

[0067] Start and end time anchoring: The system traverses the word list of the current short sentence, extracts the start_time of the first character as the in-point of the subtitle on the video timeline, and extracts the end_time of the last character as the out-point of the subtitle.

[0068] 3D Synchronous Synthesis: Through the above calculations, the accurate duration of the subtitle object was determined. The system then overlays this subtitle object onto the image clip corresponding to the current sentence. Since the display duration of this image clip has been set to match the total audio duration, strict alignment and synchronous display of the "image, audio narration, and dynamic subtitle" in the time dimension are achieved.

[0069] Step S6: Combine the dynamic subtitles, audio, and matching images to obtain a vehicle introduction video with synchronized subtitles and voice-over.

[0070] In the aforementioned method for automatically generating vehicle introduction videos, a multimodal AI text generation model directly generates descriptive text from vehicle images carrying category tags, achieving end-to-end automated production from raw images to complete videos. This eliminates the need for manual script writing, significantly reducing production barriers and labor costs. By reverse-engineering the optimal matching images for narration phrases to form image-text pairs, precise alignment between visual presentation and voice narration is ensured, enhancing narrative coherence and the viewer experience. Adjusting image duration by acquiring audio timestamp information achieves precise audio-visual synchronization, eliminating the asynchronicity often found in traditional methods. Intelligent line breaks based on text length, combined with audio timestamps to control subtitle duration, optimize subtitle readability and visual presentation. The entire process requires no manual intervention, eliminating subjective differences and ensuring consistent style and stable, controllable quality for batch-generated videos.

[0071] In one embodiment, generating vehicle description text information corresponding to the image content by calling a multimodal AI text generation model (doubao-1.5-vision-pro) based on the image and its category labels includes:

[0072] Based on the category label of the image, match the corresponding descriptive dimension guide from the preset guide word library;

[0073] The image and the descriptive dimension guidance are input into the multimodal AI text generation model, enabling the model to recognize the visual content of the image and, in conjunction with the descriptive dimension defined by the guidance, generate industry-specific vehicle description text information. The multimodal AI text generation model uses doubao-1.5-vision-pro.

[0074] In one embodiment, determining the best matching image for each explanatory phrase to form a sequence of image-text pairs includes:

[0075] Based on the text group to which the current explanatory phrase belongs, images with consistent category labels are selected from the multiple images to form a candidate image set;

[0076] The pre-trained multimodal semantic matching model is invoked to project the current explanatory phrase and the candidate image set into the same high-dimensional cross-modal semantic space.

[0077] Calculate the cosine similarity between the feature vector of the explanatory phrase and the feature vector of each candidate image;

[0078] The image with the highest similarity is selected as the initial matching image for the current explanatory phrase; if the initial matching image is the same as the matching image of the previous explanatory phrase, the image with the second highest similarity is selected as the final matching image to form a sequence of image-text pairs.

[0079] In this embodiment, the order of the short sentence texts generated in the preceding steps is kept unchanged. For each short sentence in the sequence, the API maps it and the image list under the current category to the same high-dimensional vector space. The system calculates the cosine similarity between the current short sentence vector and the vector of each image in the list, thereby quantifying the semantic association between the text and the image.

[0080] Top-N Sorting and Selection: Images are sorted from highest to lowest based on similarity score. The system defaults to selecting the image with the highest score as the illustration for each sentence. A strict "one sentence, one image" mapping relationship is established to ensure that each sentence is supported by the most appropriate image without disrupting the original textual narrative logic.

[0081] When confirming the final image, the system performs deduplication. It checks if the best image matched for the current sentence (Top 1) is the same as the image matched for the previous sentence (Prev_Image). If they are the same (i.e., Image_Current == Image_Prev), to prevent the video from remaining static between two consecutive sentences, the system automatically backtracks and selects the image with the second-highest similarity score (Top 2) as the image for the current sentence. This mechanism maximizes the visual richness of the video while ensuring semantic relevance.

[0082] In one embodiment, the multimodal semantic matching model is constructed using a dual-tower contrastive language-image pre-training architecture, comprising:

[0083] Candidate images and explanatory phrases are projected into the same high-dimensional cross-modal semantic space using a visual encoder and a text encoder, respectively.

[0084] A contrastive learning strategy is adopted to train the visual encoder and text encoder on a large-scale image-text pair dataset to maximize the cosine similarity of feature vectors of related image-text pairs in the semantic space, thereby achieving semantic alignment between images and text.

[0085] In one embodiment, the vehicle description text information is segmented into sentences to obtain multiple explanatory short sentences, including:

[0086] Step S51: Using regular expressions, the vehicle description text information is initially segmented based on punctuation marks in the text to obtain text fragments, and the punctuation marks are retained as delimiters for each fragment;

[0087] Step S52: Initialize an empty cache region;

[0088] Step S53: Append each text fragment sequentially to the end of the cache to form a temporary sentence, and perform the following judgment after each append:

[0089] If the length of the temporary sentence exceeds the preset minimum length threshold for the first time, then continue to add subsequent segments until the next punctuation-type segmentation point is encountered, resulting in a new temporary sentence. If the length of the new temporary sentence exceeds the preset maximum length threshold, then revert to all content before the most recent addition and output it as an explanatory short sentence. Otherwise, directly output the temporary sentence as an explanatory short sentence and then clear the cache.

[0090] Step S54: Repeat step S53 until all segments have been processed.

[0091] In one embodiment, generating at least one subtitle line based on the length of the explanatory phrase and its text content through line breaking to obtain the subtitle display result includes:

[0092] Determine whether the number of characters in the explanatory phrase exceeds a preset single-line display threshold. If not, treat the explanatory phrase as a single subtitle line; if so, divide the explanatory phrase into multiple sub-text segments, each sub-text segment corresponding to a subtitle line.

[0093] In this embodiment, the intelligent subtitle line break module is invoked. This function receives a list of words returned by TTS. If the total number of words exceeds the maximum number of words per line (e.g., 30 words), it prioritizes finding breakpoints at punctuation marks such as commas or semicolons in the list. If no suitable punctuation is found, the subtitle is broken at approximately two-fifths of the way through.

[0094] In one embodiment, the method further includes the following steps before synthesizing the dynamic subtitles, the audio, and the matching image:

[0095] The display effects for the dynamic subtitles are set, including: font size, font style, and color;

[0096] Configure background music for the voice audio and mix it with the audio after adjusting its volume to be lower than that of the voice audio.

[0097] Randomly selected transition effects are inserted between the images.

[0098] In this embodiment, the matching image is loaded using the "ImageClip" function of the "moviepy" library. To enhance visual dynamism, a time-dependent scaling transformation is applied, causing the image to slowly zoom in during playback.

[0099] Use "TextClip" to create subtitles, set the font and color, and add a stroke to ensure readability. Finally, overlay the timed and positioned subtitle clips onto the image clips.

[0100] Vocal Track Synthesis: The "concatenate_audioclips" function is used to sequentially concatenate all independent WAV audio files into a complete narration track. A background music (BGM) is randomly selected from the built-in music library. The system adjusts the BGM's length based on its relationship to the narration track's length: if the BGM is longer, it's truncated to the same length as the narration; if the BGM is shorter, it loops until it covers the entire narration duration. The BGM volume is adjusted to a low level (e.g., 30%), and then mixed with the vocal narration track using "CompositeAudioClip" to ensure clear narration.

[0101] After all segments are generated in parallel, the video compositing module begins. The system collects the paths of all generated video segments, calls the video merging service, and inserts a randomly selected transition effect between every two segments to make the video flow more natural.

[0102] The spliced ​​video stream and mixed audio track are then combined for final merging. A uniform watermark is overlaid on the video footage. A fade-out effect is applied at the end of the main video segment. Finally, it is spliced ​​with a pre-made uniform end credits video to output the final MP4 video file.

[0103] In one embodiment, an automatic generation system for vehicle introduction videos is provided, comprising:

[0104] Image acquisition module: used to acquire multiple images of the vehicle, the images carrying category labels that identify the type of the captured component;

[0105] Text generation module: used to generate vehicle description text information corresponding to the image content by calling a multimodal AI text generation model based on the image and its category labels;

[0106] The image-text matching module is used to segment the vehicle description text information into sentences to obtain multiple explanatory sentences; and to determine the best matching image for each explanatory sentence in turn to form an image-text pair sequence.

[0107] The audio-visual adjustment module is used to convert the narration phrases into audio, and obtain the start and end timestamp information of the audio corresponding to each narration phrase; and adjust the duration of the image corresponding to the narration phrase according to the start and end timestamp information of the audio corresponding to the narration phrase.

[0108] The subtitle generation module is used to generate at least one subtitle line based on the length of the explanatory phrase and its text content through line breaking to obtain the subtitle display result; and to generate the duration of the subtitle display result based on the start and end timestamp information of the audio of the explanatory phrase, thereby generating dynamic subtitles.

[0109] Vehicle introduction video output module: used to synthesize the dynamic subtitles, the audio, and the matching image to obtain a vehicle introduction video containing synchronized subtitles and voice-over.

[0110] The advantages of this application are:

[0111] 1. Significantly improves production efficiency: Through automated end-to-end processes, the traditional hours of manual production are reduced to minutes. The multi-threaded parallel architecture further reduces core generation time by approximately 75%, thereby supporting large-scale video production needs.

[0112] 2. Ensure high content relevance: Semantic image-text matching and deduplication algorithms ensure accurate correspondence between the explanatory text and the visual content, and avoid visual repetition, thereby improving the accuracy of information delivery and the viewing experience.

[0113] 3. Enhance the professionalism and naturalness of videos: Intelligent sentence segmentation makes TTS narration more in line with the rhythm of natural language; intelligent subtitle line breaking function ensures the readability of subtitles; the combination of dynamic visual effects, random transitions and adaptive background music reduces the mechanical feel of templated videos and improves the overall viewing experience.

[0114] 4. Significantly reduced operating costs: Automated processes replace editing and voice-over roles, greatly reducing labor and time costs, making it economically feasible to provide dedicated video introductions for a large number of vehicles.

[0115] 5. Ensure brand image consistency: Standardized automated processes ensure that all produced videos maintain consistency in style, narration quality, and subtitle style, which helps to establish and consolidate a unified brand image.

[0116] Specific limitations regarding the automatic generation system for vehicle introduction videos can be found in the above description of the automatic generation method for vehicle introduction videos, and will not be repeated here. Each module in the aforementioned automatic generation system for vehicle introduction videos can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0117] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned method for automatically generating a vehicle introduction video.

[0118] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored relating to all or part of the processes in the methods of the above embodiments.

[0119] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0120] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for automatically generating vehicle introduction videos, characterized in that, The method includes: Acquire multiple images of the vehicle, each image carrying a category label identifying the type of part it was photographed; Based on the image and its category labels, a multimodal AI text generation model is invoked to generate vehicle description text information corresponding to the image content; The vehicle description text information is segmented into sentences to obtain multiple explanatory sentences; the best matching image is determined for each explanatory sentence in turn to form a sequence of image-text pairs; The narration phrases are converted into audio, and the start and end timestamps of the audio corresponding to each narration phrase are obtained; the duration of the image corresponding to the narration phrase is adjusted according to the start and end timestamps of the audio corresponding to the narration phrase. Based on the explanatory short sentence, at least one subtitle line is generated through line breaking based on the length of its text content to obtain the subtitle display result; and based on the start and end timestamp information of the audio of the explanatory short sentence, the duration of the subtitle display result is generated to generate dynamic subtitles; The dynamic subtitles, the audio, and the matching image are combined to obtain a vehicle introduction video containing synchronized subtitles and voice-over.

2. The method for automatically generating vehicle introduction videos according to claim 1, characterized in that, The step of generating vehicle description text information corresponding to the image content by calling a multimodal AI text generation model based on the image and its category labels includes: Based on the category label of the image, match the corresponding descriptive dimension guide from the preset guide word library; The image and the descriptive dimension guide are input into the multimodal AI text generation model so that the multimodal AI text generation model can recognize the visual content of the image and, in combination with the descriptive dimension defined by the guide, generate vehicle description text information with industry professionalism.

3. The method for automatically generating vehicle introduction videos according to claim 1, characterized in that, The step of determining the best matching image for each explanatory phrase to form a sequence of image-text pairs includes: Based on the text group to which the current explanatory phrase belongs, images with consistent category labels are selected from the multiple images to form a candidate image set; The pre-trained multimodal semantic matching model is invoked to project the current explanatory phrase and the candidate image set into the same high-dimensional cross-modal semantic space. Calculate the cosine similarity between the feature vector of the explanatory phrase and the feature vector of each candidate image; The image with the highest similarity is selected as the initial matching image for the current explanatory phrase; if the initial matching image is the same as the matching image of the previous explanatory phrase, the image with the second highest similarity is selected as the final matching image to form a sequence of image-text pairs.

4. The method for automatically generating vehicle introduction videos according to claim 3, characterized in that, The multimodal semantic matching model is constructed using a dual-tower contrastive language-image pre-training architecture, including: Candidate images and explanatory phrases are projected into the same high-dimensional cross-modal semantic space using a visual encoder and a text encoder, respectively. A contrastive learning strategy is adopted to train the visual encoder and text encoder on a large-scale image-text pair dataset to maximize the cosine similarity of feature vectors of related image-text pairs in the semantic space, thereby achieving semantic alignment between images and text.

5. The method for automatically generating vehicle introduction videos according to claim 1, characterized in that, The vehicle description text information is segmented into sentences to obtain multiple explanatory short sentences, including: Step S51: Using regular expressions, the vehicle description text information is initially segmented based on punctuation marks in the text to obtain text fragments, and the punctuation marks are retained as delimiters for each fragment; Step S52: Initialize an empty cache region; Step S53: Append each text fragment sequentially to the end of the cache to form a temporary sentence, and perform the following judgment after each append: If the length of the temporary sentence exceeds the preset minimum length threshold for the first time, then continue to add subsequent segments until the next punctuation-type segmentation point is encountered, resulting in a new temporary sentence. If the length of the new temporary sentence exceeds the preset maximum length threshold, then revert to all content before the most recent addition and output it as an explanatory short sentence. Otherwise, directly output the temporary sentence as an explanatory short sentence and then clear the cache. Step S54: Repeat step S53 until all segments have been processed.

6. The method for automatically generating vehicle introduction videos according to claim 1, characterized in that, Based on the explanatory short sentences, and according to the length of their text content, at least one subtitle line is generated through line breaking to obtain the subtitle display result, including: Determine whether the number of characters in the explanatory phrase exceeds a preset single-line display threshold. If not, treat the explanatory phrase as a single subtitle line; if so, divide the explanatory phrase into multiple sub-text segments, each sub-text segment corresponding to a subtitle line.

7. The method for automatically generating vehicle introduction videos according to claim 1, characterized in that, Before synthesizing the dynamic subtitles, the audio, and the matching image, the following steps are also included: The display effects for the dynamic subtitles are set, including: font size, font style, and color; Configure background music for the voice audio and mix it with the audio after adjusting its volume to be lower than that of the voice audio. Randomly selected transition effects are inserted between the images.

8. An automatic generation system for vehicle introduction videos, characterized in that, The system includes: Image acquisition module: used to acquire multiple images of the vehicle, the images carrying category labels that identify the type of the captured component; Text generation module: used to generate vehicle description text information corresponding to the image content by calling a multimodal AI text generation model based on the image and its category labels; The image-text matching module is used to segment the vehicle description text information into sentences to obtain multiple explanatory sentences; and to determine the best matching image for each explanatory sentence in turn to form an image-text pair sequence. The audio-visual adjustment module is used to convert the narration phrases into audio, and obtain the start and end timestamp information of the audio corresponding to each narration phrase; and adjust the duration of the image corresponding to the narration phrase according to the start and end timestamp information of the audio corresponding to the narration phrase. The subtitle generation module is used to generate at least one subtitle line based on the length of the explanatory sentence and its text content through line breaking to obtain the subtitle display result; and to generate the duration of the subtitle display result based on the start and end timestamp information of the audio of the explanatory sentence, thereby generating dynamic subtitles. Vehicle introduction video output module: used to synthesize the dynamic subtitles, the audio, and the matching image to obtain a vehicle introduction video containing synchronized subtitles and voice-over.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images