Driving video generation method and device, electronic equipment and storage medium

Abstract

The invention provides a driving video generation method and device, electronic equipment and a storage medium, and relates to the technical field of video processing. The method comprises the following steps: acquiring an initial driving video of a target automobile in a to-be-recorded driving process and a driving diary text of the to-be-recorded driving process; based on the driving diary text, adding explanation audio and / or explanation subtitles in the initial driving video to generate a target driving video; the explanation audio is generated based on the driving diary text, and the explanation subtitles are generated based on the driving diary text; wherein the initial driving video comprises multiple frames of driving images, and any driving image is obtained by performing image acquisition on an external environment or an internal environment of the target automobile in the to-be-recorded driving process; the driving diary text is obtained by reasoning the driving record information of the to-be-recorded driving process based on the large model. According to the invention, a user does not need to manually dub or distribute subtitles, so that the convenience of driving video generation is improved.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a driving video generation method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, people have higher and higher requirements for the intelligence of cars. In people's daily driving process, especially in self-driving tours, most users will choose to share their driving videos through short video platforms. In the traditional way, most of the videos are shot manually and edited manually to obtain the driving videos that can be released. However, this requires a lot of manpower and material resources, and requires users to have video editing capabilities, so it cannot meet the user's intelligent needs. Based on this, how to intelligently generate driving videos is a technical demand that needs to be solved urgently.

[0003] At present, the voice data of passengers is collected through the voice collection device on the car, which helps users record the voice data during driving and transcribes the voice data into voice text, so that users do not need to actively recall the voice content. However, the voice text cannot be directly used as the audio text or subtitle text of the driving video, that is, the user still needs to dub or add subtitles, which makes the generation of driving videos still inconvenient. Summary of the invention

[0004] The present invention provides a driving video generation method, device, electronic device and storage medium, which are used to solve the defect that the generation of driving videos in the prior art is lack of convenience and realize intelligent generation of driving videos.

[0005] The present invention provides a driving video generation method, comprising: Acquire an initial driving video of the target vehicle during the driving process to be recorded, and a driving diary text of the driving process to be recorded; Based on the driving diary text, adding commentary audio and / or commentary subtitles to the initial driving video to generate a target driving video; the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text; The driving diary text is obtained by inferring the driving record information of the driving process to be recorded based on a large model.

[0006] According to a driving video generation method provided by the present invention, the driving diary text is determined based on the following method: Acquiring driving record information of the driving process to be recorded; Input the driving record information into a text generation model to obtain the driving diary text output by the text generation model; the text generation model is a large model; Among them, the driving record information includes at least one of time information, travel route information, weather information along the way, passenger monitoring information and multimedia application usage information; the travel route information is determined based on the navigation information of the driving process to be recorded and / or the positioning information of the target vehicle; the weather information along the way is determined based on the positioning information of the target vehicle; the passenger monitoring information is obtained by monitoring each person in the target vehicle; and the multimedia application usage information is obtained by recording the usage of multimedia applications.

[0007] According to a driving video generation method provided by the present invention, the passenger monitoring information includes monitoring information of a plurality of first preset key time points in the driving process to be recorded; and / or,

[0008] The passenger monitoring information includes at least one of emotion monitoring information, behavior monitoring information and voice recognition information.

[0009] According to a driving video generation method provided by the present invention, the initial driving video includes multiple frames of driving images, and any of the driving images is obtained by capturing images of the exterior environment or interior environment of the target vehicle during the driving process to be recorded.

[0010] According to a driving video generation method provided by the present invention, the initial driving video is generated based on the following method: Acquire multiple driving segment videos and / or multiple driving images of the target vehicle during the driving process to be recorded; any of the driving segment videos includes multiple frames of driving images; When the plurality of driving segment videos are acquired, generating the initial driving video based on the plurality of driving segment videos; When the plurality of driving images are acquired, generating the initial driving video based on the plurality of driving images; When the plurality of driving segment videos and the plurality of driving images are acquired, the initial driving video is generated based on the plurality of driving segment videos and the plurality of driving images.

[0011] According to a driving video generation method provided by the present invention, the multiple driving segment videos include driving segment videos collected at multiple second preset key time points in the driving process to be recorded; The multiple driving images include driving images captured at multiple second preset key time points in the driving process to be recorded.

[0012] According to a driving video generation method provided by the present invention, when a passenger in the target car exhibits preset behavior during the driving process to be recorded, the multiple driving clip videos include an outside-car video that captures the outside environment of the target car at the time when the preset behavior occurs, and / or the multiple driving images include outside-car images that capture the outside environment of the target car at the time when the preset behavior occurs.

[0013] According to a driving video generation method provided by the present invention, any of the vehicle exterior videos is collected based on the following method: During the driving process to be recorded, real-time monitoring of the behavior of passengers in the target car; If it is determined based on the behavior monitoring result that the passenger in the target car has a preset behavior, collecting voice data of the passenger in the target car; If a preset wake-up word is recognized based on the voice data, the external environment of the target car is collected to obtain the external video.

[0014] According to a driving video generation method provided by the present invention, after collecting the external environment of the target vehicle to obtain the external video, the method further includes: The behavior monitoring result and the voice data are recorded so that the driving record information includes the behavior monitoring result and the voice data.

[0015] According to a driving video generation method provided by the present invention, when there is a preset recording object in the external environment of the target vehicle during the driving process to be recorded, the multiple driving clip videos include videos captured of the preset recording object, and / or the multiple driving images include images captured of the preset recording object.

[0016] According to a driving video generation method provided by the present invention, based on the driving diary text, adding commentary audio and / or commentary subtitles to the initial driving video to generate a target driving video includes: In the case where there is real person voice feature data, speech synthesis is performed on the driving diary text based on the real person voice feature data to obtain commentary audio, and the commentary audio is added to the initial driving video to generate a target driving video; the real person voice feature data is generated based on real person voice data collected from the user; In the absence of real-person voice feature data, speech synthesis is performed on the driving diary text based on preset sound feature data to obtain commentary audio, which is added to the initial driving video to generate a target driving video, and / or, based on the driving diary text, commentary subtitles are added to the initial driving video to generate a target driving video.

[0017] According to a driving video generation method provided by the present invention, the adding of the commentary audio to the initial driving video to generate a target driving video comprises: Inputting the initial driving video and the commentary audio into a video generation model to obtain a target driving video output by the video generation model; The video generation model is a large model, and the video generation model is used to match the timestamp of the initial driving video with the timestamp of the commentary audio.

[0018] The present invention also provides a driving video generating device, comprising: An acquisition module, used to acquire an initial driving video of the target vehicle during the driving process to be recorded, and a driving diary text of the driving process to be recorded; A generating module, configured to add commentary audio and / or commentary subtitles to the initial driving video based on the driving diary text, so as to generate a target driving video; the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text; The driving diary text is obtained by inferring the driving record information of the driving process to be recorded based on a large model.

[0019] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the driving video generating method described in any one of the above methods is implemented.

[0020] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the driving video generation method as described in any one of the above is implemented.

[0021] The driving video generation method, device, electronic device and storage medium provided by the present invention obtain the initial driving video of the target car in the driving process to be recorded, and the driving diary text of the driving process to be recorded, and the driving diary text is obtained by reasoning the driving record information of the driving process to be recorded based on a large model, so that the driving record information recorded in the driving process can be intelligently inferred to obtain the driving diary text by using the large model, without the need for users to write the driving diary text by themselves, thereby improving the convenience of driving video generation. At the same time, the driving diary text can be directly used as the audio text or subtitle text of the initial driving video, so that based on the driving diary text, commentary audio and / or commentary subtitles are automatically added to the initial driving video to generate a target driving video with matching audio content and video content, or to generate a target driving video with matching subtitles and video content, so that under the premise of ensuring the accuracy of driving video generation, there is no need for users to manually dub or add subtitles, thereby improving the convenience of driving video generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 This is one of the flow charts of the driving video generation method provided by the present invention.

[0024] Figure 2 This is the second flow chart of the driving video generation method provided by the present invention.

[0025] Figure 3 It is a structural schematic diagram of the driving video generating device provided by the present invention.

[0026] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] With the rapid development of science and technology, people have higher and higher requirements for the intelligence of cars. In people's daily driving process, especially in self-driving tours, most users will choose to share their driving videos through short video platforms, such as sharing their travel diaries during self-driving trips. In the traditional way, most of them are manually shot and manually edited to obtain the final publishable driving video, which includes the cutting and merging of multiple videos, as well as subtitles or dubbing, but this requires a lot of manpower and material resources, that is, the production of driving videos requires users to spend a lot of energy and time. For example, after the trip, it takes a lot of time to summarize the materials, and it is necessary to recall the content, select the content, create the content, etc.; and users are required to have video editing capabilities, so it cannot meet the user's intelligent needs. Based on this, how to intelligently generate driving videos is a technical demand that needs to be solved urgently.

[0029] Before dubbing or subtitles are added, a driving diary text is usually generated first. Existing driving diary texts are generally pre-installed in text editing software on the car computer or mobile phone, and users actively enter content by manually entering text, which is a traditional electronic diary recording method. However, this method requires users to actively enter text or adjust and modify the text, which lacks convenience.

[0030] At present, the voice data of passengers is collected through the voice collection device on the car, which helps users record the voice data during driving and transcribes the voice data into voice text, so that users do not need to actively recall the voice content. However, the voice text cannot be directly used as the audio text or subtitle text of the driving video, that is, the user still needs to dub or add subtitles, which makes the generation of driving videos still inconvenient.

[0031] Although there are technologies such as voice transcription and role separation that actively or passively transcribe the voice statements and conversations of users in the car, and when encountering multiple people, distinguish the roles and mark them with text to finally extract the voice text, it still cannot be directly used as the audio text or subtitle text of the driving video, and thus cannot automatically generate the driving video that can be finally published, resulting in the lack of convenience in the generation of driving videos.

[0032] In view of the above problems, the present invention proposes the following embodiments. Figure 1-Figure 2 The driving video generation method of the present invention is described.

[0033] Figure 1 is one of the flow charts of the driving video generation method provided by the present invention, such as Figure 1 As shown, the driving video generation method includes the following steps 110 and 120.

[0034] Step 110, obtaining an initial driving video of the target vehicle during the driving process to be recorded, and a driving diary text of the driving process to be recorded.

[0035] The driving diary text is obtained by inferring the driving record information of the driving process to be recorded based on a large model.

[0036] In the embodiment of the present invention, the execution subject of the driving video generation method can be a user terminal (such as a smart phone, a tablet computer, a desktop computer, a laptop computer, etc.), or a vehicle computer of the target car, or a server, etc., which is not specifically limited here.

[0037] Here, the target car is the car corresponding to the driving video to be generated, and the user completes the driving process to be recorded based on the target car. The driving process to be recorded is the trip corresponding to the driving video to be generated, that is, the trip to be recorded, for example, the driving process of the car from the departure point to the destination.

[0038] Here, the initial driving video is a video recorded by the target car during the driving process to be recorded. In one embodiment, the initial driving video is a complete video shot from the beginning to the end of the driving process to be recorded. In another embodiment, the initial driving video is generated based on multiple driving segment videos and / or multiple driving images recorded by the target car during the driving process to be recorded; any driving segment video includes multiple frames of driving images.

[0039] In one embodiment, the initial driving video or driving image can be acquired by a collection device carried by the target vehicle, thereby eliminating the need for the user to shoot the initial driving video by himself, thereby improving the convenience of generating the driving video.

[0040] For example, the acquisition device is a dual-view camera at the rearview mirror, so that real-time front and rear view video recording or image acquisition can be performed, that is, the external environment or the internal environment of the target car can be acquired. Of course, the acquisition device can also be other cameras built into the target car, so there is no need to increase hardware costs.

[0041] Here, the driving diary text is a text content, which is the diary text corresponding to the driving process to be recorded. The driving record information is the information recorded during the driving process to be recorded, and the driving record information may include information of multiple dimensions; for example, the driving record information may include but is not limited to at least one of the following: time information, itinerary route information, weather information along the way, passenger monitoring information, and multimedia application usage information, etc.

[0042] In one embodiment, the driving record information can be obtained by recording the target car, so that the user does not need to record it by himself, and no other additional equipment is needed for recording, thereby improving the convenience of generating driving videos.

[0043] It should be noted that since the driving diary text is obtained by inferring the driving record information based on the large model, the driving diary text can be directly used as the audio text or subtitle text of the driving video, so that the user does not need to manually dub or add subtitles, thereby improving the convenience of driving video generation. For example, the large model is the Spark large model.

[0044] Furthermore, the driving diary text is obtained by inferring the driving record information and the initial driving video based on the large model. Since the driving diary text is obtained by inferring the driving record information and the initial driving video based on the large model, the driving diary text can be directly used as the audio text or subtitle text of the initial driving video, thereby eliminating the need for users to manually dub or add subtitles, thereby improving the convenience of driving video generation.

[0045] Furthermore, the driving diary text can be recorded from a first-person perspective. For example, the driving diary text is "Today is Sunday, June 16, 2024. The weather is nice today, with a breeze, so I drove to Huangshan. I was very excited when I set off. I drove while listening to singer A's songs, ..."

[0046] Furthermore, the number of words in the driving diary text may be limited, for example, the number of words in the driving diary text ranges from 50 to 200 words.

[0047] Furthermore, the driving record information and the initial driving video can be collected and recorded only after the user sets the driving record mode (such as the self-driving travel mode) to be turned on, and the collected raw data can be processed only after the driving process to be recorded is completed.

[0048] Step 120 , based on the driving diary text, add commentary audio and / or commentary subtitles to the initial driving video to generate a target driving video.

[0049] Among them, the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text.

[0050] Here, the audio commentary is directly generated based on the driving diary text, that is, the text content of the audio commentary is the driving diary text; the subtitles are directly generated based on the driving diary text, that is, the text content of the subtitles is the driving diary text. Based on this, the initial driving video is automatically dubbed or subtitled, thereby improving the convenience of driving video generation.

[0051] Here, the target driving video is a driving video with dubbing or subtitles. The target driving video can be used as a Vlog diary. In one embodiment, the target driving video has narration audio. In another embodiment, the target driving video has narration subtitles. In another embodiment, the target driving video has both narration audio and narration subtitles.

[0052] In one embodiment, the driving diary text is voice synthesized based on real person voice feature data to obtain commentary audio; the real person voice feature data is generated based on real person voice data collected from the user. By collecting the real person voice data of the user, real person voice feature data with the user's voice features is generated, and then using voice synthesis technology, the commentary audio with the user's voice is reproduced, thereby obtaining a more vivid target driving video and improving the user experience.

[0053] In another embodiment, the driving diary text is voice synthesized based on preset sound feature data to obtain the commentary audio. The preset sound feature data is pre-set sound feature data, so that the commentary audio with standard timbre (preset timbre) is obtained by using speech synthesis technology.

[0054] In a specific embodiment, the initial driving video and the commentary audio are input into the video generation model to obtain the target driving video output by the video generation model. The video generation model is a large model, and the video generation model is used to match the timestamp of the initial driving video with the timestamp of the commentary audio, thereby using the reasoning ability of the large model to match the commentary content of the commentary audio with the video content of the initial driving video, and read out the commentary audio at a moderate speed and insert it into the initial driving video to ensure the accuracy of the intelligent dubbing, thereby improving the accuracy of the generation of the target driving video, and further improving the user experience.

[0055] In a specific embodiment, the initial driving video and the captions are input into the video generation model to obtain the target driving video output by the video generation model. The video generation model is a large model, and the video generation model is used to match the timestamp of the initial driving video with the timestamp of the captions, thereby using the reasoning ability of the large model to match the captions of the captions with the video content of the initial driving video, and displaying the captions at a moderate speed in the screen of the initial driving video to ensure the accuracy of the intelligent captioning, thereby improving the accuracy of the generation of the target driving video, and further improving the user experience.

[0056] Furthermore, the generated target driving video is automatically pushed to the user client, and the user can publish it after further review, thereby improving convenience and user experience. And the target driving video can be released after being accompanied by pre-set background music.

[0057] The driving video generation method provided by the embodiment of the present invention obtains the initial driving video of the target car in the driving process to be recorded, and the driving diary text of the driving process to be recorded, and the driving diary text is obtained by reasoning the driving record information of the driving process to be recorded based on the big model, so that the driving record information recorded in the driving process can be intelligently inferred to obtain the driving diary text by using the big model, without the need for the user to write the driving diary text by themselves, thereby improving the convenience of driving video generation. At the same time, the driving diary text can be directly used as the audio text or subtitle text of the initial driving video, so that based on the driving diary text, commentary audio and / or commentary subtitles are automatically added to the initial driving video to generate a target driving video with matching audio content and video content, or to generate a target driving video with matching subtitles and video content, thereby ensuring the accuracy of driving video generation without the need for the user to manually dub or add subtitles, thereby improving the convenience of driving video generation.

[0058] Based on any of the above embodiments, in this method, the driving diary text is determined based on the following method: Acquiring driving record information of the driving process to be recorded; The driving record information is input into a text generation model to obtain the driving diary text output by the text generation model; the text generation model is a large model.

[0059] In one embodiment, the driving diary text may be determined by an execution subject of the driving video generation method. In another embodiment, the driving diary text may be determined by other devices; for example, the driving diary text is determined by the target vehicle, that is, the target vehicle obtains the driving record information of the driving process to be recorded, inputs the driving record information into the text generation model, and obtains the driving diary text output by the text generation model, that is, the text generation model is a large vehicle-mounted model.

[0060] Among them, the driving record information includes at least one of time information, travel route information, weather information along the way, passenger monitoring information and multimedia application usage information; the travel route information is determined based on the navigation information of the driving process to be recorded and / or the positioning information of the target vehicle; the weather information along the way is determined based on the positioning information of the target vehicle; the passenger monitoring information is obtained by monitoring each person in the target vehicle; and the multimedia application usage information is obtained by recording the usage of multimedia applications.

[0061] Here, the time information is used to characterize the travel time of the driving process to be recorded. In one embodiment, the time information includes the travel date of the driving process to be recorded, such as Sunday, June 16, 2024. In one embodiment, the time information includes the recording time of other driving record information.

[0062] Here, the itinerary route information may include but is not limited to at least one of the following: departure place information, destination information, scenic spot (scenic area) information to be recorded during the driving process, service area information, gas station information, number of visits to the destination, highway information to be recorded during the driving process, national highway information to be recorded during the driving process, city information (including city, district, county, etc.) to be recorded during the driving process, number of cities to be recorded during the driving process, length of stay in the service area, number of stays in the service area, etc. Among them, the scenic spot (scenic area) information to be recorded during the driving process may include the name of the scenic spot and the introduction information of the scenic spot.

[0063] In one embodiment, the travel route information can be directly obtained through the navigation information, or the travel route information can be indirectly obtained through the navigation information. Exemplarily, data embedding is performed in the onboard navigation software of the target car to obtain the navigation information.

[0064] In one embodiment, the travel route information can be directly obtained through the positioning information, or the travel route information can be indirectly obtained through the positioning information. Exemplarily, data points are buried in the onboard navigation software of the target car to obtain the positioning information to monitor the movement of the target car.

[0065] Here, the weather information along the way includes the weather information along the way of the driving process to be recorded. Through the positioning information of the target car, the position of the target car at each time point can be monitored in real time, and then the weather information along the way can be obtained based on each position.

[0066] Here, the passenger monitoring information is obtained by monitoring each person in the target car (including the driver and other passengers). The passenger monitoring information may include but is not limited to at least one of the following: emotion monitoring information, behavior monitoring information, and voice recognition information, etc.

[0067] In one embodiment, the passenger monitoring information can be obtained by monitoring the DMS (Driver Monitor System) or OMS (Occupancy Monitor System) of the target car, so that the user does not need to record it by himself, and no other additional equipment is required for recording, which improves the convenience of generating driving videos. For example, the passenger is monitored using the in-car camera device (such as an infrared sensor camera) of the target car.

[0068] Here, the multimedia application usage information is obtained by recording the usage of the multimedia application. The multimedia application may include but is not limited to at least one of the following: music player application, radio station, online video application and karaoke software, etc.

[0069] Here, the multimedia application usage information may include but is not limited to at least one of the following: a playlist played during the driving process to be recorded, radio station information played during the driving process to be recorded, video information played during the driving process to be recorded, karaoke information during the driving process to be recorded, usage time of each multimedia application, operation time of each multimedia application, singer information most frequently listened to during the driving process to be recorded, song information most frequently listened to during the driving process to be recorded, radio station information most frequently listened to during the driving process to be recorded, video information most frequently watched during the driving process to be recorded, etc. It should be understood that the multimedia application usage information is synchronously recorded with corresponding time information.

[0070] Exemplarily, data embedding is performed in each multimedia application to obtain multimedia application usage information of the driving process to be recorded.

[0071] In one embodiment, the multimedia application usage information may be usage information of some original applications of the target vehicle, thereby eliminating the need for the user to record the information by themselves and eliminating the need for other additional equipment to record the information, thereby improving the convenience of generating driving videos.

[0072] Furthermore, the driving record information and the initial driving video are input into the text generation model to obtain the driving diary text output by the text generation model; based on this, since the driving diary text is obtained by inferring the driving record information and the initial driving video based on the large model, the driving diary text can be directly used as the audio text or subtitle text of the initial driving video, thereby eliminating the need for users to manually dub or add subtitles, thereby improving the convenience of driving video generation.

[0073] The driving video generation method provided by the embodiment of the present invention, the driving diary text is obtained by inferring driving record information of various dimensions based on a large model, so that the driving record information of various dimensions can be intelligently inferred by the large model to obtain a more accurate and complete driving diary text, without the need for users to write the driving diary text themselves, thereby improving the convenience of driving video generation. At the same time, the more accurate and complete driving diary text can be directly used as the audio text or subtitle text of the initial driving video, so that based on the driving diary text, commentary audio and / or commentary subtitles are automatically added to the initial driving video, thereby improving the convenience of driving video generation, and improving the generation accuracy of the driving diary text, and ultimately improving the generation accuracy of the driving video.

[0074] Based on any of the above embodiments, in the method, it is considered that if passenger monitoring is performed for the entire driving process to be recorded, the passenger monitoring information will be too redundant; based on this, the passenger monitoring information includes monitoring information of multiple first preset key time points in the driving process to be recorded.

[0075] Here, multiple first preset key time points can be set according to actual needs. For example, the multiple first preset key time points may include but are not limited to at least one of the following: the departure time when the driving process to be recorded just starts, the time after the first preset duration of the journey, the time approaching the scenic spot along the way, the time when the journey is halfway through, the time when the vehicle is close to the destination and less than the first preset distance, the time when the vehicle stops after arriving at the destination, the server stay time, etc.; for example, the first preset duration is 15 minutes, and the first preset distance is 100 meters.

[0076] For example, by monitoring passengers in the above manner, it is captured that at the beginning of the journey, the driver's mood is happy + talking, fifteen minutes into the journey, the mood is calm, halfway through the journey, he is slightly tired, and he is in a good mood after arrival, etc.

[0077] The driving video generation method provided by the embodiment of the present invention, the passenger monitoring information includes monitoring information of multiple first preset key time points in the driving process to be recorded, so that there is no need to monitor the passengers for the entire driving process to be recorded, ensuring that the obtained passenger monitoring information is the monitoring information of the key time points, that is, obtaining more representative passenger monitoring information, thereby improving the generation accuracy of the driving diary text, and ultimately improving the generation accuracy of the driving video.

[0078] Based on any of the above embodiments, in the method, the passenger monitoring information includes at least one of emotion monitoring information, behavior monitoring information and voice recognition information.

[0079] Here, the emotion monitoring information may be happiness, surprise, anger, sadness or calmness, etc. The behavior monitoring information may be conversation behavior monitoring, fatigue monitoring information, etc. The voice recognition information is information obtained by performing voice recognition on the person in the target car, for example, the scenery-praising sentences such as "the scenery is beautiful", "the scenery outside the window is beautiful", and "the scenery outside is beautiful" are detected through voice recognition.

[0080] In the driving video generation method provided by an embodiment of the present invention, the passenger monitoring information includes at least one of emotion monitoring information, behavior monitoring information and voice recognition information, so as to monitor passengers from various dimensions, improve the comprehensiveness and accuracy of the passenger monitoring information, and then improve the generation accuracy of the driving diary text, and finally improve the generation accuracy of the driving video.

[0081] Based on any of the above embodiments, in the method, the initial driving video includes multiple frames of driving images, and any of the driving images is obtained by capturing images of the exterior environment or interior environment of the target vehicle during the driving process to be recorded.

[0082] It should be noted that by collecting images or videos of the environment outside the car, it is possible to record the scenery outside the car along the way during the driving process to be recorded; by collecting images or videos of the environment inside the car, it is possible to record the behavior status of the passengers in the car (including the driver), thereby making the generated target driving video more colorful, and providing more video scenes for matching with the driving diary text, thereby improving the generation effect of the driving video and thus improving the user experience.

[0083] The driving video generation method provided by the embodiment of the present invention comprises an initial driving video including multiple frames of driving images, and any driving image is obtained by capturing images of the external environment or the internal environment of the target vehicle during the driving process to be recorded. Thus, the initial driving video may include not only the video of the external environment but also the video of the internal environment, thereby making the generated target driving video more colorful and providing more video scenes for matching with the driving diary text, thereby improving the generation effect of the driving video and further improving the user experience.

[0084] Based on any of the above embodiments, Figure 2 FIG. 2 is a flow chart of the method for generating a driving video provided by the present invention. Figure 2 As shown, the initial driving video is generated based on the following method.

[0085] Step 210: Acquire a plurality of driving segment videos and / or a plurality of driving images of the target vehicle during the driving process to be recorded.

[0086] Wherein, any of the driving clip videos includes multiple frames of driving images. The driving clip video can be an exterior video of the target vehicle, or an interior video of the target vehicle. The driving image can be an interior image of the target vehicle, or an exterior image of the target vehicle.

[0087] In one embodiment, the initial driving video may be generated by an execution subject of the driving video generation method. In another embodiment, the initial driving video may be generated by other devices; for example, the initial driving video is generated by a target car.

[0088] In one embodiment, the driving footage video or driving image can be acquired by a capture device carried by the target vehicle, thereby eliminating the need for the user to shoot the driving footage video by themselves, thereby improving the convenience of generating driving videos.

[0089] It should be noted that the above method can be used to collect multiple driving segment videos and / or multiple driving images of the target car in the driving process to be recorded in segments, without the need to collect a complete video of the driving process to be recorded from beginning to end, thereby avoiding video redundancy, reducing video data, and improving processing efficiency.

[0090] In one embodiment, the plurality of driving segment videos include driving segment videos captured at a plurality of second preset time points during the driving process to be recorded.

[0091] In one embodiment, the plurality of driving images include driving images captured at a plurality of second preset time points during the driving process to be recorded, wherein one preset time point may capture one or more driving images.

[0092] In one embodiment, when a passenger in a target car has a preset behavior during the driving process to be recorded, the multiple driving clip videos include an outside-car video of the target car's outside environment captured at the time when the preset behavior occurs, and / or an inside-car video of the target car's inside environment captured at the time when the preset behavior occurs. Furthermore, during the driving process to be recorded, the behavior monitoring results of the passengers in the target car are monitored in real time; if it is determined based on the behavior monitoring results that the passenger in the target car has a preset behavior, the voice data of the passenger in the target car is collected; if a preset wake-up word is recognized based on the voice data, the outside environment of the target car is collected to obtain an outside video, and / or the inside environment of the target car is collected to obtain an inside video.

[0093] In one embodiment, when a passenger in a target car has a preset behavior during the driving process to be recorded, the multiple driving images include an exterior image of the target car's exterior environment captured at the time when the preset behavior occurs, and / or an interior image of the target car's interior environment captured at the time when the preset behavior occurs. Further, during the driving process to be recorded, the behavior monitoring results of the passengers in the target car are monitored in real time; if it is determined based on the behavior monitoring results that the passenger in the target car has a preset behavior, the voice data of the passenger in the target car is collected; if a preset wake-up word is recognized based on the voice data, the exterior environment of the target car is collected to obtain an exterior image, and / or the interior environment of the target car is collected to obtain an interior image.

[0094] In one embodiment, during the driving process to be recorded, the voice data of the passengers in the target car are collected in real time. If a preset wake-up word is recognized based on the voice data, the target car's external environment is collected to obtain an external video, and / or the target car's internal environment is collected to obtain an internal video. Further, during the driving process to be recorded, the voice data of the passengers in the target car are collected in real time; if a preset wake-up word is recognized based on the voice data, the behavior monitoring results of the passengers in the target car are monitored. If it is determined based on the behavior monitoring results that the passengers in the target car have a preset behavior, the target car's external environment is collected to obtain an external video, and / or the target car's internal environment is collected to obtain an internal video.

[0095] In one embodiment, during the driving process to be recorded, the voice data of the passengers in the target car are collected in real time. If a preset wake-up word is recognized based on the voice data, the target car's external environment is collected to obtain an external image, and / or the target car's internal environment is collected to obtain an internal image. Further, during the driving process to be recorded, the voice data of the passengers in the target car are collected in real time; if a preset wake-up word is recognized based on the voice data, the behavior monitoring results of the passengers in the target car are monitored. If it is determined based on the behavior monitoring results that the passengers in the target car have a preset behavior, the target car's external environment is collected to obtain an external image, and / or the target car's internal environment is collected to obtain an internal image.

[0096] Furthermore, while collecting driving videos and driving images, the behavior monitoring results or voice data that trigger the collection are also recorded, so that the driving record information includes the behavior monitoring results and voice data.

[0097] In one embodiment, when there is a preset recording object in the external environment of the target vehicle during the driving process to be recorded, the multiple driving clip videos include videos captured of the preset recording object, and / or the multiple driving images include images captured of the preset recording object.

[0098] Step 220: When the plurality of driving segment videos are acquired, generate the initial driving video based on the plurality of driving segment videos.

[0099] In one embodiment, the multiple driving segment videos are synthesized into an initial driving video according to the acquisition time of the multiple driving segment videos.

[0100] In another embodiment, multiple driving video clips are input into the video editing model to obtain an initial driving video output by the video editing model. Furthermore, the video editing model is a large model, so that a more appropriate initial driving video is intelligently generated by the large model, that is, the large model has video editing capabilities, so that the user does not need to manually edit the video, thereby improving the convenience of generating driving videos.

[0101] Step 230: When the plurality of driving images are acquired, the initial driving video is generated based on the plurality of driving images.

[0102] In one embodiment, the multiple driving images are synthesized into an initial driving video according to the acquisition time of the multiple driving images.

[0103] In another embodiment, multiple driving images are input into the video editing model to obtain an initial driving video output by the video editing model. Furthermore, the video editing model is a large model, so that a more appropriate initial driving video is intelligently generated by the large model, that is, the large model has video editing capabilities, so that the user does not need to manually edit the video, thereby improving the convenience of generating driving videos.

[0104] Step 240 : When the plurality of driving segment videos and the plurality of driving images are acquired, the initial driving video is generated based on the plurality of driving segment videos and the plurality of driving images.

[0105] In one embodiment, the multiple driving segment videos and the multiple driving images are synthesized into an initial driving video according to the acquisition time of the multiple driving segment videos and the multiple driving images.

[0106] In another embodiment, multiple driving video clips and multiple driving images are input into the video editing model to obtain an initial driving video output by the video editing model. Furthermore, the video editing model is a large model, so that a more appropriate initial driving video is intelligently generated by the large model, that is, the large model has video editing capabilities, so that the user does not need to manually edit the video, thereby improving the convenience of generating driving videos.

[0107] The driving video generation method provided by the embodiment of the present invention can automatically generate an initial driving video based on multiple driving clip videos and / or multiple driving images of the target car in the driving process to be recorded in the above-mentioned manner, thereby eliminating the need for users to edit videos by themselves, thereby improving the convenience of driving video generation; at the same time, multiple driving clip videos and / or multiple driving images of the target car in the driving process to be recorded can be collected in segments, without the need to collect a complete video of the driving process to be recorded from beginning to end, thereby avoiding video redundancy, reducing video data, and improving processing efficiency.

[0108] Based on any of the above embodiments, considering that if video capture or image capture is performed for the entire driving process to be recorded, the video or image will be too redundant; based on this, the multiple driving clip videos include driving clip videos captured at multiple second preset key time points in the driving process to be recorded; the multiple driving images include driving images captured at multiple second preset key time points in the driving process to be recorded.

[0109] Here, multiple second preset key time points can be set according to actual needs. For example, the multiple second preset key time points may include but are not limited to at least one of the following: the departure time when the driving process to be recorded just starts, the time after the first preset duration of the journey, the time approaching the scenic spot along the way, the time when the journey is halfway through, the time when the vehicle is close to the destination and less than the first preset distance, the time when the vehicle stops after arriving at the destination, the server stay time, etc.; for example, the first preset duration is 15 minutes, and the first preset distance is 100 meters.

[0110] Furthermore, the duration of the driving video clip collected at any second preset key time point is not less than the preset duration, for example, the preset duration is 5 seconds. The number of driving images collected at any second preset key time point is not less than the preset number, for example, the preset number is 5.

[0111] The driving video generation method provided by the embodiment of the present invention comprises multiple driving clip videos including driving clip videos collected at multiple second preset key time points in the driving process to be recorded; and multiple driving images including driving images collected at multiple second preset key time points in the driving process to be recorded, thereby eliminating the need to perform video or image collection on the entire driving process to be recorded, thereby ensuring that the obtained driving clip videos or driving images are information of key time points, that is, obtaining more representative video information, thereby improving the generation accuracy of the initial driving video, and ultimately improving the generation accuracy of the target driving video.

[0112] Based on any of the above embodiments, in addition to collecting driving clip videos or driving images at preset key time points, when the passengers in the target car exhibit preset behaviors during the driving process to be recorded, the multiple driving clip videos include exterior videos of the target car's exterior environment collected at the time when the preset behaviors occur, and / or the multiple driving images include exterior images of the target car's exterior environment collected at the time when the preset behaviors occur.

[0113] Here, the preset behavior can be set according to actual needs, for example, the preset behavior is that the user looks out the window. Specifically, the behavior monitoring results of the passengers in the target car are monitored in real time; and whether the passengers have the preset behavior is determined based on the behavior monitoring results.

[0114] In a specific embodiment, a passenger video in a target car is collected in real time, and behavior monitoring is performed based on the passenger video to obtain a behavior monitoring result. For example, by tracking the passenger's line of sight, it is monitored whether the user looks out the window.

[0115] The driving video generation method provided by the embodiment of the present invention, through the above-mentioned manner, when the passengers in the target car have preset behaviors during the driving process to be recorded, the external environment of the target car is also collected at the time when the preset behaviors occur to obtain the external video or external image, so that the video or image corresponding to the important behavior of the passenger can be automatically collected, without the need for the user to manually collect the video or image, and without the user to recall which moments need to be recorded, and important information can be automatically collected, thereby avoiding the omission of important external environment information, thereby improving the generation accuracy of the initial driving video, and ultimately improving the generation accuracy of the target driving video.

[0116] Based on any of the above embodiments, in addition to collecting driving video clips or driving images at preset key time points, voice data of passengers in the target car are collected during the driving process to be recorded; if a preset wake-up word is recognized based on the voice data, the external environment of the target car is collected to obtain the external video, and / or the external environment of the target car is collected to obtain the external image.

[0117] The driving video generation method provided by the embodiment of the present invention, through the above-mentioned manner, when the passenger in the target car says a preset wake-up word during the driving process to be recorded, the external environment of the target car is also collected to obtain an external video or external image, so that the video or image corresponding to the important sentence of the passenger can be automatically collected, without the need for the user to manually collect the video or image, and without the user to recall which moments need to be recorded, important information can be automatically collected, thereby avoiding the omission of important external environment information, thereby improving the generation accuracy of the initial driving video, and ultimately improving the generation accuracy of the target driving video.

[0118] Based on any of the above embodiments, any of the external vehicle videos is collected based on the following method: During the driving process to be recorded, real-time monitoring of the behavior of passengers in the target car; If it is determined based on the behavior monitoring result that the passenger in the target car has a preset behavior, collecting voice data of the passenger in the target car; If a preset wake-up word is recognized based on the voice data, the external environment of the target car is collected to obtain the external video.

[0119] In a specific embodiment, a passenger video in a target car is collected in real time, and behavior monitoring is performed based on the passenger video to obtain a behavior monitoring result. For example, by tracking the passenger's line of sight, it is monitored whether the user looks out the window.

[0120] In a specific embodiment, voice recognition is performed on the voice data to obtain voice text, and it is determined whether the voice text includes a preset wake-up word.

[0121] Here, the preset wake-up word can be set according to actual needs, for example, "The scenery is beautiful", "The scenery outside the window is beautiful", "The scenery outside is beautiful", etc.

[0122] For example, when the user is detected looking out the window and saying "the view is beautiful", "the view outside the window is beautiful", and "the view outside is beautiful", the video outside the car is captured in time.

[0123] The driving video generation method provided by the embodiment of the present invention, through the above-mentioned method, collects the passenger's voice data only when the passenger in the target car exhibits preset behavior, thereby eliminating the need to monitor in real time whether there is a preset wake-up word; at the same time, in order to avoid mistakenly collecting unimportant videos outside the car, the video outside the car is collected only when the passenger simultaneously performs the preset behavior and says the preset wake-up word, thereby improving the generation accuracy of the initial driving video, and ultimately improving the generation accuracy of the target driving video.

[0124] Based on any of the above embodiments, any of the vehicle exterior images is collected based on the following method: During the driving process to be recorded, real-time monitoring of the behavior of passengers in the target car; If it is determined based on the behavior monitoring result that the passenger in the target car has a preset behavior, collecting voice data of the passenger in the target car; If a preset wake-up word is recognized based on the voice data, the external environment of the target car is collected to obtain the external image.

[0125] Exemplarily, when the user is detected looking out the window and saying "the scenery is beautiful", "the scenery outside the window is beautiful", and "the scenery outside is beautiful", the image outside the vehicle is captured in time.

[0126] Furthermore, the behavior monitoring results and voice data are recorded so that the driving record information includes the behavior monitoring results and voice data, so that the important behavior monitoring results and voice data can be recorded synchronously to make the driving record information more complete, thereby improving the accuracy of the generation of driving diary text, and ultimately improving the accuracy of the generation of driving videos.

[0127] The driving video generation method provided by the embodiment of the present invention, through the above-mentioned method, collects the passenger's voice data only when the passenger in the target car exhibits preset behavior, thereby eliminating the need to monitor in real time whether there is a preset wake-up word; at the same time, in order to avoid mistakenly collecting unimportant images outside the vehicle, the image outside the vehicle is collected only when the passenger simultaneously performs the preset behavior and says the preset wake-up word, thereby improving the generation accuracy of the initial driving video, and ultimately improving the generation accuracy of the target driving video.

[0128] Based on any of the above embodiments, after collecting the external environment of the target vehicle to obtain the external video, the method further includes: The behavior monitoring result and the voice data are recorded so that the driving record information includes the behavior monitoring result and the voice data.

[0129] The driving video generation method provided in the embodiment of the present invention can synchronously record important behavior monitoring results and voice data in the above manner to make the driving record information more complete, thereby improving the generation accuracy of the driving diary text and ultimately improving the generation accuracy of the driving video.

[0130] Based on any of the above embodiments, in addition to collecting driving clip videos or driving images at preset key time points, when there is a preset recording object in the external environment of the target vehicle during the driving process to be recorded, the multiple driving clip videos include videos collected of the preset recording object, and / or the multiple driving images include images collected of the preset recording object.

[0131] Here, the preset recording objects can be set according to actual needs, for example, mountains, rivers, lakes, grasslands, roads, ancient buildings, tunnels, rain, animals, traffic lights, service areas, etc.

[0132] In a specific embodiment, an image or video of the external environment of the target vehicle is collected in real time, and based on the image or video of the external environment, it is identified whether there is a preset recording object. Further, the image or video of the external environment is identified using a large vehicle model to determine whether there is a preset recording object.

[0133] The driving video generation method provided by the embodiment of the present invention, through the above-mentioned manner, when there is a preset recording object in the external environment of the target car in the driving process to be recorded, the preset recording object is also collected to obtain the external video or external image, so that important videos or images can be automatically collected without the user manually collecting videos or images, and without the user recalling which objects need to be recorded, important information can be automatically collected, thereby avoiding missing important external environment information, thereby improving the generation accuracy of the initial driving video, and ultimately improving the generation accuracy of the target driving video.

[0134] Based on any of the above embodiments, in the method, the above step 120 includes: In the case where there is real person voice feature data, speech synthesis is performed on the driving diary text based on the real person voice feature data to obtain commentary audio, and the commentary audio is added to the initial driving video to generate a target driving video; the real person voice feature data is generated based on real person voice data collected from the user; In the absence of real-person voice feature data, speech synthesis is performed on the driving diary text based on preset sound feature data to obtain commentary audio, which is added to the initial driving video to generate a target driving video, and / or, based on the driving diary text, commentary subtitles are added to the initial driving video to generate a target driving video.

[0135] Here, the real person voice data can be recorded in advance by the user, or it can be the driver's voice data or passenger's voice data collected during the driving process to be recorded. Based on the real person's voice feature data, the driving diary text voice can be synthesized into a commentary audio with the user's timbre, that is, the commentary audio is the same as the recorded user's real voice data. Based on this, by collecting the user's real voice data, real person voice feature data with the user's voice characteristics is generated, and then using speech synthesis technology, the commentary audio with the user's timbre is reproduced, thereby obtaining a more vivid target driving video and improving the user experience.

[0136] Here, the preset sound feature data is pre-set sound feature data, and the preset sound feature data can be generated based on the preset voice data. Further, the preset voice data can be standard voice data. Based on the pre-examination sound feature data, the driving diary text can be voice synthesized into a narration audio with a preset timbre, that is, the narration audio with a preset timbre (standard timbre) is obtained by using the speech synthesis technology.

[0137] In one embodiment, in the absence of real person voice feature data, speech synthesis is performed on the driving diary text based on preset voice feature data to obtain commentary audio, and the commentary audio is added to the initial driving video to generate the target driving video.

[0138] In another embodiment, in the absence of real person voice feature data, commentary subtitles are added to the initial driving video based on the driving diary text to generate a target driving video.

[0139] In another embodiment, in the absence of real-person voice feature data, speech synthesis is performed on the driving diary text based on preset sound feature data to obtain commentary audio, and the commentary audio is added to the initial driving video. Moreover, based on the driving diary text, commentary subtitles are added to the initial driving video, and finally a target driving video is generated.

[0140] The driving video generation method provided by the embodiment of the present invention, in the presence of real person voice feature data, performs speech synthesis on the driving diary text based on the real person voice feature data to obtain commentary audio, adds the commentary audio to the initial driving video, and generates a target driving video, and the real person voice feature data is generated based on the real person voice data collected from the user, thereby reproducing the commentary audio with the user's timbre, thereby obtaining a more vivid and vivid target driving video, thereby improving the user experience; even in the absence of real person voice feature data, it is possible to perform speech synthesis on the driving diary text based on preset sound feature data to obtain commentary audio, or, based on the driving diary text, add commentary subtitles to the initial driving video to generate the target driving video, thereby ensuring the generation of a target driving video with a driving diary, thereby improving the user experience.

[0141] Based on any of the above embodiments, in the method, adding the commentary audio to the initial driving video to generate a target driving video includes: The initial driving video and the commentary audio are input into a video generation model to obtain a target driving video output by the video generation model.

[0142] The video generation model is a large model, and the video generation model is used to match the timestamp of the initial driving video with the timestamp of the commentary audio.

[0143] For example, the sentence "I passed through continuous mountains along the way" in the commentary audio needs to be matched with the "mountains" segment in the initial driving video, that is, the playback time of the two needs to match to ensure the consistency and matching of the intelligent dubbing.

[0144] The driving video generation method provided by the embodiment of the present invention uses the reasoning ability of the large model to match the commentary content of the commentary audio with the video content of the initial driving video in the above manner, and reads the commentary audio at a moderate speed and interspersed in the initial driving video to ensure the accuracy of the intelligent dubbing, thereby improving the generation accuracy of the target driving video and further enhancing the user experience.

[0145] Based on the above embodiments, the present invention actively records scenes, behaviors, etc. during driving, and intelligently generates a customized short video for users, which not only saves users' various costs, but also improves the user experience.

[0146] The driving video generating device provided by the present invention is described below. The driving video generating device described below and the driving video generating method described above can be referenced to each other.

[0147] Figure 3 is a schematic diagram of the structure of the driving video generation device provided by the present invention, such as Figure 3As shown, the driving video generating device includes an acquisition module 310 and a generation module 320.

[0148] The acquisition module 310 is used to acquire the initial driving video of the target vehicle during the driving process to be recorded, and the driving diary text of the driving process to be recorded.

[0149] The generation module 320 is used to add commentary audio and / or commentary subtitles to the initial driving video based on the driving diary text to generate a target driving video; the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text.

[0150] The driving diary text is obtained by inferring the driving record information of the driving process to be recorded based on a large model.

[0151] The driving video generation device provided by the embodiment of the present invention obtains the initial driving video of the target car in the driving process to be recorded, and the driving diary text of the driving process to be recorded, and the driving diary text is obtained by reasoning the driving record information of the driving process to be recorded based on a large model, so that the driving record information recorded in the driving process can be intelligently inferred to obtain the driving diary text by using the large model, without the need for users to write the driving diary text by themselves, thereby improving the convenience of driving video generation. At the same time, the driving diary text can be directly used as the audio text or subtitle text of the initial driving video, so that based on the driving diary text, commentary audio and / or commentary subtitles are automatically added to the initial driving video to generate a target driving video with audio content matching the video content, or to generate a target driving video with subtitles matching the video content, so that under the premise of ensuring the accuracy of driving video generation, there is no need for users to manually dub or add subtitles, thereby improving the convenience of driving video generation. In addition, the initial driving video includes multiple frames of driving images, and any driving image is obtained by capturing images of the external environment or the internal environment of the target car during the driving process to be recorded. Therefore, the initial driving video can include not only the video of the external environment, but also the video of the internal environment, thereby making the generated target driving video more colorful and providing more video scenes for matching with the driving diary text, thereby improving the generation effect of the driving video and thus improving the user experience.

[0152] Based on any of the above embodiments, the driving diary text is determined based on a text determination module, and the text determination module is used to: Acquiring driving record information of the driving process to be recorded; Input the driving record information into a text generation model to obtain the driving diary text output by the text generation model; the text generation model is a large model; Among them, the driving record information includes at least one of time information, travel route information, weather information along the way, passenger monitoring information and multimedia application usage information; the travel route information is determined based on the navigation information of the driving process to be recorded and / or the positioning information of the target vehicle; the weather information along the way is determined based on the positioning information of the target vehicle; the passenger monitoring information is obtained by monitoring each person in the target vehicle; and the multimedia application usage information is obtained by recording the usage of multimedia applications.

[0153] Based on any of the above embodiments, the passenger monitoring information includes monitoring information of a plurality of first preset key time points in the driving process to be recorded; and / or,

[0154] The passenger monitoring information includes at least one of emotion monitoring information, behavior monitoring information and voice recognition information.

[0155] Based on any of the above embodiments, the initial driving video is generated based on a video generation module, and the video generation module is used to: Acquire multiple driving segment videos and / or multiple driving images of the target vehicle during the driving process to be recorded; any of the driving segment videos includes multiple frames of driving images; When the plurality of driving segment videos are acquired, generating the initial driving video based on the plurality of driving segment videos; When the plurality of driving images are acquired, generating the initial driving video based on the plurality of driving images; When the plurality of driving segment videos and the plurality of driving images are acquired, the initial driving video is generated based on the plurality of driving segment videos and the plurality of driving images.

[0156] Based on any of the above embodiments, the multiple driving segment videos include driving segment videos captured at multiple second preset key time points in the driving process to be recorded; The multiple driving images include driving images captured at multiple second preset key time points in the driving process to be recorded.

[0157] Based on any of the above embodiments, when the passengers in the target car exhibit preset behaviors during the driving process to be recorded, the multiple driving clip videos include exterior videos of the target car's exterior environment captured at the time when the preset behaviors occur, and / or the multiple driving images include exterior images of the target car's exterior environment captured at the time when the preset behaviors occur.

[0158] Based on any of the above embodiments, any of the vehicle-exterior videos is collected based on a video acquisition module, and the video acquisition module is used to: During the driving process to be recorded, real-time monitoring of the behavior of passengers in the target car; If it is determined based on the behavior monitoring result that the passenger in the target car has a preset behavior, collecting voice data of the passenger in the target car; If a preset wake-up word is recognized based on the voice data, the external environment of the target car is collected to obtain the external video.

[0159] Based on any of the above embodiments, the video acquisition module is further used for: The behavior monitoring result and the voice data are recorded so that the driving record information includes the behavior monitoring result and the voice data.

[0160] Based on any of the above embodiments, when there is a preset recording object in the external environment of the target vehicle during the driving process to be recorded, the multiple driving clip videos include videos captured of the preset recording object, and / or the multiple driving images include images captured of the preset recording object.

[0161] Based on any of the above embodiments, the generating module 320 is further used for: In the case where there is real person voice feature data, speech synthesis is performed on the driving diary text based on the real person voice feature data to obtain commentary audio, and the commentary audio is added to the initial driving video to generate a target driving video; the real person voice feature data is generated based on real person voice data collected from the user; In the absence of real-person voice feature data, speech synthesis is performed on the driving diary text based on preset sound feature data to obtain commentary audio, which is added to the initial driving video to generate a target driving video, and / or, based on the driving diary text, commentary subtitles are added to the initial driving video to generate a target driving video.

[0162] Based on any of the above embodiments, the generating module 320 is further used for: Inputting the initial driving video and the commentary audio into a video generation model to obtain a target driving video output by the video generation model; The video generation model is a large model, and the video generation model is used to match the timestamp of the initial driving video with the timestamp of the commentary audio.

[0163] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the driving video generation method, which includes: obtaining the initial driving video of the target car in the driving process to be recorded, and the driving diary text of the driving process to be recorded; based on the driving diary text, adding commentary audio and / or commentary subtitles to the initial driving video to generate the target driving video; the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text; wherein the driving diary text is obtained by reasoning the driving record information of the driving process to be recorded based on the large model.

[0164] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0165] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the driving video generation method provided by the above-mentioned methods, which includes: obtaining an initial driving video of the target vehicle during the driving process to be recorded, and a driving diary text of the driving process to be recorded; based on the driving diary text, adding commentary audio and / or commentary subtitles to the initial driving video to generate a target driving video; the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text; wherein the driving diary text is obtained by inferring the driving record information of the driving process to be recorded based on a large model.

[0166] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the driving video generation method provided by the above-mentioned methods, the method comprising: obtaining an initial driving video of a target vehicle during a driving process to be recorded, and a driving diary text of the driving process to be recorded; based on the driving diary text, adding commentary audio and / or commentary subtitles to the initial driving video to generate a target driving video; the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text; wherein the driving diary text is obtained by inferring the driving record information of the driving process to be recorded based on a large model.

[0167] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A driving video generation method, characterized in that: include: Acquire an initial driving video of the target vehicle during the driving process to be recorded, and a driving diary text of the driving process to be recorded; Based on the driving diary text, adding commentary audio and / or commentary subtitles to the initial driving video to generate a target driving video; the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text; The driving diary text is obtained by inferring the driving record information of the driving process to be recorded based on a large model.

2. The driving video generation method according to claim 1, characterized in that: The driving diary text is determined based on the following method: Acquiring driving record information of the driving process to be recorded; Input the driving record information into a text generation model to obtain the driving diary text output by the text generation model; the text generation model is a large model; Among them, the driving record information includes at least one of time information, travel route information, weather information along the way, passenger monitoring information and multimedia application usage information; the travel route information is determined based on the navigation information of the driving process to be recorded and / or the positioning information of the target vehicle; the weather information along the way is determined based on the positioning information of the target vehicle; the passenger monitoring information is obtained by monitoring each person in the target vehicle; and the multimedia application usage information is obtained by recording the usage of multimedia applications.

3. The driving video generation method according to claim 2, characterized in that: The passenger monitoring information includes monitoring information of a plurality of first preset key time points in the driving process to be recorded; and / or, The passenger monitoring information includes at least one of emotion monitoring information, behavior monitoring information and voice recognition information.

4. The driving video generation method according to claim 1, characterized in that: The initial driving video includes multiple frames of driving images, and any of the driving images is obtained by capturing images of the exterior environment or interior environment of the target vehicle during the driving process to be recorded.

5. The driving video generation method according to claim 4, characterized in that: The initial driving video is generated based on the following method: Acquire multiple driving segment videos and / or multiple driving images of the target vehicle during the driving process to be recorded; any of the driving segment videos includes multiple frames of driving images; When the plurality of driving segment videos are acquired, generating the initial driving video based on the plurality of driving segment videos; When the plurality of driving images are acquired, generating the initial driving video based on the plurality of driving images; When the plurality of driving segment videos and the plurality of driving images are acquired, the initial driving video is generated based on the plurality of driving segment videos and the plurality of driving images.

6. The driving video generation method according to claim 5, characterized in that: The multiple driving segment videos include driving segment videos captured at multiple second preset key time points in the driving process to be recorded; The multiple driving images include driving images captured at multiple second preset key time points in the driving process to be recorded.

7. The driving video generation method according to claim 6, characterized in that: In the case where a passenger in the target vehicle exhibits preset behavior during the driving process to be recorded, the multiple driving clip videos include exterior videos of the target vehicle captured at the time when the preset behavior occurs, and / or the multiple driving images include exterior images of the target vehicle captured at the time when the preset behavior occurs.

8. The driving video generation method according to claim 7, characterized in that: Any of the above-mentioned external vehicle videos is collected based on the following methods: During the driving process to be recorded, real-time monitoring of the behavior of passengers in the target car; If it is determined based on the behavior monitoring result that the passenger in the target car has a preset behavior, collecting voice data of the passenger in the target car; If a preset wake-up word is recognized based on the voice data, the external environment of the target car is collected to obtain the external video.

9. The driving video generation method according to claim 8, characterized in that: After collecting the external environment of the target vehicle to obtain the external video, the method further includes: The behavior monitoring result and the voice data are recorded so that the driving record information includes the behavior monitoring result and the voice data.

10. The driving video generation method according to claim 6, characterized in that: In the case where there is a preset recording object in the external environment of the target vehicle during the driving process to be recorded, the multiple driving clip videos include videos captured of the preset recording object, and / or the multiple driving images include images captured of the preset recording object.

11. The driving video generation method according to claim 1, characterized in that: The step of adding commentary audio and / or commentary subtitles to the initial driving video based on the driving diary text to generate a target driving video includes: In the case where there is real person voice feature data, speech synthesis is performed on the driving diary text based on the real person voice feature data to obtain commentary audio, and the commentary audio is added to the initial driving video to generate a target driving video; the real person voice feature data is generated based on real person voice data collected from the user; In the absence of real-person voice feature data, speech synthesis is performed on the driving diary text based on preset sound feature data to obtain commentary audio, which is added to the initial driving video to generate a target driving video, and / or, based on the driving diary text, commentary subtitles are added to the initial driving video to generate a target driving video.

12. The driving video generation method according to claim 11, characterized in that: Adding the commentary audio to the initial driving video to generate a target driving video includes: Inputting the initial driving video and the commentary audio into a video generation model to obtain a target driving video output by the video generation model; The video generation model is a large model, and the video generation model is used to match the timestamp of the initial driving video with the timestamp of the commentary audio.

13. A driving video generation device, characterized in that: include: An acquisition module, used to acquire an initial driving video of the target vehicle during the driving process to be recorded, and a driving diary text of the driving process to be recorded; A generating module, configured to add commentary audio and / or commentary subtitles to the initial driving video based on the driving diary text, so as to generate a target driving video; the commentary audio is generated based on the driving diary text, and the commentary subtitles are generated based on the driving diary text; The driving diary text is obtained by inferring the driving record information of the driving process to be recorded based on a large model.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the driving video generation method as described in any one of claims 1 to 12 is implemented.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the driving video generation method as described in any one of claims 1 to 12 is implemented.

Images