Outdoor self-driving wonderful video generation method and device and vehicle

Abstract

The invention discloses an outdoor self-driving wonderful video generation method and device and a vehicle, and relates to the technical field of video generation, and the method comprises the steps: obtaining a vehicle-mounted recorded video and first driving information based on a vehicle position after a recording instruction is received; determining a travel corresponding to the vehicle-mounted recorded video according to the vehicle-mounted recorded video and the first driving information; second driving information based on the vehicle state is obtained in real time, and a wonderful fragment is determined according to the vehicle-mounted record video, the first driving information, the second driving information and the journey; according to the wonderful fragment and the journey, acquiring environment information corresponding to the wonderful fragment; performing structured analysis according to the wonderful fragments fused with the environmental information to obtain structured information of the wonderful fragments; and generating an outdoor vehicle-mounted wonderful video according to the structured information of the wonderful clip. The wonderful video is generated by intelligently acquiring the structured information, so that the video creation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of video generation, and in particular to a method, device and vehicle for generating wonderful outdoor self-driving videos. Background Art

[0002] With the rise of the outdoor self-driving tour craze, people have a strong demand for outdoor shooting. For outdoor self-driving videos, compared with the complete and lengthy original vehicle-mounted video content, more people hope to complete the information reception of the entire shooting content by watching the wonderful video highlights. At present, when vehicles are driving in the wild, they often encounter extreme road conditions such as bumps and sharp turns, but at this time, the content shot by the vehicle can show scenes that are obviously different from those in urban areas. This type of video is extremely popular. The wonderful video highlights are mainly manually edited and produced by users (such as users of Xiaohongshu, Douyin, Kuaishou and other platforms). Users need to dig out fans' interests based on personal experience to find wonderful clips, and then use editing software to edit them. The entire wonderful video creation process is time-consuming and laborious. Summary of the invention

[0003] The main purpose of this application is to provide a method, device and vehicle for generating wonderful outdoor self-driving videos, aiming to solve the technical problem that the creation process of wonderful videos in the prior art is time-consuming and labor-intensive.

[0004] To achieve the above purpose, the present application proposes a method for generating a wonderful outdoor self-driving video, which is applied to a vehicle, wherein the cameras on the vehicle are respectively located at the front, side or rear of the vehicle, and the vehicle-mounted recorded video is obtained through the cameras, and the method comprises:

[0005] After receiving the recording instruction, obtaining the vehicle recorded video and the first driving information based on the vehicle position;

[0006] Determine the trip corresponding to the in-vehicle recorded video according to the in-vehicle recorded video and the first driving information;

[0007] Acquire second driving information based on the vehicle state in real time, and determine a highlight segment according to the vehicle recorded video, the first driving information, the second driving information and the itinerary;

[0008] According to the highlight clip and the itinerary, acquiring environmental information corresponding to the highlight clip;

[0009] Performing structured analysis on the wonderful clips integrated with the environmental information to obtain structured information of the wonderful clips;

[0010] Generate an outdoor vehicle-mounted wonderful video according to the structured information of the wonderful clip.

[0011] In one embodiment, after the step of acquiring the second driving information based on the vehicle state in real time, the method further includes:

[0012] acquiring each parameter value in real time according to the second driving information;

[0013] Setting a preset standard value of each parameter in the second driving information;

[0014] By comparing the parameter value obtained in real time with the preset standard value, the target time period and the previous candidate time period are screened out.

[0015] In one embodiment, the step of comparing the second driving information acquired in real time with the preset standard value to screen out the target time period and the first candidate time period includes:

[0016] Setting the preset standard deviation value of each parameter;

[0017] According to the difference between the preset standard value and each parameter value, a comparison is made with the preset standard deviation value;

[0018] If there are more than zero differences greater than the preset standard deviation, the current moment is obtained, and the first preset duration after the moment is used as the target time period, and the second preset duration before the moment is used as the previous candidate time period.

[0019] In one embodiment, before the step of determining a highlight segment according to the vehicle-mounted recorded video, the first driving information, the second driving information and the itinerary, the step further includes:

[0020] Detecting a first feature point in each video frame in the target time period that matches a feature point in other video frames;

[0021] and detecting a second feature point in each video frame of the first candidate time period that matches a feature point in other video frames;

[0022] A video frame feature change trajectory is obtained according to the coordinates of the first feature point and the second feature point in the image coordinate system.

[0023] In one embodiment, the step of determining a highlight segment according to the vehicle recorded video, the first driving information, the second driving information and the trip includes:

[0024] If the trajectory of the video frame feature changes is consistent with the change of the second driving information, the vehicle-recorded video after the target time period and the previous candidate time period are merged is used as the highlight segment.

[0025] In one embodiment, after the step of determining a highlight segment according to the vehicle-mounted recorded video, the first driving information, the second driving information and the itinerary, the method further includes:

[0026] If the video length of the wonderful segment is not appropriate, marking the starting video frame and the ending video frame of the wonderful segment on the vehicle-mounted recorded video;

[0027] The highlight segment is updated according to adjusting the positions of the start video frame and the end video frame on the vehicle-mounted recorded video.

[0028] In one embodiment, the step of determining the trip corresponding to the in-vehicle recorded video according to the in-vehicle recorded video and the first driving information includes:

[0029] Acquire the real-time position information of the vehicle in the first driving information;

[0030] Obtaining the recording start time and the recording end time of the vehicle-mounted recorded video;

[0031] The itinerary corresponding to the vehicle-mounted recorded video is determined according to the recording start time, the recording end time and the real-time position information of the vehicle.

[0032] In one embodiment, the step of performing structured analysis on the highlight fragment integrated with the environmental information to obtain structured information of the highlight fragment comprises:

[0033] Preprocessing the wonderful clips integrated with the environmental information to obtain preprocessed data;

[0034] Obtain relevant features based on the preprocessed data;

[0035] According to the preprocessed data and the relevant features, a structured analysis is performed to obtain the structured information of the highlight segment.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a device for generating a wonderful outdoor self-driving video, and the device for generating a wonderful outdoor self-driving video includes:

[0037] An acquisition module, used for acquiring the vehicle recorded video and the first driving information based on the vehicle position after receiving the recording instruction;

[0038] A determination module, configured to determine a trip corresponding to the vehicle-recorded video according to the vehicle-recorded video and the first driving information;

[0039] The determination module is further used to obtain second driving information based on the vehicle state in real time, and determine the highlight segment according to the vehicle recorded video, the first driving information, the second driving information and the itinerary;

[0040] The acquisition module is further used to acquire the environment information corresponding to the wonderful clip according to the wonderful clip and the itinerary;

[0041] An analysis module, configured to perform a structured analysis on the wonderful clip based on the environmental information to obtain structured information of the wonderful clip;

[0042] A generation module is used to generate outdoor vehicle-mounted wonderful videos according to the structured information of the wonderful clips.

[0043] In addition, to achieve the above-mentioned purpose, the present application also proposes a vehicle, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for generating a wonderful outdoor self-driving video as described above.

[0044] One or more technical solutions proposed in this application have at least the following technical effects:

[0045] After receiving the recording instruction, the vehicle recorded video and the first driving information based on the vehicle position are obtained; the itinerary corresponding to the vehicle recorded video is determined according to the vehicle recorded video and the first driving information; the second driving information based on the vehicle status is obtained in real time, and the highlight clip is determined according to the vehicle recorded video, the first driving information, the second driving information and the itinerary; the environmental information corresponding to the highlight clip is obtained according to the highlight clip and the itinerary; the structured information of the highlight clip is obtained according to the highlight clip integrated with the environmental information; the outdoor vehicle-mounted highlight video is generated according to the structured information of the highlight clip. Compared with the prior art, the present application generates high-quality outdoor vehicle-mounted highlight videos by intelligently obtaining the structured information of the highlight clips, thereby improving the efficiency of video creation and reducing the consumption of manpower costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A flow chart of the method for generating a wonderful outdoor self-driving video according to the present application is provided in Example 1;

[0049] Figure 2 A brief flow chart of the method for generating a wonderful outdoor self-driving video according to the present application is provided in Example 1;

[0050] Figure 3 A brief flowchart of a method for generating an outdoor self-driving wonderful video provided in Example 2 of the present application;

[0051] Figure 4 A brief flowchart of a method for generating an outdoor self-driving wonderful video provided in Example 3 of the present application;

[0052] Figure 5 A brief flowchart of a method for generating an outdoor self-driving wonderful video provided in Example 4 of the present application;

[0053] Figure 6 This is a schematic diagram of the module structure of a device for generating a wonderful outdoor self-driving video according to an embodiment of the present application;

[0054] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the method for generating a wonderful outdoor self-driving video in the embodiment of the present application.

[0055] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0056] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0057] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0058] The main solutions of the embodiments of this application are:

[0059] When vehicles are driving in the wild, they often encounter extreme road conditions such as bumps and sharp turns. However, the content captured by the vehicle at this time can show scenes that are obviously different from those in urban areas. Such videos are extremely popular. Collections of wonderful videos are mainly manually edited and produced by users (such as users of platforms such as Xiaohongshu, Douyin, and Kuaishou). The existing technology is that users find wonderful clips based on their personal experience and then use editing software to edit them. However, this method makes the entire process of creating wonderful videos time-consuming and laborious.

[0060] The present application provides a solution that generates high-quality outdoor vehicle-mounted exciting videos by intelligently acquiring structured information of exciting clips, thereby improving the efficiency of video creation and reducing labor costs.

[0061] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a vehicle, etc. capable of realizing the above functions. The following takes a vehicle as an example to illustrate this embodiment and the following embodiments.

[0062] Based on this, the embodiment of the present application provides a method for generating an outdoor self-driving wonderful video, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for generating wonderful outdoor self-driving videos of the present application.

[0063] In this embodiment, the method for generating the outdoor self-driving wonderful video includes steps S10 to S60:

[0064] Step S10, after receiving the recording instruction, obtaining the vehicle recorded video and the first driving information based on the vehicle position.

[0065] It should be noted that the recording command refers to the command to start outdoor video shooting. Once the command is turned on, the following steps will be started. The command can be a voice command. For example, when the car computer receives the voice of the person in the car shouting "start outdoor video recording", the people in the car include but are not limited to passengers or drivers. For example, the car computer has a relevant functional interface. The person in the car touches and clicks the response function module in the functional interface, and then the car computer can receive the recording command. At the same time, since outdoor travel generally takes a long time and itinerary, in order to reduce the length of the original recorded video and reduce the later computing power, a target area can be pre-delineated. The target area can be delineated on a map, or it can be delineated by outputting multiple geographic longitude and latitude coordinates. Once the vehicle is detected to enter the target area, a recording command is issued, or the person in the car is reminded to issue a recording command.

[0066] In addition, the in-vehicle recorded video is captured by a camera on the car, and the position of the camera is as mentioned above, not limited to the front, side or rear of the car. For example, the direction captured by the front of the car can be the direction corresponding to the driver driving, the side of the car can be the direction corresponding to the passenger turning his head to observe the scenery outside the car, the left camera captures the scenery on the left side of the car, and the right camera captures the scenery on the right side of the car.

[0067] In addition, the driving information is the state information of the vehicle during the shooting process, and the first driving information based on the vehicle position includes the real-time coordinate position of the vehicle. Specifically, after receiving the recording instruction, the first driving information about the vehicle coordinate position is obtained in real time.

[0068] It can be understood that performing step S10 can effectively reduce the loss of recorded videos, thereby improving the integrity and efficiency of obtaining vehicle-mounted recorded videos.

[0069] Step S20: determining the trip corresponding to the in-vehicle recorded video according to the in-vehicle recorded video and the first driving information.

[0070] It should be noted that the method for determining the itinerary corresponding to the vehicle-mounted recorded video is based on the first driving information, the recording start time and the recording end time of the vehicle-mounted recorded video. Specifically, the recording start time and the recording end time jointly define the duration of the recorded video. Of course, the duration of recording a video can also be preset, for example, setting a video to be recorded every 20 minutes. The following is an example: the recording start time of the first video is 13:00 on May 20, 2024, and the recording end time is 13:20 on May 20, 2024. The second video can be continuous or discontinuous with the first video, and the discontinuous duration can also be preset. Preferably, each video segment is continuous.

[0071] In addition, after obtaining the first driving information, the recording start time and the recording end time of the vehicle-mounted recorded video, the time and the real-time position information of the vehicle are known, so that the itinerary of the vehicle during the shooting process, that is, the path from point A to point B, can be obtained.

[0072] It can be appreciated that the determination of the vehicle's journey provides a solid foundation for subsequent steps.

[0073] Step S30, obtaining second driving information based on the vehicle state in real time, and determining a highlight segment according to the vehicle-mounted recorded video, the first driving information, the second driving information and the itinerary.

[0074] It should be noted that the second driving information based on the vehicle state includes one or more second driving information about the vehicle speed, steering wheel angle, gear position, and bumpy state during the journey.

[0075] In addition, the determined wonderful clips can be one or more, which mainly depends on the number of wonderful video effects that occur during the vehicle journey. Specifically, in outdoor travel, wonderful video effects generally include sudden changes in vehicle speed (the video may show exciting things happening, such as wild animals or sudden road conditions), sudden turns in direction (causing the vehicle to turn, and the video may have a feeling of light at the end of the tunnel), and large bumps (the video may have the wild feeling of driving outdoors).

[0076] In one possible implementation, refer to Figure 2 , after step S30, steps S31 to S32 may also be included:

[0077] Step S31: If the video length of the highlight segment is not appropriate, the start video frame and the end video frame of the highlight segment are marked on the vehicle-mounted recorded video.

[0078] It should be noted that inappropriate video length includes videos that are too long or too short. For example, when a creator wants to edit a wonderful outdoor self-driving short video, and the highlight content automatically determined by the vehicle is too long, the creator can control the starting video frame and the ending video frame of the highlight to shorten the video length. When the creator finds that the highlight content automatically determined by the vehicle is too short and lacks some key clips, the creator can mark the starting video frame and the ending video frame of the highlight on the on-board recorded video, and then determine a longer highlight video.

[0079] Step S32: updating the highlight segment according to adjusting the positions of the start video frame and the end video frame on the vehicle-mounted recorded video.

[0080] It should be noted that after the creator has adjusted the positions of the starting video frame and the ending video frame, the content of the highlight clip will be updated and confirmed. The highlight clip will only change after the creator confirms to update the highlight clip.

[0081] It is understandable that this method of intelligently determining and ultimately manually controlling the highlight clips effectively improves the efficiency of obtaining the highlight clips and the video quality.

[0082] Step S40: acquiring environmental information corresponding to the highlight segment according to the highlight segment and the itinerary.

[0083] It should be noted that the environmental information reflects the environmental information of this outdoor (wild) self-driving, such as weather (for example, videos and photos taken under scorching sun, heavy snow, heavy rain and other weather conditions are distinctive and have a strong visual impact), geographical location (for example, road names such as the G318 National Highway that is well-known in the outdoor circle and other place names with a certain influence), altitude (for example, plateaus, basins, ridges and other landforms are distinctive and have a strong visual impact), wind speed, temperature, ambient light brightness (the ambient light under the scorching sun in summer and the ambient light brightness before a rainstorm or at night are both distinctive and have a strong visual impact). One or more of them.

[0084] Step S50: performing structured analysis on the wonderful clips integrated with the environmental information to obtain structured information of the wonderful clips.

[0085] It should be noted that the structured information of the highlight clip refers to the information of the highlight video integrated with the environmental conditions, which can be decomposed into multiple interrelated components after analysis, and each component has a clear hierarchical structure. Usually, for outdoor highlight clips, the main characteristics of the highlight clip can be determined by combining the environmental information, thereby realizing the structured analysis of the highlight clip.

[0086] In addition, structured analysis usually involves decomposing video content into understandable and operable data formats, which is a complex process. Specifically, the highlight clips that incorporate environmental information are preprocessed to obtain preprocessed data, relevant features are obtained based on the preprocessed data, and structured analysis is performed based on the preprocessed data and the relevant features to obtain structured information of the highlight clips.

[0087] It can be understood that this structured information extraction method is efficient and has high scalability and replicability.

[0088] Step S60: Generate an outdoor vehicle-mounted wonderful video according to the structured information of the wonderful clip.

[0089] It should be noted that structured information can provide high-quality materials for highlights, helping creators to quickly complete the content creation of outdoor car-mounted highlights.

[0090] This embodiment provides a method for generating outdoor self-driving wonderful videos. After receiving a recording instruction, the vehicle-mounted recorded video and the first driving information based on the vehicle position are obtained; the itinerary corresponding to the vehicle-mounted recorded video is determined according to the vehicle-mounted recorded video and the first driving information; the second driving information based on the vehicle status is obtained in real time, and a wonderful clip is determined according to the vehicle-mounted recorded video, the first driving information, the second driving information and the itinerary; the environmental information corresponding to the wonderful clip is obtained according to the wonderful clip and the itinerary; a structured analysis is performed on the wonderful clip that integrates the environmental information to obtain the structured information of the wonderful clip; and an outdoor vehicle-mounted wonderful video is generated according to the structured information of the wonderful clip. By intelligently obtaining structured information, a wonderful video is generated, which improves the efficiency of video creation.

[0091] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S30 includes steps S301 to S304:

[0092] Step S301: acquiring each parameter value in real time according to the second driving information, and setting a preset standard value of each parameter in the second driving information.

[0093] It should be noted that each parameter value includes information such as vehicle speed, steering wheel angle and bump amplitude obtained from the second driving information.

[0094] In addition, the preset standard values ​​can be set independently according to user needs, and are usually parameter values ​​when performing attractive vehicle behaviors outdoors, such as a vehicle speed of 100 kilometers per hour or a steering wheel angle of 90 degrees.

[0095] Step S302: By comparing the parameter value obtained in real time with the preset standard value, the target time period and the previous candidate time period are screened out.

[0096] It should be noted that the front alternative time period is set because there may be a contrast between the videos in the two time periods before and after, that is, there is an eye-catching contrast and sudden visual effect.

[0097] Step S303: obtaining a video frame feature change trajectory through video frames in the target time period and the previous candidate time period.

[0098] It should be noted that a video frame is a basic concept in video display and video files. It is a single still image that constitutes a video sequence. It is usually a separate image in a video stream, equivalent to a frame in a movie film.

[0099] In addition, the video frame feature change trajectory refers to the description of how the feature points or feature areas of the frame move and change over time in the video sequence. These trajectories can be used in a variety of applications, such as motion detection, behavior recognition, target tracking, etc.

[0100] It can be understood that when the vehicle is in a bumpy state, the trajectory of the video frame feature changes is up and down; when the vehicle is moving straight, the trajectory of the video frame feature changes is a straight line; when the vehicle is turning, the trajectory of the video frame feature changes is a curve.

[0101] Step S304: If the trajectory of the video frame feature changes is consistent with the change of the second driving information, the vehicle-recorded video after the target time period and the previous candidate time period are merged is used as the highlight segment.

[0102] It should be noted that, taking a car turning sharply as an example, when the detected video frame feature change trajectory is consistent with the second driving information change (such as the change in the steering wheel rotation angle), it indicates that the effect of the video connection between the two time periods is consistent with the actual vehicle motion trajectory, which can reflect the vehicle's motion state, so that the vehicle-mounted recorded video merged with the target time period and the previous candidate time period can be used as a highlight clip. At the same time, if the detected feature change trajectory is inconsistent with the second driving information change, that is, the surrounding environment has not changed much, but the car is turning and braking suddenly, for example, in a curved tunnel, the scenery in the tunnel itself has hardly changed, that is, it is a cement wall, or in a dark environment, or on a test track, although the car's driving information has been changing, but the external environment is cement road and has hardly changed, it can be considered that the video connection between the two time periods cannot reflect the vehicle's motion theme and does not have a stimulating visual effect, so this video is not needed as a highlight clip.

[0103] This embodiment obtains each parameter value in real time according to the second driving information, and sets the preset standard value of each parameter in the second driving information; by comparing the parameter value obtained in real time with the preset standard value, the target time period and the previous candidate time period are screened out; the video frame feature change trajectory is obtained through the video frames of the target time period and the previous candidate time period; if the video frame feature change trajectory is consistent with the change of the second driving information, the vehicle recorded video after the target time period and the previous candidate time period are merged is used as the highlight clip. By setting the preset parameter value to determine the feature change trajectory, the highlight clip can be determined quickly and accurately.

[0104] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the first and second embodiments can be referred to the above description, and will not be described in detail later. Figure 4 , step S302 also includes steps A31 to A33:

[0105] Step A31: Calculate the difference between the preset standard value and each parameter value.

[0106] It should be noted that the difference between the preset standard value and each parameter value can be a positive number or a negative number. For example, if the preset standard value of the vehicle speed is set to 100 kilometers per hour, if the real-time vehicle speed is 60 kilometers per hour, the difference is -40 kilometers per hour. If the real-time vehicle speed is 120 kilometers per hour, the difference is 20 kilometers per hour. This difference will serve as an important comparison parameter in subsequent steps.

[0107] Step A32: setting a preset standard deviation value for each parameter, and comparing the difference between the preset standard value and each parameter value with the preset standard deviation value.

[0108] It should be noted that the steering wheel turning angle is divided into left turning and right turning, so the preset standard value for left turning and the preset standard value for right turning will also be set, but these two preset standard values ​​are usually equal. Specifically, the standard deviation value for left turning is set to 15 degrees, the standard deviation value for right turning is set to 15 degrees, the standard value for left turning is set to 30 degrees, and the standard value for right turning is set to 30 degrees. If the vehicle's left turning angle is detected to be 60 degrees, then the parameter value of the steering wheel turning to the left is 30 degrees, and 30 degrees is compared with 15 degrees; at another moment, if the steering wheel turning angle to the right is detected to be 61 degrees, then the difference between the parameter value of the steering wheel turning to the right and the preset standard value is 31 degrees, and 31 degrees is compared with 15 degrees.

[0109] Step A33: If there are more than zero difference values ​​greater than the preset standard deviation value, the current moment is obtained, and the first preset time length after the moment is used as the target time period, and the second preset time length before the moment is used as the previous candidate time period.

[0110] It should be noted that if at least one difference is greater than the standard deviation, the moment T1 at that time is obtained, and the preset time length ΔT1 after the moment T1 is used as the target time period. At the same time, the preset time length ΔT2 before T1 is used as the previous alternative time period. Both ΔT1 and ΔT2 can be set freely. Specifically, 10 minutes after the moment T1 can be set as the target time period, and 5 minutes before the moment T1 can be set as the alternative time period.

[0111] In addition, by presetting standard values ​​of the selected parameters (for example, the standard vehicle speed is 100 kilometers per hour, the standard steering wheel turning angle to the left is 30 degrees, the standard steering wheel turning angle to the right is 30 degrees, and the standard bump amplitude is 15 mm), and setting the standard vehicle speed difference to 10 kilometers per hour, the standard steering wheel turning angle difference to the left is 15 degrees, the standard steering wheel turning angle difference to the right is 15 degrees, and the standard deviation of the bump amplitude is 10 mm, and at the same time detecting the second driving information of the vehicle in real time, for example, if it is detected that the vehicle speed at a certain moment is 130 kilometers per hour, the steering wheel turning angle to the left is 5 degrees, and the bump amplitude is 15 mm, and only one vehicle speed difference is greater than the standard deviation, then the period after that moment is used as the target time period, and the period before that moment is used as the alternative time period.

[0112] This embodiment calculates the difference between the preset standard value and each parameter value; sets the preset standard deviation value of each parameter, and compares the difference between the preset standard value and each parameter value with the preset standard deviation value; if there are more than zero differences greater than the preset standard deviation value, the current moment is obtained, and the first preset time length after the moment is used as the target time period, and the second preset time length before the moment is used as the previous alternative time period. By setting the standard value and the standard deviation value to determine the target time period and the alternative time period of the highlight clip, the accuracy of obtaining the highlight clip is improved.

[0113] Based on the first and second embodiments of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the first and second embodiments can be referred to the above description, and will not be described in detail later. Figure 5 , step S303 includes steps B31 to B33:

[0114] Step B31: Detecting the first feature point in each video frame in the target time period that matches the feature points in other video frames.

[0115] It should be noted that the feature points in the video frame can be the corner points of the video frame. For example, the corner points of the video frame are detected using a corner point detection algorithm such as FAST (features from accelerated segment test), SUSAN, or Harris operator. Then, a feature point 1, i.e., the first feature point, is detected in each video frame in the target time period and matches the feature points in other video frames.

[0116] Step B32: Detecting second feature points in each video frame of the first candidate time period that match feature points in other video frames.

[0117] It should be noted that the feature point 2 in each video frame in the candidate time period before detection that matches the feature points in other video frames is the second feature point.

[0118] Step B33: Obtain the video frame feature change trajectory according to the coordinates of the first feature point and the second feature point in the image coordinate system.

[0119] It should be noted that the feature change trajectory can be obtained based on any feature point, but the video may be distorted and the distortion cannot be calculated at all, so at least two feature points are required to determine it. In order to eliminate the distortion of the video, a second feature point is introduced.

[0120] This embodiment detects the first feature point in each video frame in the target time period that matches the feature point in other video frames; detects the second feature point in each video frame in the previous candidate time period that matches the feature point in other video frames; and obtains the video frame feature change trajectory according to the coordinates of the first feature point and the second feature point in the image coordinate system. By introducing the first feature point and the second feature point, the distortion of the video is eliminated.

[0121] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method for generating wonderful outdoor self-driving videos of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0122] This application also provides a device for generating outdoor self-driving wonderful videos, please refer to Figure 6 , the device for generating the outdoor self-driving wonderful video includes:

[0123] The acquisition module 10 is used to acquire the vehicle recorded video and the first driving information based on the vehicle position after receiving the recording instruction.

[0124] The determination module 20 is used to determine the trip corresponding to the in-vehicle recorded video according to the in-vehicle recorded video and the first driving information.

[0125] The determination module 20 is further configured to obtain second driving information based on the vehicle state in real time, and determine a highlight segment according to the vehicle recorded video, the first driving information, the second driving information and the itinerary.

[0126] The acquisition module 10 is further configured to acquire environmental information corresponding to the highlight segment according to the highlight segment and the itinerary.

[0127] The analysis module 30 is used to perform a structured analysis on the highlight segment based on the environmental information to obtain structured information of the highlight segment.

[0128] The generating module 40 is used to generate an outdoor vehicle-mounted wonderful video according to the structured information of the wonderful clip.

[0129] The device for generating outdoor self-driving wonderful videos provided by the present application adopts the method for generating outdoor self-driving wonderful videos in the above-mentioned embodiment, which can solve the technical problem that the creation process of wonderful videos in the prior art is time-consuming and laborious. Compared with the prior art, the beneficial effects of the device for generating outdoor self-driving wonderful videos provided by the present application are the same as the beneficial effects of the method for generating outdoor self-driving wonderful videos provided by the above-mentioned embodiment, and other technical features in the device for generating outdoor self-driving wonderful videos are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0130] In one embodiment, the acquisition module 10 is also used to acquire each parameter value in real time according to the second driving information; set a preset standard value for each parameter in the second driving information; and screen out a target time period by comparing the parameter value acquired in real time with the preset standard value.

[0131] In one embodiment, the acquisition module 10 is also used to set a preset standard deviation value for each parameter; the difference between the preset standard value and each parameter value is compared with the preset standard deviation value; if there are greater than zero differences that are greater than the preset standard deviation value, the current moment is acquired, and the first preset time length after the moment is used as the target time period, and the second preset time length before the moment is used as the previous alternative time period.

[0132] In one embodiment, the determination module 20 is also used to detect a first feature point in each video frame in the target time period that matches a feature point in other video frames; and to detect a second feature point in each video frame in the previous alternative time period that matches a feature point in other video frames; and to obtain a video frame feature change trajectory based on the coordinates of the first feature point and the second feature point in the image coordinate system.

[0133] In one embodiment, the determination module 20 is further configured to use the vehicle recorded video after merging the target time period and the previous candidate time period as the highlight segment if the video frame feature change trajectory is consistent with the change of the second driving information.

[0134] In one embodiment, the determination module 20 is also used to mark the starting video frame and the ending video frame of the wonderful segment on the vehicle-mounted recorded video if the video length of the wonderful segment is not appropriate; and update the wonderful segment according to adjusting the positions of the starting video frame and the ending video frame on the vehicle-mounted recorded video.

[0135] In one embodiment, the determination module 20 is also used to obtain the real-time position information of the vehicle in the first driving information; obtain the recording start time and the recording end time of the on-board recorded video; and determine the itinerary corresponding to the on-board recorded video based on the recording start time, the recording end time and the real-time position information of the vehicle.

[0136] In one embodiment, the analysis module 30 is further used to preprocess the wonderful clips integrated with the environmental information to obtain preprocessed data; obtain relevant features based on the preprocessed data; and perform structured analysis based on the preprocessed data and the relevant features to obtain structured information of the wonderful clips.

[0137] The present application provides a vehicle, which is a gasoline vehicle, but is not limited to a gasoline vehicle. It can be understood that the vehicle can also be a pure electric vehicle or a gasoline-electric hybrid vehicle.

[0138] The vehicle includes a frame forming the outer shape of the vehicle and four wheels for moving the vehicle. The frame also includes a hood, doors, and a trunk lid.

[0139] The frame also includes a sunroof, through which passengers in the cabin can look above the roof.

[0140] In addition, the vehicle frame also includes a front window, side windows and a rear window. Passengers and drivers can see the scenery in front of the vehicle through the front window, the scenery to the side of the vehicle through the side windows, and the scenery behind the vehicle through the rear window.

[0141] In addition, the vehicle frame also includes a pair of side-view mirrors, which are located on both sides of the vehicle frame, and can help the driver and passengers in the vehicle observe the areas behind and on both sides of the vehicle.

[0142] In addition, the frame also includes headlights and taillights. The headlights are used to illuminate the front of the vehicle to ensure forward vision, and the taillights are used to provide prompts to the rear of the vehicle.

[0143] In addition, the vehicle frame also includes cameras, which can be set on the front window, side window and rear window. The camera can observe images of the corresponding position and its surrounding area according to its setting position.

[0144] The interior of the vehicle includes a central control front desk with a display area that can display information such as vehicle speed, fuel consumption, power, and real-time images inside and outside the vehicle.

[0145] It should be noted that the display area can be set as a touch screen, which can not only display information but also perform touch control. For example, the touch screen displays the operating system adapted to the vehicle, and receives various control commands from the user to control the vehicle through operations such as touching, clicking, dragging, etc.

[0146] The present application provides a vehicle, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating a wonderful outdoor self-driving video in the above-mentioned embodiment one.

[0147] Reference below Figure 7 , which shows a schematic diagram of a structure of a vehicle suitable for implementing the embodiment of the present application. The vehicle in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The vehicle shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0148] like Figure 7As shown, the vehicle may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for vehicle operation are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the vehicle to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a vehicle with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0149] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0150] The vehicle provided by the present application adopts the method for generating outdoor self-driving wonderful videos in the above-mentioned embodiment, which can solve the technical problem that the creation process of wonderful videos in the prior art is time-consuming and laborious. Compared with the prior art, the beneficial effects of the vehicle provided by the present application are the same as the beneficial effects of the method for generating outdoor self-driving wonderful videos provided by the above-mentioned embodiment, and other technical features in the vehicle are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0151] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0152] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0153] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the method for generating a wonderful outdoor self-driving video in the above-mentioned embodiment.

[0154] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0155] The computer-readable storage medium may be included in the vehicle, or may exist independently without being installed in the vehicle.

[0156] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the vehicle, the vehicle: after receiving the recording instruction, obtains the on-board recorded video and the first driving information based on the vehicle position; determines the itinerary corresponding to the on-board recorded video according to the on-board recorded video and the first driving information; obtains the second driving information based on the vehicle status in real time, and determines the highlight clips according to the on-board recorded video, the first driving information, the second driving information and the itinerary; obtains the environmental information corresponding to the highlight clips according to the highlight clips and the itinerary; performs structured analysis on the highlight clips integrated with the environmental information to obtain the structured information of the highlight clips; generates an outdoor on-board highlight video according to the structured information of the highlight clips.

[0157] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0158] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0159] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0160] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method for generating a wonderful outdoor self-driving video as described above.

[0161] The computer program product provided by the present application can solve the technical problem that the creation process of the wonderful video in the prior art is time-consuming and laborious. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the method for generating the wonderful outdoor self-driving video provided by the above embodiment, which will not be repeated here.

[0162] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for generating a wonderful outdoor self-driving video, characterized in that: The method for generating a wonderful outdoor self-driving video is applied to a vehicle, wherein the cameras on the vehicle are respectively located at the front, side or rear of the vehicle, and the vehicle-mounted recorded video is obtained through the cameras. The method comprises: After receiving the recording instruction, obtaining the vehicle recorded video and the first driving information based on the vehicle position; Determine the trip corresponding to the in-vehicle recorded video according to the in-vehicle recorded video and the first driving information; Acquire second driving information based on the vehicle state in real time, and determine a highlight segment according to the vehicle recorded video, the first driving information, the second driving information and the itinerary; According to the highlight clip and the itinerary, acquiring environmental information corresponding to the highlight clip; Performing structured analysis on the wonderful clips integrated with the environmental information to obtain structured information of the wonderful clips; Generate an outdoor vehicle-mounted wonderful video according to the structured information of the wonderful clip.

2. The method according to claim 1, characterized in that After the step of acquiring the second driving information based on the vehicle state in real time, the method further includes: acquiring each parameter value in real time according to the second driving information; Setting a preset standard value of each parameter in the second driving information; By comparing the parameter value obtained in real time with the preset standard value, the target time period and the previous candidate time period are screened out.

3. The method according to claim 2, characterized in that The step of comparing the second driving information acquired in real time with the preset standard value to screen out the target time period and the previous candidate time period comprises: Setting the preset standard deviation value of each parameter; According to the difference between the preset standard value and each parameter value, a comparison is made with the preset standard deviation value; If there are more than zero differences greater than the preset standard deviation, the current moment is obtained, and the first preset duration after the moment is used as the target time period, and the second preset duration before the moment is used as the previous candidate time period.

4. The method according to claim 3, characterized in that Before the step of determining a highlight segment according to the vehicle-mounted recorded video, the first driving information, the second driving information and the itinerary, the method further includes: Detecting a first feature point in each video frame in the target time period that matches a feature point in other video frames; and detecting a second feature point in each video frame of the first candidate time period that matches a feature point in other video frames; A video frame feature change trajectory is obtained according to the coordinates of the first feature point and the second feature point in the image coordinate system.

5. The method according to claim 4, characterized in that The step of determining a highlight segment according to the vehicle-mounted recorded video, the first driving information, the second driving information and the itinerary comprises: If the trajectory of the video frame feature changes is consistent with the change of the second driving information, the vehicle-recorded video after the target time period and the previous candidate time period are merged is used as the highlight segment.

6. The method according to any one of claims 1 to 5, characterized in that After the step of determining a highlight segment according to the vehicle-mounted recorded video, the first driving information, the second driving information and the itinerary, the method further includes: If the video length of the wonderful segment is not appropriate, marking the starting video frame and the ending video frame of the wonderful segment on the vehicle-mounted recorded video; The highlight segment is updated according to adjusting the positions of the start video frame and the end video frame on the vehicle-mounted recorded video.

7. The method according to any one of claims 1 to 5, characterized in that The step of determining the trip corresponding to the vehicle recorded video according to the vehicle recorded video and the first driving information comprises: Acquire the real-time position information of the vehicle in the first driving information; Obtaining the recording start time and the recording end time of the vehicle-mounted recorded video; The itinerary corresponding to the vehicle-mounted recorded video is determined according to the recording start time, the recording end time and the real-time position information of the vehicle.

8. The method according to any one of claims 1 to 5, characterized in that The step of performing structured analysis on the wonderful clips integrated with the environmental information to obtain structured information of the wonderful clips comprises: Preprocessing the wonderful clips integrated with the environmental information to obtain preprocessed data; Obtain relevant features based on the preprocessed data; According to the preprocessed data and the relevant features, a structured analysis is performed to obtain the structured information of the highlight segment.

9. A device for generating outdoor self-driving wonderful videos, characterized in that: The device for generating the outdoor self-driving wonderful video includes: An acquisition module, used for acquiring the vehicle recorded video and the first driving information based on the vehicle position after receiving the recording instruction; A determination module, configured to determine a trip corresponding to the vehicle-recorded video according to the vehicle-recorded video and the first driving information; The determination module is further used to obtain second driving information based on the vehicle state in real time, and determine the highlight segment according to the vehicle recorded video, the first driving information, the second driving information and the itinerary; The acquisition module is further used to acquire the environment information corresponding to the wonderful clip according to the wonderful clip and the itinerary; An analysis module, configured to perform a structured analysis on the wonderful clip based on the environmental information to obtain structured information of the wonderful clip; A generation module is used to generate outdoor vehicle-mounted wonderful videos according to the structured information of the wonderful clips.

10. A vehicle, characterized in that: The vehicle comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for generating a wonderful outdoor self-driving video as described in any one of claims 1 to 8.

Images