Driving video generation method and device, program product and vehicle

Abstract

The invention provides a driving video generation method and device, a program product and a vehicle. The driving video generation method provided by the invention can perform multiple screening on road sections and driving data, can accurately determine the important operation of the user in the driving process, and generates the video that the user executes the important operation based on the sensing data corresponding to the important operation. The video comprises an in-vehicle video and an out-vehicle video when the user executes the operation. Therefore, the vehicle can automatically record the important operation (such as the drifting operation) executed by the user and generate the driving video for executing the driving operation, so that the user can check and share the driving video again, diversified visual experience is provided, and the fun of driving experience is enriched.

Description

Technical Field

[0001] The present application relates to the field of automobile data processing, and in particular to a driving video generation method, device, program product and vehicle. Background Art

[0002] With the continuous development of automobile technology, the sensors integrated into vehicles are becoming more and more diverse. Smart vehicles are often integrated with multiple external cameras and internal cameras. The external cameras can be used to sense the external environment when the vehicle is driving, while the internal cameras are often only used to identify the driver of the vehicle, and their actual utilization rate is low.

[0003] Therefore, how to effectively utilize various sensor data on the vehicle (especially the in-vehicle sensor data) to improve the driver's driving experience is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a driving video generation method, device, program product and vehicle, which can determine the vehicle driving data of the user when performing important driving operations by screening the driving operations, and display the in-vehicle operation video in the vehicle driving data to improve the driver's driving experience.

[0005] In a first aspect, the present application provides a driving video generation method, comprising: recording driving data and sensor data when a vehicle is driving on a target road, wherein the driving data includes a user's driving operation and a driving state of the vehicle. Determining target driving data including a target driving operation based on the driving state of the vehicle and / or the user's driving operation. Determining target sensor data synchronized with the target driving data from the sensor data. Generating a vehicle driving video of the vehicle based on the target sensor data, wherein the vehicle driving video includes an in-vehicle operation video when performing the target driving operation and an external vehicle environment video.

[0006] In a second aspect, the present application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the driving video generation method described in the first aspect.

[0007] In a third aspect, the present application provides a computer program product, comprising: a computer program / instructions, which, when executed by a computer program / instruction processor, are used to implement the steps in the driving video generation method described in the first aspect.

[0008] In a fourth aspect, the present application provides a vehicle comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor implements the driving video generation method described in the first aspect.

[0009] The present application provides a driving video generation method, device, program product and vehicle. The driving video generation method provided in the present application can perform secondary screening of road sections and driving data, can accurately determine the important operations of the user during driving, and generate a video of the user performing the important operation based on the sensor data corresponding to the important operation, and the video includes the in-vehicle video and the out-of-vehicle video when the user performs the operation. As a result, the vehicle can automatically record the important operations performed by the user (such as drift operations) and generate a driving video of the driving operation, so that the user can review and share it, thereby providing a diverse visual experience and enriching the fun of the driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 It is a flowchart of a method for generating a driving video provided by an exemplary embodiment of the present application.

[0012] Figure 2 It is a flowchart of a vehicle control method based on a track mode provided by an exemplary embodiment of the present application.

[0013] Figure 3 It is a flowchart of a driving video generation method based on a track mode provided by an exemplary embodiment of the present application.

[0014] Figure 4 It is a flowchart of a method for determining a driving video based on target sensor data provided by an exemplary embodiment of the present application.

[0015] Figure 5 It is a system module diagram of a driving video generating device provided by an exemplary embodiment of the present application.

[0016] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] Application Overview

[0019] With the continuous advancement of vehicle technology, the sensor data on vehicles is gradually increasing. In the actual use of vehicles, only the external sensor data (such as external video, external radar data, etc.) in the sensor data will be used as log information to reflect the road conditions during the vehicle's driving process, while the sensor utilization rate of other sensors is low.

[0020] The present application creatively discovers that in order to improve the driving experience of the vehicle in track mode, the aforementioned sensor data can be used to generate a driving video of the vehicle in track mode. In particular, a video of the user performing specific driving operations (such as drifting, launch start, etc.) can be generated based on the in-vehicle data, thereby facilitating users to display on social media to improve the user's driving experience.

[0021] Specifically, the driving video generation method provided by the present application can perform secondary screening of road sections and driving data, accurately determine the important operations of the user during driving, and generate a video of the user performing the important operation based on the sensor data corresponding to the important operation, and the video includes the in-vehicle video and the out-of-vehicle video when the user performs the operation. As a result, the vehicle can automatically record the important operations performed by the user (such as drifting operations) and generate a driving video of the driving operation so that the user can review and share it, thereby providing a diverse visual experience and enriching the fun of the driving experience.

[0022] The following is combined with Figures 1 to 6 The driving video generation method, device, program product and vehicle provided in this application are described in detail.

[0023] Exemplary driving video generation method

[0024] like Figure 1 As shown, the present application provides a method for generating a driving video. Figure 1 The method shown may be executed by one or more computing devices on a vehicle (eg, an intelligent vehicle driving assistance system, an onboard video recording system, an onboard monitoring system, etc.). Figure 1 As shown, the method may include the following steps:

[0025] S110 , recording driving data and sensor data of the vehicle when traveling on the target road.

[0026] S120: Determine target driving data including a target driving operation based on the driving state of the vehicle and / or the driving operation of the user.

[0027] S130 . Determine target sensor data synchronized with the target driving data from the sensor data.

[0028] S140: Generate a vehicle driving video of the vehicle based on the target sensing data.

[0029] In the aforementioned S110, driving data may refer to the perception data of the vehicle's own body conditions during driving. Among them, the driving data may at least include driving operations that reflect the driver's operating conditions and driving status that reflects the vehicle's driving conditions. Driving operations may include the driver's (also recorded as the user) operation records of various controls in the vehicle (such as pedals, gear levers, steering wheels, etc.). The driving status may include the vehicle's motion state (such as speed, posture, etc.) and the working conditions of various devices in the vehicle (such as the braking conditions of each tire, the working conditions of the engine, etc.).

[0030] Sensor data may refer to external data (i.e., data outside the vehicle) collected by various sensors on the vehicle. For example, sensor data may include external sensor data collected by external sensors (such as external cameras, laser radars, millimeter-wave radars, etc.) (such as external monitoring data collected by external cameras, and point cloud data detected by laser radars, millimeter-wave radars, etc.). For another example, sensor data may also include internal sensor data collected by internal sensors (such as internal cameras, presence sensors, etc.) (such as internal monitoring data collected by internal cameras, passenger number information collected by presence sensors, etc.).

[0031] Considering that the present application mainly involves a method for generating a driving video, the subsequent processing of the aforementioned sensor data mainly involves the processing of monitoring data. For example, the in-vehicle monitoring data of a user performing a specific driving operation can be extracted as an in-vehicle operation video when the user performs the specific driving operation. For another example, the out-of-vehicle monitoring video can also be processed to determine an out-of-vehicle environment video reflecting the vehicle's driving environment.

[0032] The target road may refer to the road corresponding to the execution of the video driving generation method provided in this application, that is, the driving video within the target road section may be generated based on the video driving generation method provided in this application. For example, the target road may be matched with the aforementioned track mode, so that the track corresponding to the track mode (generally a closed loop track) is used as the target road.

[0033] In some embodiments, the aforementioned S110 can be executed based on the determination of the target road. For example, after the user manually selects the target road (such as the aforementioned track mode or recording through interaction), the current road can be used as the target road to collect and cache data. For another example, the data on the road can also be cached in real time, so that after the road is configured as the target road, the cached data can be called for subsequent processing. For details about video generation in track mode, please refer to Figure 2 , 3 And the related descriptions are not repeated here.

[0034] In some embodiments, the driving video generation method provided by the present application may not be limited to the track mode, and may also select a suitable road based on actual needs. For example, to facilitate the playback of vehicle accidents, the driving video generation method provided by the present application may also be applied to collision / possible collision scenarios, thereby configuring the risky road section as the aforementioned target road to determine the driving video at the time of collision / high collision risk, which may facilitate subsequent accountability.

[0035] Specifically, for a risky road section, the target road section can be determined first, and then the relevant data of the risky road section can be called. That is, the driving data and sensor data of the vehicle can be collected first. Then, the road section risk of the current road section of the vehicle is determined based on the sensor data. Finally, in response to the road section risk meeting the risk threshold condition, the current road section is determined as a target road and the driving data and sensor data of the target road are determined.

[0036] That is, the above process can be understood as real-time collection and caching of relevant data (driving data and sensor data) during the normal operation of the vehicle. Therefore, when the current road section is determined to be a risky road section, the relevant data is called for subsequent processing to determine the vehicle driving video containing the risky road section and the user's driving operation on the risky road section.

[0037] The aforementioned risky road section may refer to a road section where there is a risk of collision or other vehicle accident. In addition, a road section where a vehicle accident has occurred may also be a risky road section. Among them, the risky road section may be characterized by the road section risk, that is, the risky road section may be a road section where the risk is higher than a preset threshold (recorded as the aforementioned risk threshold condition).

[0038] In some embodiments, the road section risk can generally be determined by the distance between the vehicle and the obstacle. For example, the possibility of collision between the vehicle and the obstacle can be estimated as the road section risk based on the movement state of the obstacle, the movement state of the vehicle itself, and the relative distance between the two. In addition, the road section risk can also be adjusted according to the driving state of the vehicle itself. For example, the vehicle's own control ability (such as the output capacity of the motor, the braking ability of the brake pads, etc.) can also be used as an important parameter to adjust the road section risk. For details, please refer to the relevant technology, which will not be repeated here.

[0039] After determining the aforementioned risky sections, considering that the cause of the accident may be on the previous road, when determining the target road based on the risky sections, the various sections associated with the accident / possible accident can be determined based on the actual driving conditions, thereby taking the continuous sections as target roads.

[0040] In particular, the aforementioned target road section can also be specially configured as a road section with risks, such as a sharp turn road section, an intersection, etc. That is, the aforementioned road section risk can be corrected based on the road section type, wherein the road section that is more prone to risks (such as a complex road section) has a higher correction value.

[0041] In the aforementioned S120, the target driving operation may be an important driving operation that needs to be recorded. Therefore, the method provided by the present application can screen out the user's important driving operations to generate corresponding driving videos.

[0042] In some embodiments, the target driving operation to be identified can be configured in advance. Considering the user's desire to share, driving operations such as drifting, launch start, competitive obstacle avoidance, and speed limit removal can be configured as key driving operations. When the system detects that the corresponding driving operation is triggered, it determines that the corresponding driving operation is the target driving operation, thereby generating a corresponding driving video to facilitate the driver to share the highlight moments of driving.

[0043] Thus, the aforementioned S120 can be determined by matching the current driving operation with the preset cloud driving operation. Specifically, the matching degree between the driving operation and the cloud driving operation can be determined based on the driving state and / or the driving operation. Then, each driving operation whose matching degree meets the matching degree threshold condition is determined as the target driving operation.

[0044] Specifically, considering that the aforementioned high-beam driving operation is generally more complicated, it is necessary to combine the vehicle state caused by the driving operation when making a judgment. In this way, the driving operation and the vehicle state can be quantitatively represented, so as to facilitate the matching of the current driving operation with the cloud driving operation. For example, the driving operation and the vehicle state can be represented by a specific vector, and the similarity between the vectors can be used as the aforementioned matching degree.

[0045] In some embodiments, the cloud driving operation can be configured based on user selection. For example, the user can select the driving operation for which video generation is required in the vehicle computer interaction interface. In some embodiments, the cloud driving operation can also be determined based on the driving scenario, that is, the corresponding cloud driving operation can be automatically determined based on the target road.

[0046] In some embodiments, the aforementioned target driving operation can also be determined based on the actual scenario. That is, after the target road is determined, the specific driving operation performed in the target road can be configured as the target driving operation. For example, when the aforementioned target road is / includes a dangerous section, the corresponding target driving operation can be a driving operation related to emergency avoidance, such as emergency braking, emergency steering to avoid collision, etc. Thus, a driving video of the vehicle on a risky section can be provided when a vehicle collides, so as to assist the owner in analyzing the cause of the accident and determining the party responsible for the collision; if the vehicle does not collide, the target driving operation, that is, the emergency avoidance driving operation can also be recorded as a highlight moment, and the same driving operation (such as training data for assisted driving) can be taken when the same situation is encountered next time to avoid safety accidents.

[0047] In the aforementioned S130, the target sensor data may refer to sensor data synchronized with the aforementioned target driving operation time. That is, considering that the vehicle collects driving data and sensor data at the same time and generally sets a timestamp (or other time identifier) ​​in the data. Then in the aforementioned S130, based on the timestamp of the aforementioned target driving data containing the target driving operation, the data with the same timestamp can be called from the sensor data as the target sensor data. Thus, the target sensor data may include monitoring data when performing the target driving operation, so that the corresponding vehicle driving video can be generated in the subsequent steps.

[0048] In the aforementioned S140, the vehicle driving video may refer to a video file that can reflect the target driving operation. Specifically, the vehicle driving video includes an in-vehicle operation video and an out-of-vehicle environment video when the target driving operation is performed.

[0049] The in-vehicle operation video can be determined based on the in-vehicle monitoring video of the in-vehicle sensor (i.e., the in-vehicle sensor data in the target sensor data), that is, a portion of the video when the user performs the target driving operation in the in-vehicle monitoring video can be intercepted as the in-vehicle operation video. Similarly, a portion of the video when the target driving operation is performed in the out-vehicle monitoring video (i.e., the out-vehicle sensor data in the target sensor data) can also be intercepted as the out-vehicle environment video.

[0050] In some embodiments, considering that the external sensors are generally unable to obtain monitoring images of the vehicle on the road, data fusion rendering is also performed when determining the external environment video. For details, please refer to Figure 4 and its related description.

[0051] In some embodiments, when determining the vehicle driving video, the in-vehicle operation video and the vehicle exterior environment video may be mixed and cut, thereby reasonably splicing the in-vehicle operation video and the vehicle exterior environment video to generate an ornamental vehicle driving video. For example, the execution time of the aforementioned target driving operation may be used as a key editing time point, and the script or transition of the racing scene in the movie / TV series may be applied to the key editing time point to highlight the target driving operation performed by the user (such as generating a close-up shot of the target driving operation based on the in-vehicle operation video), thereby improving the user's satisfaction in performing the target driving operation.

[0052] In some embodiments, the aforementioned in-vehicle operation video and the video of the outside environment may not be edited, but may be directly spliced ​​based on the acquisition time to ensure synchronous playback of the in-vehicle operation video and the video of the outside environment, thereby facilitating the review of specific scenes (such as the review of traffic accidents). In addition, when splicing directly, the parameters inside the vehicle may also be displayed in real time in the video, providing convenience for vehicle testing.

[0053] Therefore, based on the aforementioned driving video generation method, multiple screening of road sections and driving data can be performed to accurately determine the important operations of the user during driving, and a video of the user performing the important operation can be generated based on the sensor data corresponding to the important operation, and the video includes the in-car video and the out-of-car video when the user performs the operation. Therefore, the vehicle can automatically record the important operations performed by the user (such as drifting operations) and generate a driving video of the driving operation so that the user can review and share it, thereby providing a diverse visual experience and enriching the fun of the driving experience.

[0054] Track mode and driving video generation based on track mode

[0055] Track mode may refer to a special driving mode for a vehicle that is suitable for a racetrack. In track mode, the vehicle's parameters are adapted to the racetrack driving requirements to improve the vehicle's performance on the track and provide the user with a racing-like driving experience. To avoid interfering with normal vehicle operation, track mode is generally turned on in a closed-loop track to ensure that the vehicle does not affect normal traffic. The loop track may be a racetrack or a closed circular venue selected by the user, which is not specifically limited here.

[0056] Before further describing the generation of driving videos in the track mode, the present application describes in detail the configuration of the vehicle in the track mode. That is, the present application provides a vehicle control method based on the track mode ( Figure 2 ).like Figure 2 As shown, process P200 may include the following steps:

[0057] S210 . In response to a user's interactive operation on a track mode control, control the vehicle to be in track mode.

[0058] S220: Determine a track map of at least a portion of the closed-loop track based on the position information of the vehicle.

[0059] S230: Determine a track starting point of at least a portion of the closed-loop track based on the track map.

[0060] In the aforementioned S210, considering that the track mode generally touches some restrictions of the vehicle (such as the speed limit), the user's manual confirmation is generally required when the vehicle enters the track mode. For example, the user can select the track mode and enter it through the interactive control of the vehicle system. For another example, the vehicle can recommend the track mode to the user based on its own position (such as being within the position range corresponding to the track), and enter the track mode through the user's manual confirmation.

[0061] When the vehicle enters the track mode, the vehicle will self-check its own situation and adjust the body configuration to adjust the vehicle's motion orientation to meet the requirements of the track. For example, during the self-check, you can check whether the brake pads are dedicated to the track, whether the vehicle's tire pressure meets the track requirements, etc. For another example, when adjusting the body configuration, you can adjust the chassis, suspension and other body structures to make the vehicle's orientation more sporty to meet the track requirements. For the specific configuration of the track mode, please refer to the relevant technology, which will not be repeated here.

[0062] In the aforementioned S220, the closed-loop track may refer to an actual road of the vehicle in the racing venue, and the at least partially closed-loop track may refer to an area where the vehicle is traveling. The at least partially closed-loop track may include a partial track and a multi-circle track, which may be specifically set in the vehicle. The at least partially closed-loop track is generally configured as an integer number of laps of the closed-loop track.

[0063] To facilitate vehicle driving, in track mode, the vehicle may provide a track map of a closed-loop track and a track starting point. The track map may call a track map of at least part of the closed-loop track from a database based on the vehicle's location information. The track starting point may be marked in the track map and thus determined.

[0064] In some embodiments, considering the situation that there is no pre-setting, the aforementioned track map can also be determined based on the test driving data of the track map. That is, the test driving data of the vehicle driving on at least a part of the closed-loop track can be determined first. Then, the track map of at least a part of the closed-loop track can be determined based on the historical position data and historical sensor data in the test driving data.

[0065] For example, when the closed-loop track of the current scene is not pre-set in the database, the user can drive the vehicle on the closed-loop track for a test drive to determine the test drive data (mainly involving the sensor data of the external sensors). In the subsequent steps, the closed-loop track can be modeled based on the collection position of each sensor data to determine the track map. Exemplarily, the aforementioned process can be implemented by SLAM (Simultaneous Localization and Mapping). In particular, the vehicle can perform multiple laps during the test, thereby optimizing the aforementioned track map based on the loop frame.

[0066] In addition, to accurately determine the track map, when the track map can be called, the user can also drive the vehicle to determine the test driving data, thereby constructing an accurate mapping of the track map to the actual track and modifying the track map.

[0067] In some embodiments, in the aforementioned S230, the track starting point in the track mode may be the actual starting point of the closed-loop track or the starting point manually set by the user. Specifically, the track starting point may be determined based on the user's interactive behavior. For example, the closed-loop track may be displayed on the vehicle computer interactive page, and the user may mark / select the track starting point.

[0068] In some embodiments, when the closed-loop track has an actual starting point and is not marked on the map, the map can be made based on the aforementioned test driving data. For example, the ground mark can be determined through point cloud data to determine the actual starting point.

[0069] In some embodiments, in order to improve the user's driving experience on a closed-loop track, when the user drives based on the track mode, multiple driving suggestions can be provided to the user based on the track map to prompt the user to perform appropriate driving operations. That is, the aforementioned P200 can also include the following steps:

[0070] S240: Determine multiple driving suggestions for multiple road sections in the track map based on the track map.

[0071] Driving advice may refer to prompt information presented in an audio-visual form to guide the driver to perform a specific driving operation. In particular, the driving advice may include prompt information of a suggested driving operation that matches a corresponding road section. The prompt information may be specifically used to prompt the user to perform the corresponding suggested driving operation on the corresponding road section.

[0072] For example, driving advice can be characterized as a continuous voice reminder to perform a specific operation. Taking assisted drift as an example, the user can be prompted to "please step on the brakes deeply", "please turn the steering wheel in the direction of XX", etc. For another example, driving advice can also be presented in the car interface. Exemplarily, driving advice can include driving direction suggestions for the current location. Specifically, the target driving method can be displayed in the car, and the user is prompted when the current driving direction is different from the target driving direction (such as by color distinction, green indicates the correct direction, and red indicates the wrong direction).

[0073] In some embodiments, the aforementioned S240 may determine corresponding driving suggestions based on each road section in the track map. Specifically, driving suggestions may be pre-configured for each road section, thereby associating each road section in the track map with corresponding driving suggestions. For example, in a starting section, the driving suggestion may be a driving suggestion for a launch start. In a turning section, the driving suggestion may be a driving suggestion for a drift operation. In a long straight section, the driving suggestion may be a driving suggestion for a temporary speed limit release operation. For details, please refer to more descriptions of related technologies.

[0074] Based on the aforementioned track mode, the present application also provides a driving video generation method based on the track mode ( Figure 3 ),like Figure 3 As shown, process P300 may include the following steps:

[0075] S310: In response to the vehicle being in the track mode, configure at least a portion of the closed-loop track where the vehicle is located as a target road.

[0076] S320: When the vehicle passes the starting point of at least a portion of the closed-loop track, start recording the driving data and sensor data of at least a portion of the closed-loop track until the vehicle leaves the at least portion of the closed-loop track.

[0077] That is, based on the aforementioned S310 and S320, the target road can be determined and the driving data and sensor data of the target road can be collected.

[0078] After determining the aforementioned driving data and sensor data, the target driving data including the target driving operation can be determined based on the aforementioned P100 related method (S120, S130), and then the target sensor data synchronized with the target driving data can be called to determine the vehicle driving video.

[0079] In some embodiments, at least a portion of the closed-loop track may also be provided with a track end point, so that when the vehicle passes the track end point of the last lap, it is determined that the vehicle has left the at least a portion of the closed-loop track and completed the track driving. The track end point is similar to the aforementioned track start point, and its determination method can refer to the relevant description of the track start point.

[0080] In some embodiments, considering the aforementioned driving suggestions, the target driving operation may also be determined based on the current driving operation and the driving suggestions of the corresponding road section. Accordingly, the aforementioned P300 may further include the following steps:

[0081] S330: In response to the vehicle passing through a target section of the track map, determining a matching degree between the driving operation of the vehicle on the target section and the driving suggestion based on driving data of the vehicle on the target section.

[0082] S340: Determine each driving operation whose matching degree satisfies a matching degree threshold condition as a target driving operation.

[0083] S350: Determine target driving data based on the target driving operation.

[0084] Similar to the matching degree of the aforementioned cloud driving operation, the matching degree between the driving operation of the vehicle on the target road section and the driving suggestion can also be understood as the similarity between the two driving operations or / and driving states. For example, it can be represented as the similarity of feature vectors.

[0085] In some embodiments, the aforementioned S330 to S350 can be executed during the execution of the aforementioned S320. That is, at this time, the target road section can reflect the current road section, and it can be determined whether the driving operation of each current road section is consistent with the driving suggestion as the vehicle travels. In some embodiments, the aforementioned S330 to S350 can also be executed after the execution of the aforementioned S320. That is, at this time, the target road section can be any of the sections in at least part of the closed-loop track, that is, after the data is collected, it can be determined one by one whether the driving operation of each road section matches the driving suggestion.

[0086] After determining the aforementioned target driving operation and target sensor data, a corresponding vehicle driving video can be generated. For the vehicle driving video in track mode, the target driving operation generally includes a launch start at the start of the track, a drift operation at a curve, and a speed limit release on a straight section. Thus, a vehicle driving video of the user performing each of the aforementioned driving operations can be generated, wherein each driving operation can also provide a close-up of the user's driving operation in the car to enhance the user's driving experience.

[0087] In some embodiments, in order to enable users to more intuitively feel the difference between their driving skills and those of other users and enhance competitiveness, in some embodiments, in response to the vehicle completing at least part of a closed-loop track, the driving performance of the vehicle in at least one lap of the track can be determined. Specifically, the driving performance may include the completion time in at least part of the closed-loop track, and may also include parameters such as the score of the driving operation and the completion time ranking.

[0088] As an example, the driving operation can be scored according to the completion of the user's driving operation in at least one lap of the track. For example, when the current driving operation is determined to be the target driving operation (i.e., the perfect driving operation), points can be added accordingly; when it is detected that the current driving operation deviates from the correct driving operation and the driving operation is not standardized, points can be deducted accordingly. Finally, when at least part of the closed-loop track is completed, the overall driving operation score is calculated.

[0089] In some embodiments, a timer or timing system may be used to record the time taken by the vehicle to complete at least a portion of the closed-loop track, and a ranking may be generated by comprehensively comparing the cloud data based on the driving operation score and completion time obtained above. The cloud data may include driving results contributed by other users and the user's own driving results of completing at least a portion of the closed-loop track in different time periods.

[0090] In some embodiments, the driving performance of the vehicle on the at least partially closed-loop track may be displayed on a display screen of the vehicle.

[0091] In some embodiments, a video report can be generated based on target driving data, driving suggestions, and driving performance. In order to enhance interactivity and satisfy the user's desire to share, the user can transmit the driving report to a website / program corresponding to the sharing space, such as WeChat, QQ, Moments, etc., through interactive operations on the sharing control to achieve sharing of driving performance.

[0092] Exemplary method for determining driving video

[0093] Figure 4 FIG. 1 is a flow chart of a method for determining a driving video based on target sensor data provided by an exemplary embodiment of the present application. Figure 4 As shown, process P400 may include the following steps:

[0094] S410 , determining the in-vehicle operation video based on the in-vehicle monitoring data of the target sensing data.

[0095] S420: Determine at least one vehicle exterior environment video based on the vehicle exterior monitoring data of the target sensing data.

[0096] S430: Edit the in-vehicle operation video and the out-of-vehicle environment video based on the target driving operation to determine the vehicle driving video.

[0097] As mentioned above, the in-vehicle operation video reflects the in-vehicle driving operation when executing the target driving data, and the outside-vehicle environment video reflects the outside-vehicle driving environment and the vehicle when executing the target driving data. In order to make the vehicle driving video highlight the target driving operation, the vehicle driving video often needs to synchronously display the in-vehicle operation video and the outside-vehicle environment video corresponding to the target driving operation.

[0098] In some embodiments, when determining the aforementioned in-vehicle operation video and at least one vehicle exterior environment video, the time information of the relevant data may be retained, so that when executing S430, the in-vehicle operation video and at least one vehicle exterior environment video are aligned based on the time information, thereby ensuring the synchronization of the in-vehicle operation video and at least one vehicle exterior environment video in the vehicle driving video. Thus, the vehicle driving video may synchronously display the in-vehicle operation video and the vehicle exterior environment video corresponding to the target driving operation.

[0099] In some embodiments, considering that the off-vehicle monitoring data of the aforementioned target sensor data generally does not include the vehicle itself, that is, the off-vehicle sensor is generally set on the vehicle and facing outward, the field of view of the off-vehicle sensor does not include the vehicle itself, that is, it is difficult to directly render a video of the vehicle driving on the road based on the aforementioned off-vehicle monitoring data.

[0100] In some embodiments, to determine the vehicle exterior environment video including the vehicle itself, the aforementioned S420 may further include the following steps:

[0101] S421. Determine, based on the external vehicle data of the target sensing data, at least one angle of the external driving environment of the vehicle when the vehicle performs the target driving operation.

[0102] S422: Display the vehicle in the external driving environment at various angles based on the video features corresponding to the target driving operation, and determine at least one external environment video.

[0103] The vehicle exterior driving environment may refer to a video / 3D model of the vehicle exterior environment that does not include the vehicle itself and is determined based on vehicle exterior data at a specific angle. Considering that the vehicle is in continuous motion, when determining the vehicle exterior environment video, although the vehicle itself is not sensed by the vehicle exterior sensor, it can be filled based on historical data frames, thereby determining the vehicle exterior environment video that does not include the vehicle.

[0104] After determining the aforementioned vehicle exterior environment video (i.e., vehicle exterior driving environment), the vehicle can be rendered in the vehicle exterior driving environment. Specifically, the vehicle model can be mapped to a corresponding position in the video according to the vehicle's motion state, thereby determining the vehicle exterior environment video.

[0105] It is understandable that the above process is similar to the panoramic image (bird's eye view) function of the vehicle. Different from the panoramic image, the generation process of the above-mentioned vehicle exterior environment video is not limited to fixing the vehicle in the center of the picture. For example, a certain road section can be fixedly rendered, and then according to the movement trajectory of the vehicle, the movement process of the vehicle entering and leaving in the fixed scene can be rendered.

[0106] In some embodiments, in order to further improve the characterization capability of the target driving operation, special effects can be added based on the target driving operation when rendering the vehicle in the off-vehicle driving environment. That is, based on the triggering of the target driving operation, the rendering special effects of the corresponding vehicle can be determined, and the special effects can be rendered to the vehicle in the vehicle driving video. For example, when the target driving operation is "drifting" or high-speed driving, smoke or spark special effects can be added to the exhaust pipe at the rear of the car to increase the visual impact. Furthermore, engine sound effects can be added to the driving video at the same time to increase the sense of presence.

[0107] In some embodiments, in response to the user's interactive operation on the interactive control of the style theme, a rendering effect of the corresponding style can be added to the driving video. The interactive control of the style theme may include themes such as GT racing, Asphalt, Need for Speed, Forza Motorsport, QQ Speed, etc.; it may also be a custom interactive control, and the user can set the theme style he likes by himself, which is not specifically limited here.

[0108] In some embodiments, the user can save the above special effects to the driving video by saving the interactive control.

[0109] In some embodiments, in response to the interactive operation of the interactive control of the user perspective control, the video of the corresponding perspective can be enlarged in the driving video. Exemplarily, the perspective control may include a bird's-eye view, a tire perspective, a driver's perspective in the car, a co-driver's perspective in the car, an internal DMS expression perspective, etc., which are not specifically limited here.

[0110] Exemplary electronic devices and computer program products

[0111] Combination of the above Figures 1 to 4 , describes the method embodiment of the present application in detail, and the device embodiment of the present application is described in detail below. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the part not described in detail can refer to the previous method embodiment.

[0112] The present application also provides a driving video generation device, including a module for implementing the driving video generation method provided by the present application. Figure 5 As shown, Figure 5 It is a system module diagram of a driving video generating device provided by an exemplary embodiment of the present application. The driving video generating device 500 provided by the present application may include a data acquisition module 510, a data screening module 520 and a video generating module 530.

[0113] The data acquisition module 510 can be used to record driving data and sensor data when the vehicle is traveling on the target road, wherein the driving data includes the user's driving operation and the driving status of the vehicle.

[0114] The data screening module 520 may be used to determine target driving data including a target driving operation based on the driving state of the vehicle and / or the driving operation of the user, and may also be used to determine target sensing data synchronized with the target driving data from the sensing data.

[0115] The video generation module 530 can be used to generate a vehicle driving video of the vehicle based on the target sensing data, wherein the vehicle driving video includes an in-vehicle operation video and an out-of-vehicle environment video when performing the target driving operation.

[0116] For the specific definition of the driving video generation device, please refer to the definition of the driving video generation method above, which will not be repeated here. Each module in the above-mentioned driving video generation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0117] The present application also provides a vehicle, which includes the vehicle system provided by the present application and can execute the driving video generation method provided by the present application. The vehicle provided by the present application can be a fully automatic driving vehicle or a semi-automatic driving vehicle. The specific model of the vehicle provided by the present application can be a car, a commercial vehicle, a truck, a bus, a tram, etc.

[0118] The present application also provides an electronic device, such as Figure 6 The electronic device 600 provided in the present application includes a memory 610, a processor 620, and an input / output interface 630. The memory 610, the processor 620, and the input / output interface 630 are connected through an internal connection path, the memory 610 is used to store instructions, and the processor 620 is used to execute the instructions stored in the memory 610 to control the input / output interface 630 to receive input data and information, and output data such as operation results.

[0119] It should be understood that in the embodiment of the present application, the processor 620 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the technical solution provided in the embodiment of the present application.

[0120] The memory 610 may include a read-only memory and a random access memory, and provides instructions and data to the processor 620. A portion of the processor 620 may also include a nonvolatile random access memory. For example, the processor 620 may also store information on the device type.

[0121] During the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 620 or an instruction in software form. The vehicle auxiliary braking method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or it can be completed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 610, and the processor 620 reads the information in the memory 610 and completes the steps of the above method in combination with its hardware. To avoid repetition, it is not described in detail here. The present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction processor in the computer program product provided by the present application is executed, the driving video generation method provided by the present application can be implemented.

[0122] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0123] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0124] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0127] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, operation and maintenance management (operation administration and maintenance, OAM) or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0128] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A driving video generation method, characterized in that: include: Recording driving data and sensor data when the vehicle is traveling on a target road, wherein the driving data includes a user's driving operation and a driving state of the vehicle; determining target driving data including a target driving operation based on the driving state of the vehicle and / or the driving operation of the user; determining target sensor data synchronized with the target driving data from the sensor data; A vehicle driving video of the vehicle is generated based on the target sensing data, wherein the vehicle driving video includes an in-vehicle operation video and an out-of-vehicle environment video when the target driving operation is performed.

2. The driving video generation method according to claim 1, characterized in that: The recording of driving data and sensor data when the vehicle is traveling on the target road includes: In response to the vehicle being in a track mode, configuring at least a portion of a closed-loop track on which the vehicle is located as the target road; When the vehicle passes the starting point of the at least partially closed-loop track, the vehicle starts recording the driving data and sensor data of the at least partially closed-loop track until the vehicle leaves the at least partially closed-loop track.

3. The driving video generation method according to claim 2, characterized in that: The method further comprises: In response to a user's interactive operation on a track mode control, controlling the vehicle to be in the track mode; Determining a track map of the at least partially closed-loop track based on the position information of the vehicle; A track starting point of the at least partially closed-loop track is determined based on the track map.

4. The driving video generation method according to claim 3, characterized in that: The determining of the track map of the at least partially closed-loop track based on the position information of the vehicle comprises: Determining test driving data of the vehicle when driving on the at least partially closed-loop track; A track map of the at least partially closed-loop track is determined based on the historical position data and the historical sensor data in the test driving data.

5. The driving video generation method according to claim 3, characterized in that: The method further comprises: Determining a plurality of driving suggestions for a plurality of road sections in the track map based on the track map, wherein the driving suggestions include prompting information of a suggested driving operation matched with a corresponding road section, and the prompting information is used to prompt a user to perform the corresponding suggested driving operation on the corresponding road section; The determining of target driving data including a target driving operation based on the driving state and / or the driving operation includes: In response to the vehicle passing through a target section of the track map, determining a matching degree between the driving operation of the vehicle on the target section and the driving suggestion based on driving data of the vehicle on the target section; Each driving operation whose matching degree satisfies a matching degree threshold condition is determined as the target driving operation.

6. The driving video generation method according to claim 3, characterized in that: Also includes: In response to the vehicle completing the at least partially closed-loop track, determining and displaying a driving performance of the vehicle on the at least partially closed-loop track, wherein the driving performance includes a completion time on the at least partially closed-loop track; In response to the user's interactive operation on the sharing control, the driving performance and / or the vehicle driving video is transmitted to a website / program corresponding to the sharing control.

7. The driving video generation method according to claim 1, characterized in that: The recording of driving data and sensor data of the vehicle on the target road includes: Collecting driving data and sensor data of the vehicle; Determining a road section risk of a current road section of the vehicle based on the sensor data; In response to the road section risk satisfying a risk threshold condition, the current road section is determined to be the target road and the driving data and the sensor data of the target road are recorded.

8. The driving video generation method according to claim 1, characterized in that: The determining target driving data based on the driving state and / or the driving operation includes: Determining a matching degree between the driving operation and a cloud-based driving operation based on the driving state and / or the driving operation; Each driving operation whose matching degree satisfies a matching degree threshold condition is determined as the target driving operation.

9. The driving video generation method according to claim 1, characterized in that: The generating of the vehicle driving video of the vehicle based on the target sensor data comprises: determining an in-vehicle operation video based on the in-vehicle monitoring data of the target sensing data, wherein the in-vehicle operation video reflects the in-vehicle driving operation when executing the target driving data; Determine at least one vehicle exterior environment video based on the vehicle exterior monitoring data of the target sensing data, wherein the vehicle exterior environment video reflects the vehicle exterior driving environment and the vehicle when the target driving data is executed; The in-vehicle operation video and the out-vehicle environment video are edited based on the target driving operation to determine the vehicle driving video, wherein the vehicle driving video synchronously displays the in-vehicle operation video and the out-vehicle environment video corresponding to the target driving operation.

10. The driving video generation method according to claim 9, characterized in that: The determining of at least one vehicle exterior environment video based on the vehicle exterior monitoring data of the target sensor data comprises: Determining at least one angle of the vehicle's external driving environment when the vehicle performs the target driving operation based on the external vehicle data of the target sensing data; The at least one vehicle exterior environment video is determined by displaying the vehicle in the vehicle exterior driving environment at various angles based on video features corresponding to the target driving operation.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the driving video generation method according to any one of claims 1 to 10.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction processor is executed, it is used to implement the steps in the driving video generation method as described in any one of claims 1-10.

13. A vehicle, characterized in that: The vehicle comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores instructions that can be executed by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor implements the driving video generation method described in any one of claims 1-10.

Images