Video generation method, video generation device, vehicle-mounted device and storage medium

Abstract

The present application provides a video generation method, a video generation device, a vehicle-mounted device and a storage medium, the method comprising: obtaining a first video shot during vehicle driving and vehicle amplitude information when the first video is shot; the first video comprises a plurality of first video frames; determining an image quality threshold according to the vehicle amplitude information; detecting the image quality of each of the first video frames, determining the first video frame whose image quality is greater than or equal to the image quality threshold as a second video frame; identifying sensitive features in the second video frame, masking the sensitive features in the second video frame, and generating a second video based on the masked second video frame. The present application can assist in video generation and effectively improve the generation efficiency and quality of the video.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video generation method, a video generation device, a vehicle-mounted device and a storage medium. Background Art

[0002] Driving is one of the most popular ways to travel in recent years, and many users will share the video images taken along the way on social platforms. During the driving process, users are usually required to assist in shooting dynamic videos along the way, and the users are required to process sensitive objects (such as faces, license plates, etc.) that appear in the captured videos. In addition, due to the bumps and vibrations of the vehicle caused by complex road conditions, the quality of the video taken during the driving of the vehicle will also be poor. For example, a large number of blurred video frames will appear, which requires users to edit them later. These processes will consume a lot of users' time, the efficiency of generating videos is low, and the quality of the videos cannot be guaranteed. Summary of the invention

[0003] In view of the above, it is necessary to provide a video generation method, a video generation device, a vehicle-mounted device and a storage medium that can assist in video generation and effectively improve the efficiency and quality of generated videos.

[0004] According to one aspect of an embodiment of the present application, a video generation method is provided, the method comprising: obtaining a first video shot during vehicle driving and vehicle amplitude information when the first video is shot; the first video comprises multiple first video frames; determining an image quality threshold according to the vehicle amplitude information; detecting the image quality of each first video frame, and determining a first video frame whose image quality is greater than or equal to the image quality threshold as a second video frame; identifying sensitive features in the second video frame, and performing mask processing on the sensitive features in the second video frame to obtain a third video frame, and generating a second video based on the third video frame.

[0005] According to one aspect of an embodiment of the present application, a video generating device is provided, the video generating device comprising:

[0006] An acquisition module, used to acquire a first video shot during vehicle driving and vehicle amplitude information when the first video is shot; the first video includes a plurality of first video frames;

[0007] A determination module, used for determining an image quality threshold according to the vehicle amplitude information;

[0008] A detection module, used to detect the image quality of each first video frame, and determine the first video frame whose image quality is greater than or equal to the image quality threshold as the second video frame;

[0009] The recognition module is used to recognize sensitive features in the second video frame, perform mask processing on the sensitive features in the second video frame, obtain a third video frame, and generate a second video based on the third video frame.

[0010] According to one aspect of an embodiment of the present application, a vehicle-mounted device is provided, comprising a memory and at least one processor, wherein at least one instruction is stored in the memory, and a video generation method is implemented when the at least one instruction is executed by the at least one processor.

[0011] According to one aspect of an embodiment of the present application, a vehicle-mounted device is provided, wherein the vehicle-mounted device includes a memory and at least one processor, the memory stores at least one instruction, and the at least one instruction implements the above-mentioned video generation method when executed by at least one processor.

[0012] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, a video generation method is implemented.

[0013] The video generation method provided by the embodiment of the present application can achieve at least the following beneficial effects: determining the image quality threshold according to the amplitude range of the vehicle amplitude, screening the first video frame according to the image quality threshold, so that the image quality screening of the video frame is targeted and effective, avoiding the elimination of too many continuous first video frames of different image qualities due to a single image quality threshold, ensuring the coherence of the extracted video frames, thereby improving the video generation quality and efficiency, and reducing the video editing cost. At the same time, the sensitive features in the video frames that meet the image quality threshold are masked, thereby improving the security of the video frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0015] Figure 1 It is a flowchart of the video generation method provided in an embodiment of the present application.

[0016] Figure 2 It is a flowchart of establishing a mapping relationship between vehicle amplitude information and image quality threshold provided in an embodiment of the present application.

[0017] Figure 3 This is a flowchart for obtaining a first video provided in an embodiment of the present application.

[0018] Figure 4 This is an example diagram of the shooting request interface provided in an embodiment of the present application.

[0019] Figure 5 This is a flowchart for obtaining a third video provided in an embodiment of the present application.

[0020] Figure 6 This is an example diagram of the editing request information interface provided in an embodiment of the present application.

[0021] Figure 7 It is a flowchart of similarity detection provided in an embodiment of the present application.

[0022] Figure 8 This is a flowchart for obtaining the fourth video provided in an embodiment of the present application.

[0023] Fig. 9 It is a flowchart for determining vehicle amplitude information corresponding to a target time node provided by an embodiment of the present application.

[0024] Fig.10 It is an architecture diagram of the hardware device of the video generation method provided in an embodiment of the present application.

[0025] Fig.11 It is a module schematic diagram of the video generating device provided in an embodiment of the present application.

[0026] The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0027] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present application. The embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0030] To solve the above-mentioned background technical problems, Figure 1As shown, the video generation method provided in the embodiment of the present application specifically includes the following steps. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0031] In this embodiment, the video generation method can be applied to a vehicle-mounted device and a terminal (for example, a vehicle-mounted device) that are connected to each other by communication (for example, by communication via Wi-Fi or Bluetooth). Fig.10 As shown), for a vehicle-mounted device that needs to perform video generation, the function for video generation provided by the method of the present application can be directly integrated on the vehicle-mounted device, or run on the vehicle-mounted device in the form of a software development kit (SDK).

[0032] It should be noted that the application on the vehicle-mounted device is only an example and is not limited to this in actual application. For example, it can also be applied to terminal devices that communicate with the vehicle-mounted device, such as mobile phones, computers and other electronic devices.

[0033] 101. Acquire a first video shot during vehicle driving and vehicle amplitude information when the first video is shot; the first video includes a plurality of first video frames.

[0034] The vehicle-mounted device provided in an embodiment of the present application includes: one or more collection devices (such as a driving recorder, a camera device, etc. in the vehicle) installed at preset positions of the vehicle, and the preset positions may include but are not limited to the front windshield of the vehicle, the rear windshield of the vehicle, the side windows, etc.).

[0035] In addition, the collection device can also be connected to a user terminal (such as a mobile phone, tablet computer, notebook, etc.) for communication (such as Fig.10 The collection device may respond to the user at the user terminal (for example Fig.10 The terminal 2) shown in the figure initiates a shooting request to start the corresponding acquisition device to shoot the first video. For example, assuming that an application (Application, App) that controls the acquisition device is installed in the user terminal, the user can initiate a request or perform other operations through the App, thereby eliminating the need for the user to operate the acquisition device to shoot the first video, saving the user's time.

[0036] The first video is an original video including continuous first video frames, and the vehicle amplitude information indicates the maximum displacement or amplitude of the vehicle body from the oscillation center in the fluctuation or vibration caused by the vehicle being affected by different terrains when the vehicle is traveling. In one embodiment, the specific implementation method of obtaining the first video and the vehicle amplitude information will be described in conjunction with the specific flowchart below.

[0037] 102. Determine an image quality threshold according to the vehicle amplitude information.

[0038] Among them, the image quality of the first video frame can be expressed as one or more of the image attributes such as clarity, granularity / noise, color, brightness and tone reproduction / contrast, and optical distortion of the video frame. The quality of the image depends on the relative state between the acquisition device and the captured environment. Generally, the greater the relative state change between the acquisition device and the captured environment, the lower the image quality of the first video frame captured. The acquisition device installed on the vehicle is generally affected by the vehicle amplitude information. For example, when the vehicle suddenly drives to a raised or sunken ground, the vehicle suddenly decelerates or accelerates, and the vehicle shakes due to inertia (the vehicle amplitude is large at this time). At this time, the acquisition device installed on the vehicle changes greatly with the shooting environment due to the shaking of the vehicle, so that the image quality of the captured video frame is lower than the video frame captured when the vehicle is driving stably. It should be noted that the vehicle amplitude information changes during the driving process of the vehicle, that is, the image quality of each first video frame captured is not the same. Therefore, in order to maintain the coherence of the second video finally generated, the image quality threshold corresponding to different amplitude information is used to achieve image quality screening of the first video frame.

[0039] The image quality threshold may be negatively correlated with the vehicle amplitude information, that is, the larger the vehicle amplitude information is, the lower the image quality threshold is; and the smaller the vehicle amplitude information is, the higher the image quality threshold is.

[0040] In one embodiment, the video generation method further includes: detecting the vibration state of the vehicle; and determining the vehicle amplitude information (hereinafter referred to as amplitude) according to the vibration state.

[0041] In one embodiment, the vehicle-mounted device further includes one or more sensor devices (eg Fig.10 The sensor device includes a vibration sensor for detecting the vibration state of the vehicle.

[0042] In one embodiment, the vibration state includes the state of the vehicle when it vibrates up and down due to uneven road surface (hereinafter referred to as the target time node), and the amplitude or displacement (for example, 7 cm) of the up and down vibration of the vehicle can be determined according to the vibration sensor, and the amplitude or displacement is used as the amplitude of the vehicle.

[0043] In one embodiment, the amplitude of the vehicle may affect the image quality of the first video frame in the first video. For example, when the amplitude is large, the image quality of the corresponding first video frame will be low.

[0044] In one embodiment, if there is only one image quality threshold, if the image quality threshold is set high, the first video frames captured at the target time node with large vehicle amplitude may all be smaller than the image quality threshold. As a result, in the subsequent step 103, too many first video frames need to be eliminated at a certain time node, and fewer second video frames that meet the image quality requirements are left, resulting in the problem of incoherence of the generated video at a certain target time node. If the image quality threshold is set low, it may result in more image frames with poor image quality in the remaining second video frames.

[0045] By setting different image quality thresholds according to different amplitude ranges, the quality requirements for the first video frame taken when the amplitude of the vehicle is large can be reduced, the problem of video incoherence can be avoided, as many second video frames as possible can be obtained, the coherence of the generated video can be improved, and user needs can be better met. At the same time, the quality requirements for the first video frame can be increased when the amplitude of the vehicle is small, and the first video frame with poor image quality can be eliminated.

[0046] In one embodiment, a specific implementation of determining the image quality threshold will be described in conjunction with the specific flowchart below.

[0047] 103. Detect image quality of each first video frame, and determine a first video frame having an image quality greater than or equal to an image quality threshold as a second video frame.

[0048] In one embodiment, when the vehicle amplitude is detected to exceed a preset amplitude threshold (e.g., 5 cm), the image quality detection can be performed in real time on the first video frame within a preset time length before and after the vibration state occurs. The preset time length can be set according to the amplitude of the vehicle, and the larger the vehicle amplitude, the longer the preset time length.

[0049] In one embodiment, image quality detection can also be performed on the first video frame at preset time intervals (for example, 0.5 seconds). This can not only reduce the computing power of image processing and improve efficiency, but also to a certain extent meet the needs of screening images during the time when blurred images are easily captured, thereby achieving targeted image quality detection.

[0050] In one embodiment, detecting the image quality of each first video frame includes: detecting the clarity of each first video frame based on a reblur algorithm.

[0051] In one embodiment, the image quality is determined by factors such as image brightness, contrast, hue, boundary, noise, and clarity, wherein the higher the clarity of the image, the higher the image quality. The image quality of each first video frame may be detected according to the following steps.

[0052] Specifically, based on the secondary blur algorithm, detecting the clarity of each first video frame includes:

[0053] Based on a preset low-pass filter, each first video frame is subjected to low-pass filtering to obtain a first blurred image of each first video frame; the first blurred image is subjected to low-pass filtering to obtain a second blurred image of each first video frame; the difference in pixel values ​​of adjacent pixels in the first blurred image is determined to form a first difference matrix; the difference in pixel values ​​of adjacent pixels in the second blurred image is determined to form a second difference matrix; according to the first difference matrix and the second difference matrix, the degree of change between the first blurred image and the second blurred image is determined; and the clarity of the corresponding first video frame is determined according to the degree of change.

[0054] For example, according to the first difference matrix and the second difference matrix, the degree of change between the first blurred image and the second blurred image is determined, including: calculating the difference between the elements at corresponding positions in the first difference matrix and the second difference matrix to obtain the difference matrix; normalizing all elements of the difference matrix to obtain a normalized matrix; and taking the average value (belonging to [0,1]) of all elements in the normalized matrix as the degree of change.

[0055] In one embodiment, the definition of the first video frame obtained by the quadratic blur algorithm can be set to 1-degree of change, so that the definition can be determined more intuitively by the numerical value. The smaller the degree of change, the higher the definition and quality of the corresponding first video frame; the larger the degree of change, the lower the definition and quality of the corresponding first video frame.

[0056] In one embodiment, the aforementioned secondary blur algorithm is a reference-free image quality detection algorithm, which is suitable for performing quality detection on a first video frame without a reference image.

[0057] In addition, other non-reference or semi-reference image quality detection algorithms may be used to detect the quality of the first video frame. For example, the clarity of the first video frame may be detected based on the Brenner gradient function, and the clarity and noise of the first video frame may be detected using a No-Reference Image Quality Assessment using Blur and Noise algorithm.

[0058] In one embodiment, the first video frame is screened for image quality, and the second video frame whose image quality is greater than or equal to the image quality threshold is a video frame with higher image quality. A second video can also be generated based on the second video frame, and the first video and the second video are sent to the cloud or terminal for backup.

[0059] 104. Identify sensitive features in the second video frame, perform mask processing on the sensitive features in the second video frame to obtain a third video frame, and generate a second video based on the third video frame.

[0060] In one embodiment, the sensitive features include sensitive information of the vehicle during driving, such as faces, license plates, etc. In response to the user's operation of adding or deleting sensitive features in the App, the sensitive features that the user believes need to be shielded or covered in the subsequent process can be determined.

[0061] In one embodiment, sensitive features are determined and masked to remove sensitive information from the generated video, thereby improving the security of video dissemination for users, such as uploading the video to a social platform.

[0062] In one embodiment, the basic principle of identifying sensitive features in the second video frame includes: classifying the objects in each second video frame into regions; performing matching detection on the classified regions to determine whether the objects in the regions are sensitive features; and once the sensitive features are determined, segmenting or calibrating the regions where the sensitive features are located.

[0063] Algorithms for implementing target detection include but are not limited to: convolutional neural network algorithms, such as Faster R-CNN, regression-based target detection (You Only Look Once, YOLO), Single Shot MultiBox Detector (SSD), etc.

[0064] In one embodiment, since the sensitive features contain sensitive information such as faces, a masking operation can be performed on the sensitive features to shield the sensitive features using masks. For example, the area where the sensitive features are located can be masked using mosaics, set colors, or textures, thereby eliminating sensitive information, protecting the privacy of others, and facilitating users to spread the captured videos.

[0065] After the sensitive features in the second video frame are identified and mask processing is performed on the sensitive features in the second video frame, the second video frame may be determined as the third video frame.

[0066] In the embodiment of the present application, by determining the image quality threshold according to the amplitude range of the vehicle amplitude, the first video frame is screened according to the image quality threshold, so that the image quality screening of the video frame is targeted and effective, and it is avoided that too many continuous first video frames with different image qualities are eliminated due to a single image quality threshold, that is, the coherence of the extracted video frames is guaranteed, thereby improving the video generation quality and video generation efficiency, and reducing the video editing cost. At the same time, the sensitive features in the video frames that meet the image quality threshold are masked, which improves the security of the video frames.

[0067] In an embodiment of the present application, generating a second video based on a third video frame includes: deleting first video frames except the third video frame in the first video to obtain the second video.

[0068] By deleting the first video frame whose image quality does not meet the image quality threshold, it is ensured that the image quality of the obtained second video is relatively stable.

[0069] In one embodiment, before determining the image quality threshold according to the vehicle amplitude information in step 102, the method further includes, for example Figure 2 The following steps 201 to 203 for establishing a mapping relationship between vehicle amplitude information and image quality thresholds are shown:

[0070] 201. Obtain image quality of multiple test video frames corresponding to amplitude information of each vehicle.

[0071] In one embodiment, the test video frames may include historical video frames shot based on different vehicle amplitude information during vehicle driving, or the acquisition device acquires video frames in different vehicle amplitude information obtained by simulation. The acquired multiple test video frames are subjected to image quality detection, wherein the method for performing image quality detection on the test video frames is the same as the method in step 103, and will not be described again.

[0072] 202. For each piece of vehicle amplitude information, calculate an average value of image quality of a plurality of corresponding test video frames.

[0073] In one embodiment, the image quality of the test video frames captured when the same vehicle amplitude information is captured under different environmental conditions is different, for example, the image quality of the test video frames captured under brighter light conditions is better than that captured under darker light conditions, or the performance of different acquisition devices is different, or the shooting angles are different, which will affect the image quality of the test video frames. Therefore, to determine the image quality threshold of each vehicle amplitude information, it is necessary to collect multiple different test video frames of the same vehicle amplitude information according to different environmental conditions, so that the image quality threshold finally calculated has reference and multi-scenario applicability. Calculating the average value of the image quality of the corresponding multiple test video frames may include any of the following methods: first remove the test video frames with the smallest and largest image quality in the same vehicle amplitude information, calculate the image quality of the remaining test video frames for average calculation, and obtain the average value of the image quality of the test video frames; calculate the image quality of all test video frames of the same vehicle amplitude information for average calculation, and obtain the average value of the image quality of the test video frames; in the test video frames of the same vehicle amplitude information, the image quality with the highest frequency is used as the average value of the image quality of the test video frames; in the test video frames of the same vehicle amplitude information, the median of the image quality is used as the average value of the image quality of the test video frames. Calculating the average value of the image quality of the test video frame is not limited to the above method and is not limited here. In general, when the vehicle amplitude is larger, the average value of the image quality of the test video frame is smaller.

[0074] 203. Use the average value as the image quality threshold of the vehicle amplitude information, and establish a mapping relationship between the vehicle amplitude information and the image quality threshold.

[0075] In one embodiment, a one-to-one mapping relationship is established between the image quality threshold and the vehicle amplitude, wherein the larger the vehicle amplitude is, the smaller the image quality threshold is, that is, the image quality threshold is negatively correlated with the vehicle amplitude information.

[0076] For example, when the vehicle amplitude information is 6 centimeters, the corresponding image quality threshold may be 0.8; when the vehicle amplitude information is 9 centimeters, the corresponding image quality threshold may be 0.6.

[0077] In one embodiment, the mapping relationship may be saved in a database in a table or other form. When determining the image quality threshold according to the vehicle amplitude information in step 102 , the vehicle amplitude information is directly searched in the database to determine the corresponding image quality threshold.

[0078] In one embodiment, obtaining a first video captured during vehicle driving includes: Figure 3 The following steps 1011 to 1013 for obtaining the first video are shown:

[0079] 1011. Obtain shooting request information, where the shooting request information includes acquisition device information and acquisition duration.

[0080] In one embodiment, the shooting request information includes but is not limited to: acquisition device information, acquisition duration, sensitive features, and video backup request. Figure 4 The figure shown is an example diagram of the shooting request interface provided in an embodiment of the present application.

[0081] In one embodiment, the acquisition device information includes the model of the acquisition device (e.g. Figure 4 The brand model shown); the collection device information may also include a number pre-set for each collection device, for example, the number is named in the form of numbers, letters, or a combination of numbers and letters.

[0082] In addition, the acquisition device information may also include: the location information of the acquisition device in the vehicle, the battery power and remaining memory capacity of the acquisition device, the frame rate and resolution of the video captured by the acquisition device (for example, Figure 4 1920×1080) etc., to facilitate user selection.

[0083] For example, the frame rate of the information collected by the device A is 60 frames per second (FPS, Frames per Second) (e.g. Figure 4 60FPS), indicating that the first video captured by the acquisition device A includes 60 first videos per second.

[0084] In one embodiment, the capture duration includes the duration of the first video that the user wants to capture. Figure 4 As shown), the time when the target acquisition device is started is used as the time to start shooting the first video. After the shooting time, the target acquisition device is automatically turned off to obtain the first video.

[0085] In one embodiment, the video backup request includes whether to back up the captured video, which can generally be backed up by default. Whether to back up the video is determined by responding to the user's selection operation on whether to back up the video in the App. Backing up the video can facilitate users to retrieve the video and avoid video loss. At the same time, an automatic deletion function can also be set. The user can set a preset time period to delete the backup videos that exceed the time period. For example, when the backup videos that are more than one month old are set to be automatically deleted, memory usage can be reduced.

[0086] In addition, the video can be sent to a preset cloud or terminal for backup. For example, the cloud can be the cloud bound to the user account in the App, the terminal can be the terminal where the App is located, etc. The video can be transmitted and sent via communication connections such as Wi-Fi or Bluetooth.

[0087] 1012. Start a collection device in the vehicle corresponding to the collection device information.

[0088] In one embodiment, in response to the user's selection operation on the collection device information in the App, the collection device corresponding to the collection device information can be used as the target collection device and the target collection device can be started.

[0089] 1013. Obtain, through a collection device, a first video of a collection duration.

[0090] In one embodiment, in response to the user's operation in the App, the target acquisition device is turned on to obtain the first video, and the target acquisition device is automatically turned off according to the acquisition time set by the user. The first video can be automatically shot, thereby improving the efficiency of video shooting and saving the user's shooting time.

[0091] In one embodiment, after the third video frame generates the second video in step 104, the video generation method further includes, for example Figure 5 The following steps 105 to 106 in the flowchart of obtaining the third video are shown:

[0092] 105. Obtain editing request information, where the editing request information includes video processing content defined by a user.

[0093] In one embodiment, the updated video can be sent to the App for the user to view, and the user's editing request information in the App can be obtained. The user-defined video processing content includes one or more of selecting background music, selecting a mask template to cover sensitive features, setting video duration, and adding filters.

[0094] 106. Edit the second video according to the video processing content to obtain a third video.

[0095] In one embodiment, a variety of background music may be provided in the App for the user to choose from (e.g. Figure 6 As shown in the figure, the background music selected by the user is added to the second video; different mask templates can also be provided for the user to select different mask images to mask different sensitive features in the second video frame, thereby improving the aesthetics of the generated third video. For example, in response to the user's selection operation, a pink heart-shaped template can be used to mask a face, etc. (for example Figure 6 shown).

[0096] In one embodiment, the playback speed of the second video frame may also be adjusted according to the video duration set by the user, so that the duration of the generated third video is equal to the video duration set by the user.

[0097] Specifically, the time interval between every two second video frames can be reset according to the chronological order, so as to adjust the playback speed of the second video frames to a predetermined speed, so that the duration of the third video is equal to the video duration set by the user, and is smoother and more coherent. The predetermined speed can be a frame rate calculated based on the video duration set by the user and the number of second video frames, for example, 30 frames per second.

[0098] In one embodiment, the duration of the updated video corresponding to different frame rates can also be displayed to the user in the App so that the user can select the required duration, and the frame rate corresponding to the duration selected by the user is used as the predetermined speed.

[0099] In one embodiment, in step 103, the first video frame whose image quality is greater than or equal to the image quality threshold is determined as the second video frame, and further includes, for example Figure 7 Steps 1031 to 1034 in the flowchart of similarity detection shown are:

[0100] 1031. Use a first video frame whose image quality is greater than or equal to an image quality threshold as a fourth video frame, and perform similarity detection on adjacent fourth video frames to obtain multiple similarity values.

[0101] Since there are multiple first video frames captured within 1 second, and the surrounding environment changes during vehicle driving are usually not significant, that is, the continuously captured video frames may have repeated scenes. In order to enable the generated video to play the changing scenes in the entire acquisition process in a short time, especially for short video types, some repeated first video frames can be deleted. Generally speaking, the similarity between two adjacent first video frames whose image quality is greater than or equal to the image quality threshold (hereinafter referred to as the first video frame) is the highest. Therefore, the first video frame whose image quality is greater than or equal to the image quality threshold is used as the fourth video frame, and the similarity detection is performed between adjacent fourth video frames, so as to obtain a more comprehensive similarity detection result.

[0102] The adjacent fourth video frames may be two adjacent fourth video frames; or, may be more than two adjacent fourth video frames, such as three adjacent fourth video frames, four adjacent fourth video frames, etc.

[0103] In one embodiment, the number of adjacent fourth video frames (for example, an integer greater than or equal to 2) can be preset. For example, three adjacent fourth video frames can be used as a cycle, and similarity detection can be performed between three adjacent fourth video frames in each cycle.

[0104] The number of the fourth video frames may also be determined according to the appearance time of the fourth video frame in the first video. For example, with 0.5 seconds as a cycle, similarity detection is performed on all the fourth video frames in each cycle in the first video.

[0105] In one embodiment, the method for performing similarity detection may be a perceptual hash algorithm (PHA), and the perceptual hash algorithm includes any one or more combinations of algorithms such as a DHash (difference hash) algorithm.

[0106] Exemplarily, the multiple first video frames may be reduced to the same ratio to obtain multiple thumbnails having the same number of pixels (e.g., 9×8) and image size (e.g., 8 rows of pixels, 9 pixels in each row);

[0107] Based on the average value method in the grayscale algorithm, multiple thumbnails are converted into grayscale images;

[0108] Starting from the first row of pixels in each grayscale image, the difference value of the color intensity (grayscale value) between the previous pixel and the next pixel in each row of pixels is determined, including: if the color intensity of the previous pixel is greater than the color intensity of the next pixel, the difference value is 1; if the color intensity of the previous pixel is not greater than the color intensity of the next pixel, the difference value is 0;

[0109] Treat each row difference value of each grayscale image as a string (for example, an 8-bit string that only includes binary values ​​0 and 1), arrange all the strings of each grayscale image in row order (for example, from row 1 to row 8), and obtain the DHash value corresponding to each grayscale image (for example, 8×8=64 bits);

[0110] The Hamming distances between the multiple DHash values ​​corresponding to the multiple first video frames are calculated, and the reciprocal of each Hamming distance is used as the similarity value between the corresponding first video frames.

[0111] In one embodiment, the DHash algorithm is a similarity detection algorithm based on gradient, which can achieve better similarity detection results in a shorter time, and can improve the efficiency and accuracy of similarity detection. The larger the value of the Hamming distance obtained by the DHash algorithm, the smaller the similarity value, and the smaller the similarity between the corresponding second video frames.

[0112] 1032. Determine whether the similarity value is greater than a preset similarity threshold. If the similarity value is less than or equal to the preset similarity threshold, execute 1033. If the similarity detection result is greater than the preset similarity threshold, execute 1034.

[0113] In one embodiment, the similarity threshold can be set according to user needs, for example, 60 / 64. When the similarity value is greater than the similarity threshold, the fourth video frame with the highest image quality can be selected from multiple adjacent fourth video frames as the second video frame.

[0114] 1033. Determine a fourth video frame whose similarity is less than a preset similarity threshold as the second video frame.

[0115] 1034. From the fourth video frames whose similarity values ​​are greater than or equal to a preset similarity threshold, extract the fourth video frame with the highest image quality as the second video frame.

[0116] In one embodiment, the fourth video frame is screened for similarity, thereby eliminating duplicate video frames in the fourth video frame, which can achieve the effect of streamlining and optimizing the video. A backup video can also be generated based on the fourth video frame, and the backup video can be sent to the terminal for backup according to the video backup request.

[0117] In one embodiment, after obtaining the second video in step 104, the method further includes, for example Figure 8 The following steps 107 to 109 in the flowchart of obtaining the fourth video are shown:

[0118] 107. Perform image recognition on each third video frame in the second video to obtain environmental information corresponding to the second video.

[0119] In one embodiment, the method used for performing image recognition on the third video frame is similar to the method for identifying sensitive features in step 104, and will not be described again. The environmental information includes the category of the actual environment, such as desert, grassland, city, country, etc. For example, when the result of image recognition indicates that the second video frame contains low houses and farmland, the environmental information corresponding to the second video corresponds to country.

[0120] 108. Obtain background music corresponding to the environmental information.

[0121] In one embodiment, the corresponding background music can be retrieved and obtained according to the environmental information. For example, the country music indicated by the environmental information can be searched and downloaded on the Internet. The duration of the background music is correspondingly cut so that the duration of the background music is the same as the duration of the second video.

[0122] 109. Add background music to the second video to obtain a fourth video.

[0123] In one embodiment, the environmental information of the video is automatically identified and background music corresponding to the environmental information is added to improve the viewing quality of the video.

[0124] In addition, the record in step 106 can be combined to search and download multiple background music corresponding to the environmental information for the user to choose.

[0125] In one embodiment, in addition to performing image quality detection in real time according to the amplitude of the vehicle and performing image quality detection at preset time intervals in the above-mentioned embodiment, image quality detection can also be performed after the first video is captured. Therefore, the video generation method also includes the following steps: Fig. 9 The steps of determining the vehicle amplitude information corresponding to the target time node shown in the flowchart, according to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted, as shown below.

[0126] 401. Determine, based on a sensor device in a vehicle, a time when a first video is shot and a time when a vibration state occurs.

[0127] In one embodiment, in addition to a vibration sensor for detecting the vibration state of the vehicle, the vehicle-mounted device also includes a time sensor (e.g., a timer), which can be used to determine the time to start shooting the first video (or the time to start the acquisition device corresponding to the shooting request, the time to start acquiring the first video); the time when the vibration state occurs can be determined based on the time sensor and the vibration sensor.

[0128] 402. The time when the first video starts to be shot is taken as an initial time node.

[0129] In the time axis starting from the initial time node, each first video frame has a corresponding time label.

[0130] 403. According to the initial time node, the time when the vibration state occurs is used as the target time node.

[0131] In one embodiment, according to the initial time node, taking the time when the vibration state occurs as the target time node includes: in the time axis starting from the initial time node, determining the time when the vibration state occurs after the initial time node as the target time node.

[0132] Since the vibration state may occur multiple times, the target time node may include multiple time nodes, and the time length occupied by each target time node may be the same, for example, 0.25 seconds, and the first video frame included in each target time node may be determined according to the time tag of the first video frame. When 60 first video frames are included per second, each target time node may correspond to 60 frames / second×0.25 seconds=15 first video frames.

[0133] 404. Determine vehicle amplitude information corresponding to each target time node.

[0134] In one embodiment, the amplitude at the target time node is selected from amplitude data recorded by a vibration sensor.

[0135] In one embodiment, after determining the amplitude of the vehicle corresponding to the target time node, similar to steps 102 to 103, image quality detection is performed on the first video frame according to the amplitude of the vehicle, and the first video frame greater than or equal to the image quality threshold is used as the second video frame.

[0136] The beneficial effects of the video generation method provided in the embodiment of the present application include: automatically starting the acquisition device to shoot the first video of the acquisition time required by the user, saving the user's shooting time; setting multiple image quality thresholds according to the amplitude range of the vehicle amplitude to avoid eliminating too many continuous video frames of low quality when the vehicle is in a vibrating state, thereby ensuring the continuity of the video; eliminating repeated video frames with high similarity to streamline the video; masking sensitive features in the video frames to protect the privacy of others and improve the security of the video; providing users with editing and operation interfaces to meet user needs; and backing up videos in a timely manner to prevent video loss and facilitate user retrieval.

[0137] Above Figure 1 The video generation method of this application is described in detail. Fig.10 , the architecture of the hardware device for implementing the video generation method is introduced.

[0138] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0139] See also Fig.10 , which is a schematic diagram of the structure of the vehicle-mounted device provided in a preferred embodiment of the present application.

[0140] In a preferred embodiment of the present application, the vehicle-mounted device 3 includes a video generation device 30, a memory 31, at least one processor 32, at least one acquisition device 33, and at least one sensor device 34. Those skilled in the art should understand that Fig.10 The structure of the vehicle-mounted device shown does not constitute a limitation of the embodiments of the present application, and can be either a bus structure or a star structure. The vehicle-mounted device 3 can also include more or less other hardware or software than shown in the figure, or a different component arrangement.

[0141] In some embodiments, the vehicle-mounted device 3 includes a terminal that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, a dedicated integrated circuit, a programmable gate array, a digital processor, and an embedded device.

[0142] It should be noted that the vehicle-mounted device 3 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the protection scope of the present application and included here by reference.

[0143] In some embodiments, the memory 31 is used to store program codes and various data. For example, the memory 31 can be used to store video data 35 (for example, the first video, the second video frame, etc. mentioned above), and realize high-speed and automatic access to programs or data during the operation of the vehicle-mounted device 3. The memory 31 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable storage medium that can be used to carry or store data.

[0144] In some embodiments, at least one processor 32 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. At least one processor 32 is the control core (ControlUnit) of the vehicle-mounted device 3, and uses various interfaces and lines to connect various components of the entire vehicle-mounted device 3, and executes various functions and processes data of the vehicle-mounted device 3 by running or executing programs or modules stored in the video generation device 30 and the memory 31, and calling data stored in the memory 31, such as executing Figure 1 The video shown generates the functionality.

[0145] In some embodiments, the video generating device 30 runs in the vehicle-mounted device 3. The video generating device 30 may include a plurality of functional modules composed of program code segments. The program code of each program segment in the video generating device 30 may be executed by at least one processor 32 to implement Figure 1 The video shown generates the functionality.

[0146] In this embodiment, the video generating device 30 can be divided into multiple functional modules according to the functions it performs. Fig.11 As shown, the device includes an acquisition module 301, a determination module 302, a detection module 303, and an identification module 304. The module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions.

[0147] Specifically, the acquisition module 301 is used to acquire a first video shot during vehicle driving and the vehicle amplitude information when the first video is shot; the first video includes multiple first video frames; the determination module 302 is used to determine the image quality threshold according to the vehicle amplitude information; the detection module 303 is used to detect the image quality of each of the first video frames, and determine the first video frame whose image quality is greater than or equal to the image quality threshold as the second video frame; the identification module 304 is used to identify sensitive features in the second video frame, and mask the sensitive features in the second video frame to obtain a third video frame, and generate a second video based on the third video frame.

[0148] Although not shown, the vehicle-mounted device 3 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to at least one processor 32 through a power management device, so that the power management device can manage charging, discharging, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure test circuits, power converters or inverters, power status indicators, and other arbitrary components. The vehicle-mounted device 3 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0149] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0150] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a vehicle-mounted device (which can be a server, a personal computer, etc.) or a processor to execute parts of the methods of various embodiments of the present application.

[0151] The memory 31 stores program codes, and at least one processor 32 can call the program codes stored in the memory 31 to execute related functions. The program codes stored in the memory 31 can be executed by at least one processor 32, thereby realizing the functions of each module to achieve the purpose of video generation.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

[0153] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0155] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is limited by the attached claims rather than the above description, so it is intended to include all changes that fall within the meaning and scope of the equivalent elements of the claims in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the above preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A video generation method, characterized in that: The method comprises: Acquire a first video shot during vehicle driving and vehicle amplitude information when the first video is shot; the first video includes a plurality of first video frames; Determining the image quality threshold according to the vehicle amplitude information includes: determining the image quality threshold corresponding to the vehicle amplitude information according to a one-to-one mapping relationship between the vehicle amplitude information and the image quality threshold; detecting an image quality of each of the first video frames, and determining a first video frame whose image quality is greater than or equal to the image quality threshold as a second video frame; Identify sensitive features in the second video frame, perform masking on the sensitive features in the second video frame to obtain a third video frame, and generate a second video based on the third video frame.

2. The video generation method according to claim 1, characterized in that: Before determining the image quality threshold according to the vehicle amplitude information, the method further includes: Obtaining image quality of multiple test video frames corresponding to amplitude information of each vehicle; For each piece of vehicle amplitude information, calculating an average value of the image quality of the corresponding plurality of test video frames; The average value is used as the image quality threshold of the vehicle amplitude information, and a mapping relationship between the vehicle amplitude information and the image quality threshold is established.

3. The video generation method according to claim 1, characterized in that: The step of obtaining a first video shot during the driving of the vehicle includes: Acquire shooting request information, wherein the shooting request information includes acquisition device information and acquisition duration; Starting a collection device in the vehicle corresponding to the collection device information; The first video of the acquisition duration is acquired through the acquisition device.

4. The video generation method according to any one of claims 1 to 3, characterized in that: After generating the second video based on the third video frame, the method further includes: Acquire editing request information, wherein the editing request information includes video processing content defined by a user; The second video is edited according to the video processing content to obtain a third video.

5. The video generation method according to claim 3, characterized in that: The generating the second video based on the third video frame comprises: The first video frames except the third video frame in the first video are deleted to obtain the second video.

6. The video generation method according to claim 1, characterized in that: The step of determining the first video frame whose image quality is greater than or equal to the image quality threshold as the second video frame further includes: taking a first video frame whose image quality is greater than or equal to the image quality threshold as a fourth video frame, and performing similarity detection on adjacent fourth video frames to obtain multiple similarity values; Extracting the fourth video frame with the highest image quality from a plurality of consecutive fourth video frames whose similarity values ​​are greater than or equal to a preset similarity threshold as the second video frame; A fourth video frame whose similarity value is less than the preset similarity threshold is determined as the second video frame.

7. The video generation method according to claim 5, characterized in that: After obtaining the second video, the method further includes: Performing image recognition on each of the third video frames in the second video to obtain environmental information corresponding to the second video; Acquire background music corresponding to the environmental information; The background music is added to the second video to obtain a fourth video.

8. A video generating device, characterized in that: The video generating device comprises: An acquisition module, used to acquire a first video shot during vehicle driving and vehicle amplitude information when the first video is shot; the first video includes a plurality of first video frames; a determination module, configured to determine an image quality threshold according to the vehicle amplitude information, and specifically to determine the image quality threshold corresponding to the vehicle amplitude information according to a one-to-one mapping relationship between the vehicle amplitude information and the image quality threshold; A detection module, configured to detect the image quality of each of the first video frames, and determine the first video frame whose image quality is greater than or equal to the image quality threshold as the second video frame; An identification module is used to identify sensitive features in the second video frame, mask the sensitive features in the second video frame to obtain a third video frame, and generate a second video based on the third video frame.

9. A vehicle-mounted device, characterized in that: The vehicle-mounted device includes a memory and at least one processor, wherein the memory stores at least one instruction, and when the at least one instruction is executed by the at least one processor, the video generation method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the video generating method according to any one of claims 1 to 7 is implemented.

Images