Automobile data recorder video processing method and system

Abstract

The invention discloses an automobile data recorder video processing method and system, and the method comprises the steps: firstly obtaining original video data collected by a vehicle-mounted camera in a first time period, and extracting a Mel spectrum of the original video data in the first time period; and performing fine-grained spectral analysis on the Mel spectrum of the original video data by adopting a voice recognition technology based on artificial intelligence so as to judge whether the voice data in the first time period triggers a specific scene or not, and determining whether to store the original high-definition video data or to compress the video data according to a judgment result. Thus, through intelligent identification and management of the video data of the automobile data recorder, it can be ensured that the automobile data recorder can completely store key information when encountering a specific scene, and therefore the data storage efficiency of the automobile data recorder is further improved.

Description

Technical Field

[0001] The present application relates to the field of video processing, and more specifically, to a driving recorder video processing method and system. Background Art

[0002] As an important part of modern automobile safety systems, dashcams are widely used in traffic accident evidence collection, driving behavior monitoring, and vehicle anti-theft. Since dashcams need to record videos continuously, they will generate massive amounts of data over a long period of time. Most dashcams rely on built-in or external storage cards (such as SD cards) to save recorded video data. The capacity of these storage media is usually fixed, and the capacity will quickly reach the upper limit as the recording time increases, causing the early recorded content to be overwritten by new data, resulting in missing data in critical time periods. To address the above issues, the dashcam video data can be compressed to reduce the amount of memory occupied, thereby ensuring the integrity of the data in the dashcam.

[0003] Existing video data compression methods usually reduce file size by cutting the number of video frames or reducing the overall picture resolution, but this may result in the deletion of key frames at the moment of the accident, or the road information and license plate number at the time of the accident cannot be clearly identified due to low resolution. And as driving time increases, even with conventional compression methods, high-definition video over a long period of time will still take up a lot of storage space, forcing the device to continuously overwrite old data, resulting in users being unable to obtain records of earlier events.

[0004] Therefore, in order to overcome the risk of key information loss and storage space limitation of traditional driving recorders, a more efficient driving recorder video processing method is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a driving recorder video processing method and system, firstly obtaining the original video data within a first time period collected by a vehicle-mounted camera, and extracting the mel-spectrogram of the original video data within the first time period, and then using artificial intelligence-based speech recognition technology to perform fine-grained spectrum analysis on the mel-spectrogram of the original video data, so as to determine whether the speech data within the first time period triggers a specific scene, and determine whether to save the original high-definition video data or compressed video data based on the judgment result. In this way, by intelligently identifying and managing the video data of the driving recorder, it can be ensured that the driving recorder can completely save key information when encountering specific scenes, thereby further improving the data storage efficiency of the driving recorder.

[0006] According to one aspect of the present application, a driving recorder video processing method is provided, which includes:

[0007] Acquire raw video data collected by the vehicle-mounted camera within a first time period;

[0008] Performing video compression processing on the original video data to obtain compressed video data;

[0009] If the voice data in the first time period triggers a specific scene, saving the original video data;

[0010] If the voice data within the first time period does not trigger a specific scene, the compressed video data is saved, wherein if the voice data within the first time period triggers a specific scene, the original video data is saved, including: extracting the Mel frequency spectrum features of the voice data within the first time period to obtain a scene sound Mel frequency spectrum graph; performing block segmentation and embedding coding on the scene sound Mel frequency spectrum graph to obtain a sequence of scene sound Mel frequency spectrum block embedding coding vectors; performing contextual semantic association coding on the sequence of scene sound Mel frequency spectrum block embedding coding vectors to obtain a sequence of scene sound Mel frequency spectrum block contextual semantic association coding features; performing sequence conditional distinguishable enhancement on the sequence of scene sound Mel frequency spectrum block contextual semantic association coding features to obtain a sequence of scene sound Mel frequency spectrum block contextual semantic association distinguishable enhancement coding features; based on the sequence of scene sound Mel frequency spectrum block contextual semantic association distinguishable enhancement coding features, determine that the voice data within the first time period triggers a specific scene.

[0011] According to another aspect of the present application, a driving recorder video processing system is provided, comprising:

[0012] A mel spectrum feature extraction module, used to extract mel spectrum features of speech data in a first time period to obtain a scene sound mel spectrum graph;

[0013] A scene sound Mel-spectrogram block embedding coding module, used for performing block segmentation and embedding coding on the scene sound Mel-spectrogram to obtain a sequence of scene sound Mel-spectrogram block embedding coding vectors;

[0014] A context semantic association coding module, used for performing context semantic association coding on the sequence of scene sound Mel-spectrogram blocks embedded in the coding vector to obtain a sequence of context semantic association coding features of the scene sound Mel-spectrogram blocks;

[0015] A distinguishable enhancement module, used for performing sequence conditional distinguishable enhancement on the sequence of the scene sound Mel-spectrogram block contextual semantic association coding features to obtain a sequence of scene sound Mel-spectrogram block contextual semantic association distinguishable enhancement coding features;

[0016] A specific scene trigger judgment module is used to determine whether the speech data in the first time period triggers a specific scene based on a sequence of distinguishable enhanced coding features based on the contextual semantic association of the scene sound mel-spectrogram block.

[0017] Compared with the prior art, the present application provides a method and system for processing dashcam video. First, the original video data collected by the vehicle camera in the first time period is obtained, and the mel-spectrogram of the original video data in the first time period is extracted. Then, the mel-spectrogram of the original video data is subjected to fine-grained spectrum analysis using speech recognition technology based on artificial intelligence, so as to determine whether the speech data in the first time period triggers a specific scene, and to determine whether to save the original high-definition video data or compressed video data based on the judgment result. In this way, by intelligently identifying and managing the video data of the dashcam, it can be ensured that the dashcam can completely save key information when encountering a specific scene, thereby further improving the data storage efficiency of the dashcam. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 is a flowchart of a driving recorder video processing method according to an embodiment of the present application;

[0020] Figure 2 A schematic diagram of data flow of a driving recorder video processing method according to an embodiment of the present application;

[0021] Figure 3 Flow chart of sub-step S3 of the driving recorder video processing method according to an embodiment of the present application;

[0022] Figure 4 is a block diagram of a driving recorder video processing system according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0024] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0025] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0026] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0027] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0028] In the technical solution of the present application, a driving recorder video processing method is proposed. Figure 1 The present invention is a flowchart of a method for processing driving recorder video according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the driving recorder video processing method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the dash cam video processing method according to the embodiment of the present application includes the steps of: S1, acquiring original video data within a first time period collected by a vehicle-mounted camera; S2, performing video compression processing on the original video data to obtain compressed video data; S3, if the voice data within the first time period triggers a specific scene, saving the original video data; S4, if the voice data within the first time period does not trigger a specific scene, saving the compressed video data.

[0029] In particular, the S1 obtains the raw video data within the first time period collected by the vehicle-mounted camera. In a specific example, the driving recorder needs to be equipped with at least one high-quality vehicle-mounted camera, which is the sensory organ of the entire system. The choice of camera is crucial. It should have sufficient resolution (for example, 1080p or higher) and excellent light adaptability to ensure that clear images can be captured under different lighting conditions. In addition, the location of the camera is also extremely important. It is usually fixed on the front windshield of the vehicle to ensure that the field of view in front of the vehicle can be covered. At the same time, it is also necessary to consider whether it is necessary to install additional cameras for monitoring the rear or sides. The camera should be installed in a stable position to avoid blurring or distortion of the picture due to vibration during driving, which affects the subsequent video analysis effect.

[0030] One of the core functions of a dash cam is to continuously record video, which means that the camera is always in operation and continuously generates a video stream. To achieve this, the dash cam is equipped with a processor or chipset dedicated to video capture. This component is responsible for receiving image signals from the camera in real time and converting them into a sequence of video frames in digital format. These video frames are stored continuously in a temporary buffer in chronological order, forming the so-called "rolling recording". The rolling recording mechanism allows the dash cam to keep the latest video records within a limited storage space. When new video data comes in, the oldest data will be automatically overwritten, but the premise is that these data have been confirmed by the system as non-critical content.

[0031] In order to determine a specific time period as the object of subsequent processing, the dashcam system needs to define the so-called "first time period". This is usually achieved by setting a fixed time window, such as every 5 minutes or 10 minutes as a time period unit. When the dashcam detects that the vehicle starts to move, it will start a new time period recording; and when the vehicle stops for more than a certain time or reaches a preset time length, the current time period will be closed and the next new time period will be prepared. This mechanism ensures that the video data in each time period is independent and complete, which is convenient for subsequent management and retrieval.

[0032] For each defined first time period, the dash cam must accurately extract the corresponding raw video data from the rolling video. This is done by recording the timestamp of each video frame. Each video frame is accurately time-stamped when it is captured, so that even if the video is recorded in a continuous manner, the video clips within the specified time period can be accurately captured based on the start time and end time. This part of the raw video data includes complete uncompressed image information, retaining all details without any form of compression or modification. High-fidelity raw video data is the prerequisite for subsequent intelligent analysis and management, especially when encountering specific scenarios, such as traffic accidents or other important events, the system can identify and completely save high-quality raw video data to ensure that no potentially important details are lost.

[0033] After determining the first time period to be processed, the dashcam transfers the raw video data within that time period from the temporary buffer to a more permanent storage medium, such as an SD card or other type of flash memory device. During this process, some preliminary organization and sorting of the video data may be performed, such as adding metadata (such as GPS location information, speed, etc.) to facilitate subsequent retrieval and analysis. The addition of metadata not only enhances the information richness of the video data, but also facilitates future queries. Users can easily access historical records through map positioning, timeline browsing, etc.

[0034] It is worth noting that in addition to video data, the dashcam can also collect audio information at the same time. In this case, the original video data also includes a sound track recorded synchronously with the video. The audio data also carries a timestamp, which corresponds one-to-one with the video frame to ensure the temporal consistency between the two. This part of the audio information is particularly important for subsequent scene recognition, because certain specific events (such as sudden braking, collision sounds) are often accompanied by unique audio features, which can be used as trigger conditions to decide whether to save the high-definition version of the video data. By extracting the Mel spectrum features of the audio signal, various environmental sounds can be effectively captured, such as normal traffic noise, conversations inside the vehicle, sudden braking sounds or collision sounds, etc., to help the system accurately distinguish between daily driving and ordinary emergencies. In practical applications, through fine-grained spectrum analysis, it is possible to determine whether the voice data in the first time period triggers a specific scene, and determine whether to save the original high-definition video data or compressed video data based on the judgment results.

[0035] In particular, S2 performs video compression processing on the original video data to obtain compressed video data. The core of video compression is to remove redundant information, that is, to identify and eliminate unnecessary or repeated data parts in the video through an algorithm, so as to achieve the purpose of reducing the file size. Common compression methods include two types: inter-frame compression and intra-frame compression. Inter-frame compression uses the similarity between adjacent frames to reduce the amount of data, while intra-frame compression focuses on optimizing the information inside a single image frame. Driving recorders usually adopt a hybrid coding strategy, combining the advantages of these two methods to achieve higher compression efficiency. For example, under normal driving conditions, the changes between consecutive frames are often small, so the amount of data can be greatly reduced by saving only the changed parts instead of the entire picture; for a single frame, the system will try to find the best prediction model to estimate the current pixel value, and calculate the residual (the difference between the actual value and the predicted value), and then convert these residuals to the frequency domain, so that the high-frequency components are more easily discarded without affecting the overall perception.

[0036] Before compression begins, the system performs a series of preprocessing operations on the raw video data. This step ensures that all information is complete and accurate, providing a good foundation for subsequent compression. First, the dash cam checks the video's timestamps and metadata to ensure that there are no omissions or errors. Then, the color format of the video is adjusted as needed, such as converting from RGB to YUV, because some compression standards are more suitable for specific color representations. In addition, the preprocessing stage may also involve cropping or scaling the video resolution to better suit the requirements of the target output format. This step is particularly important for dash cams, because it needs to ensure that even after compression, the video can still clearly display key details such as road information and license plate numbers.

[0037] Choosing the right compression standard is the first step to implement efficient compression. Currently, H.264 / AVC (Advanced Video Coding) and H.265 / HEVC (High Efficiency Video Coding) are the two most commonly used compression standards. H.264 provides a good compression ratio and compatibility, suitable for most dashcam devices; while H.265 further improves the compression efficiency, which can reduce the file size by about 50% at the same visual quality, which is very suitable for those who want to maximize storage capacity. In addition, according to different needs, dashcams may also support other compression standards, such as MPEG-4 or VP9. Each compression standard has its own characteristics and scope of application. Choosing the standard that best suits the dashcam application scenario is the key to ensuring the compression effect.

[0038] After selecting the appropriate compression standard, the specific compression algorithm is applied. During this process, the dashcam's processor performs a series of complex and sophisticated operations. For each individual image frame, the system tries to find the best prediction model to estimate the current pixel value and calculates the residual (the difference between the actual value and the predicted value). These residuals are then fed into a mathematical transformation function (such as discrete cosine transform DCT) to convert them to the frequency domain, making it easier for high-frequency components (usually noise or less detailed parts) to be discarded without affecting the overall perception. The transformed coefficients are then quantized, that is, they are mapped to integer values ​​in a smaller range. This process actually reduces the accuracy, but also greatly reduces the amount of data. The choice of quantization parameters is very critical because it directly determines the balance between compression rate and the quality of the reconstructed image. The quantized data is further compressed by entropy coding (such as Huffman coding or arithmetic coding). Entropy coding is a lossless compression technique that uses statistical probabilities to represent symbols, with more frequently occurring symbols represented by shorter codes and rare symbols corresponding to longer codes. The result is a significant reduction in the length of the bitstream without losing any information. In addition to the above operations on a single frame, the dashcam also uses inter-frame prediction to improve compression efficiency. Specifically, it analyzes the difference between the previous and next frames and only saves the changed parts instead of the entire picture. This method is particularly suitable for dashcam scenarios because under normal driving conditions, the changes between consecutive frames are often small, so the amount of data can be greatly reduced in this way.

[0039] After compression is completed, the dash cam will store the generated compressed video data on a persistent medium, such as an SD card or other type of flash memory device. Prior to this, the system may add additional metadata, such as compression ratio, codec version used, etc., to facilitate subsequent reading and decoding. In addition, in order to ensure the consistency and smoothness of video playback, the dash cam also needs to ensure that the compressed video conforms to the established standard format so that users can view it without obstacles on various players or platforms. This approach not only enhances the information richness of the video data, but also facilitates future queries. Users can easily access historical records through map positioning, timeline browsing, etc.

[0040] In particular, in S3, if the voice data in the first time period triggers a specific scene, the original video data is saved. Figure 3As shown, the S3 includes: S31, extracting the Mel-frequency spectrum features of the speech data within the first time period to obtain a scene sound Mel-frequency spectrogram; S32, performing block segmentation and embedding coding on the scene sound Mel-frequency spectrogram to obtain a sequence of scene sound Mel-frequency spectrogram block embedding coding vectors; S33, performing contextual semantic association coding on the sequence of scene sound Mel-frequency spectrogram block embedding coding vectors to obtain a sequence of scene sound Mel-frequency spectrogram block contextual semantic association coding features; S34, performing sequence conditional distinguishable enhancement on the sequence of scene sound Mel-frequency spectrogram block contextual semantic association coding features to obtain a sequence of scene sound Mel-frequency spectrogram block contextual semantic association distinguishable enhancement coding features; S35, determining that the speech data within the first time period triggers a specific scene based on the sequence of scene sound Mel-frequency spectrogram block contextual semantic association distinguishable enhancement coding features.

[0041] Specifically, the S31 extracts the Mel-frequency spectrum features of the speech data in the first time period to obtain the scene sound Mel-spectrogram. It should be understood that Mel-frequency cepstral coefficients (MFCCs) are a set of characteristic parameters widely used in the field of audio signal processing and speech recognition. By using a Mel filter bank, MFCCs can effectively capture important energy distribution information in the audio signal, especially in the low frequency band, which is very important for the characterization of speech and other natural sounds. In the technical solution of the present application, by extracting the Mel-frequency spectrum features of the speech data in the first time period, various environmental sounds can be effectively identified, such as normal traffic noise, vehicle internal conversation, emergency brake sound or collision sound, etc., to help the system accurately distinguish between daily driving and ordinary emergencies.

[0042] Specifically, the S32, the scene sound Mel spectrogram is segmented and embedded to obtain a sequence of scene sound Mel spectrogram block embedding coding vectors. That is, in a specific example of the present application, first, the scene sound Mel spectrogram is segmented to obtain a sequence of scene sound Mel spectrogram blocks. Since there are rich change patterns in the time domain and frequency domain of the audio signal, and these patterns are often reflected in a shorter period of time, the entire spectrogram can be decomposed into a plurality of smaller areas by segmenting the Mel spectrogram into a sequence of blocks, each block representing a short period of time. The spectrum information enables the system to capture the local features of the scene audio in more detail, such as short-term sound events or changes in specific frequency components. In addition, compared to the entire spectrogram, block segmentation allows the model to focus on finer-grained time-frequency characteristics, thereby providing a richer description of scene details. For example, when identifying the sound of sudden braking or collision, this refined feature capture helps to improve recognition accuracy. Next, it should be understood that the audio environment faced by the dashcam is often complex and changeable, including but not limited to different background noises under various road conditions such as urban roads, highways, and rural roads. These sound events often do not exist in isolation, but are closely related to other sounds in the previous and subsequent time periods. By embedding the Mel-spectrogram blocks, it can be ensured that the audio features of the time period represented by each block are accurately preserved. At the same time, in the embedding coding process, the semantic correlation between different blocks can be captured to form a coherent overall description to obtain a sequence of embedded coding vectors of the scene sound Mel-spectrogram blocks. This temporal continuity and contextual awareness are crucial to understanding complex traffic environments. It allows the system to focus not only on individual sound events, but also understand how these events evolve over time.

[0043] Specifically, the S33, the sequence of the scene sound Mel-spectrogram block embedding coding vector is contextually semantically associated to obtain a sequence of scene sound Mel-spectrogram block contextual semantically associated coding features. In a specific example of the present application, the sequence of the scene sound Mel-spectrogram block embedding coding vector is input into the audio spectrogram transformer network to obtain a sequence of the scene sound Mel-spectrogram block contextual semantically associated coding vector. It should be understood that the dash cam often needs to process sound or video events spanning a long period of time, which contains rich time series information. For example, in an urban traffic environment, continuous traffic light changes, vehicle queues and pedestrians crossing the road may be encountered. These events usually have dynamic changes in timing, and there may be complex interactions between each other. Although traditional convolutional neural networks or recurrent neural networks can effectively extract local features, they have limitations in capturing long-distance dependencies. Audio Spectrogram Transformer (Spectrogram Transformer, referred to as AST) is a deep learning model specially designed for processing audio data. The audio spectrogram transformer network uses a self-attention mechanism to focus on any position in the input sequence without considering the distance, so as to better capture global context information. Therefore, in the technical solution of the present application, the sequence of the scene sound Mel-spectrogram blocks embedded in the coding vector is input into the audio spectrogram transformer network to capture the semantic association of the sequence of Mel-spectrogram blocks in the time dimension, so as to obtain the sequence of the contextual semantic association coding vector of the scene sound Mel-spectrogram blocks. For example, when a sudden braking sound is detected, the transformer network can help the model understand other related events that may occur before and after this sound, such as vehicle deceleration, passenger reaction, etc., and thus provide a more comprehensive and accurate description of the scene.

[0044] Specifically, the S34 performs sequence conditional distinguishable enhancement on the sequence of the scene sound Mel-spectrogram block context semantic association coding features to obtain a sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhanced coding features. Considering that in the application scenario of the driving recorder, different types of traffic events will produce similar but different sound signals. For example, the sound of car horns, the sound of tires rubbing against the ground during emergency braking, the sound of sirens, etc. may all appear in similar frequency ranges. When it comes to specific tasks such as hazard warnings or accident reproduction, it is crucial to have high-quality and distinguishable feature representations. Therefore, when processing the audio data of the driving recorder, it is necessary not only to accurately capture the audio features at each time point, but also to understand the position of these features in the entire audio sequence and their relationship with other parts. Therefore, in order to further improve the model's ability to understand complex audio environments, so that the differences between the contextual semantic association coding features of each scene sound Mel-spectrogram block are more obvious, thereby better capturing the subtle differences between different scenes or elements in the video clips, the present application introduces a sequence conditional distinguishable enhancement mechanism to enhance the sequence of contextual semantic association coding features of the scene sound Mel-spectrogram block to obtain a sequence of contextual semantic association distinguishable enhanced coding features of the scene sound Mel-spectrogram block. In this way, the quality of the spectral features in a single scene sound Mel-spectrogram block is improved, and the coherence and expressiveness of the entire sequence are enhanced, providing more accurate and rich data support for subsequent generation of type labels.

[0045] Specifically, the specific process of performing sequence conditional distinguishable enhancement on the sequence of the scene sound Mel-spectrogram block context semantic association coding features includes: first, performing field mapping on each scene sound Mel-spectrogram block context semantic association coding vector in the sequence of the scene sound Mel-spectrogram block context semantic association coding vector to obtain a sequence of entry scene sound Mel-spectrogram block context semantic association coding vectors. That is, performing field mapping on each of the scene sound Mel-spectrogram block context semantic association coding vectors to convert these scene audio features into a form that is more suitable for the current model or task requirements. More specifically, performing field mapping on each of the scene sound Mel-spectrogram block context semantic association coding vectors in the sequence of the scene sound Mel-spectrogram block context semantic association coding vectors to obtain a sequence of entry scene sound Mel-spectrogram block context semantic association coding vectors according to the following formula; wherein the formula is:

[0046] h i =W1I i W2

[0047] Among them, I iis the i-th scene sound Mel-spectrogram block context semantic association coding vector in the sequence of the scene sound Mel-spectrogram block context semantic association coding vector, W1 and W2 are the first weight matrix and the second weight matrix respectively, h i isI i The corresponding entry scene sound Mel-spectrogram patch context semantic association encoding vector.

[0048] Next, the essential features of the sequence of the entry scene sound Mel-spectrogram block context semantic association coding vectors are calculated to obtain the scene sound Mel-spectrogram block semantic association field essential feature vector. In a specific example, the field depth factors of each entry scene sound Mel-spectrogram block context semantic association coding vector of the sequence of the entry scene sound Mel-spectrogram block context semantic association coding vectors are calculated to obtain the sequence of entry scene sound Mel-spectrogram block context semantic association field depth factors; based on the sequence of entry scene sound Mel-spectrogram block context semantic association field depth factors, the scene sound Mel-spectrogram block context semantic association field essential feature vector of the sequence of entry scene sound Mel-spectrogram block context semantic association coding vectors is calculated. That is, by calculating the field depth factors of each entry scene sound Mel-spectrogram block context semantic association coding vector in the sequence of the entry scene sound Mel-spectrogram block context semantic association coding vectors, the relative importance of the scene feature point in the time period to which it belongs is measured to obtain the sequence of entry scene sound Mel-spectrogram block context semantic association field depth factors. Based on this, the field effect normalization mechanism is used to calculate the field essential features of the sequence of contextual semantic association encoding vectors of the entry scene sound Mel-spectrogram blocks to obtain the field essential feature vector of the contextual semantic association of the scene sound Mel-spectrogram blocks.

[0049] Among them, based on the sequence of the entry scene sound Mel-spectrogram block context semantic association field depth factors, the scene sound Mel-spectrogram block context semantic association field essential feature vector of the sequence of the entry scene sound Mel-spectrogram block context semantic association coding vector is calculated, including: using a softmax function to process the sequence of the entry scene sound Mel-spectrogram block context semantic association field depth factors to obtain a sequence of entry scene sound Mel-spectrogram block context semantic association weight factors; based on the sequence of the entry scene sound Mel-spectrogram block context semantic association weight factors, calculating the position-weighted sum of the sequence of the entry scene sound Mel-spectrogram block context semantic association coding vectors to obtain the scene sound Mel-spectrogram block context semantic association field essential feature vector.

[0050] More specifically, calculating the essential features of the sequence of the context semantic association coding vectors of the entry scene sound Mel-spectrogram block to obtain the scene sound Mel-spectrogram block semantic association field essential feature vector, including: calculating the essential features of the sequence of the context semantic association coding vectors of the entry scene sound Mel-spectrogram block using the following essential feature calculation formula to obtain the scene sound Mel-spectrogram block semantic association field essential feature vector; wherein the essential feature calculation formula is:

[0051]

[0052] a i =softmax(e i )

[0053]

[0054] Among them, ||·|| 2 To calculate the square of the norm of a vector, represents the logarithmic function with base 2, f(h i ) is used to calculate h i The depth of field factor, e i h i The corresponding entry scene sound Mel spectrum block context semantic association field depth factor, softmax(·) is the softmax function, a i h i The corresponding entry scene sound Mel-spectrogram block context semantic association weight factor, v c It is the essential feature vector of the semantic association field of the scene sound Mel-spectrogram patch.

[0055] Then, using the scene sound Mel-spectrogram block context semantic association field essential feature vector as the conditional feature vector, each scene sound Mel-spectrogram block context semantic association coding vector in the sequence of the scene sound Mel-spectrogram block context semantic association coding vector is distinguishably enhanced to obtain a sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhancement coding vectors as a sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhancement coding features. That is, using the scene sound Mel-spectrogram block context semantic association field essential feature vector as additional conditional information, the sequence of scene sound Mel-spectrogram block context semantic association coding vectors is distinguishably enhanced element by element, so as to specifically amplify those features that are directly related to traffic safety and vehicle status assessment, while suppressing background noise and other interference parts that are easily confused or have no practical significance, so as to achieve the purpose of optimizing feature expression.

[0056] More specifically, the scene sound Mel-spectrogram block context semantic association field essential feature vector is used as a conditional feature vector, and each scene sound Mel-spectrogram block context semantic association coding vector in the sequence of the scene sound Mel-spectrogram block context semantic association coding vector is distinguishably enhanced using the following distinguishable enhancement formula to obtain a sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhancement coding vectors as a sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhancement coding features; wherein the distinguishable enhancement formula is:

[0057] O i =I i ⊙[Sigmoid(M m v c +H m I i +b m )]

[0058] Among them, M m is the contextual semantic essential weight matrix of the scene sound Mel-spectrogram patch, H m is the contextual semantic weight matrix of the local scene sound Mel-spectrogram patch, b m is the bias vector, sigmoid(·) is the sigmoid function, ⊙ is the position point multiplication, O i Yes i The contextual semantic association of the scene sound mel-spectrogram patches after semantic enhancement can distinguish the enhanced encoding vector.

[0059] Specifically, the S35 determines that the speech data in the first time period triggers a specific scene based on the sequence of distinguishable enhanced coding features of the scene sound mel-spectrogram block context semantic association. In a specific example of the present application, the S35 includes: inputting the sequence of distinguishable enhanced coding features of the scene sound mel-spectrogram block context semantic association into a classifier to obtain a scene sound prediction category label; in response to the scene sound prediction category label belonging to the type label set of the specific scene, determining that the speech data in the first time period triggers a specific scene.

[0060] More specifically, the classifier uses a linear output layer to classify the sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhanced coding vectors. That is, the sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhanced coding vectors is arranged in chronological order as a one-dimensional feature vector, and then input into the linear output layer for processing, and the input one-dimensional feature vector is linearly transformed by the weight and bias parameters of the linear output layer to map it to the final prediction category label space to determine the scene type corresponding to the original video data, such as normal driving, vehicle collision, pedestrian fall, traffic congestion, etc. Further, when the scene sound prediction category label belongs to the type label set of the specific scene, it means that the voice data in the first time period triggered a specific scene. For example, if the scene sound prediction category label is "vehicle collision", it can be judged that a vehicle collision event occurred during this time period. At this time, high-definition original video data should be saved.

[0061] Preferably, inputting the sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhanced coding vectors into a linear output layer to obtain the scene sound prediction category label comprises:

[0062] Cascading the sequence of the scene sound Mel-spectrogram block contextual semantic association distinguishable enhanced coding vectors to obtain a scene sound Mel-spectrogram block semantic cascade coding vector;

[0063] The feature set of the scene sound Mel-spectrogram block semantic cascade coding vector is subjected to feature clustering to obtain a scene sound Mel-spectrogram block semantic cascade coding in-class feature set and a scene sound Mel-spectrogram block semantic cascade coding out-class feature set, namely:

[0064]

[0065] in, represents the semantic cascade encoding intra-class feature set of the scene sound Mel spectrogram patch, f 1i represents the feature values ​​of the semantic cascade encoding feature set within the scene sound Mel spectrogram block, f 2j Representing each feature value of the semantic cascade encoding out-of-class feature set of the scene sound Mel-spectrogram block;

[0066] The ratio of the number of eigenvalues ​​in the feature set of the scene sound Mel-spectrogram block semantic cascade coding class to the number of eigenvalues ​​in the feature set of the scene sound Mel-spectrogram block semantic cascade coding vector is calculated, that is:

[0067]

[0068] Wherein, k represents the number of eigenvalues ​​in the feature set of the scene sound Mel spectrogram block semantic cascade coding class, n represents the number of eigenvalues ​​in the feature set of the scene sound Mel spectrogram block semantic cascade coding vector, and λ is the ratio between k and n;

[0069] Calculate the ratio of the λth power of the sum of the absolute values ​​of all eigenvalues ​​in the feature set of the scene sound Mel spectrogram block semantic cascade coding class to the λth power of the sum of the absolute values ​​of all eigenvalues ​​in the feature set of the scene sound Mel spectrogram block semantic cascade coding vector to obtain the scene sound Mel spectrogram block semantic cascade coding modulation weight, that is:

[0070]

[0071] Among them, f p represents each feature value in the feature set of the semantic cascade coding vector of the scene sound Mel-spectrogram block, and w1 represents the modulation weight of the semantic cascade coding of the scene sound Mel-spectrogram block;

[0072] Calculate the ratio of the λ / 2 power of the sum of the squares of all eigenvalues ​​in the feature set of the scene sound Mel-spectrogram block semantic cascade coding class to the λ / 2 power of the sum of the squares of all eigenvalues ​​in the feature set of the scene sound Mel-spectrogram block semantic cascade coding vector to obtain the scene sound Mel-spectrogram block semantic cascade coding harmonic weight, that is:

[0073]

[0074] Wherein, w2 represents the semantic cascade coding and harmonic weight of the scene sound Mel spectrogram block;

[0075] For each feature value in the feature set of the scene sound Mel spectrogram block semantic cascade coding class, calculate the product of the feature value and the scene sound Mel spectrogram block semantic cascade coding harmonic weight plus the scene sound Mel spectrogram block semantic cascade coding modulation weight to obtain the optimized feature value, that is:

[0076] f′ 1i =w2×f 1i +w1

[0077] Among them, f′ 1i Representing the optimized feature values ​​of each feature value of the feature set within the semantic cascade coding class of the scene sound Mel spectrogram block;

[0078] For each feature value in the scene sound Mel spectrogram block semantic cascade coding out-of-class feature set, calculate the product of the feature value and the scene sound Mel spectrogram block semantic cascade coding tone value weight to obtain the optimized feature value, that is:

[0079] f 2j′=w1×f 2j

[0080] Among them, f 2j ′ represents the optimized feature value of each feature value of the out-of-class feature set of the scene sound Mel spectrogram block semantic cascade coding;

[0081] The optimized scene sound mel-spectrogram patch semantic cascade coding vector based on the optimized feature values ​​of the scene sound mel-spectrogram patch semantic cascade coding in-class feature set and the scene sound mel-spectrogram patch semantic cascade coding out-class feature set is input into the linear output layer to obtain the scene sound prediction category label.

[0082] Here, in the case where each scene sound mel-spectrogram block context semantic association coding vector in the sequence of scene sound mel-spectrogram block context semantic association coding vectors respectively represents the mel-spectrogram semantic features of each block in the scene sound mel-spectrogram of the speech data within the first time period, when performing feature distinguishability enhancement based on sequence condition description, the sequence condition descriptive differences caused by the mel-spectrogram image semantic differences between each scene sound mel-spectrogram block are taken into consideration, and it is expected to improve the semantically consistent aggregation expression effect of the scene sound mel-spectrogram block semantic cascade coding vector obtained by feature distinguishability enhancement.

[0083] Therefore, while performing feature clustering on the scene sound Mel-spectrogram patch semantic cascade coding vector, the interaction description of the key feature information of the scene sound Mel-spectrogram patch semantic cascade coding vector in the clustering process is carried out, and the geometric equivariant topology of the feature is constructed by low-rank harmonic modulation of the clustering features of the scene sound Mel-spectrogram patch semantic cascade coding vector and the equivariance of the feature as a whole, so as to obtain the schematic distribution translation and rotation symmetry of the clustering features of the scene sound Mel-spectrogram patch semantic cascade coding vector relative to the feature as a whole, thereby introducing geometric message passing into the feature expression of the scene sound Mel-spectrogram patch semantic cascade coding vector and realizing the cluster mapping symmetry of the scene sound Mel-spectrogram patch semantic cascade coding vector through irreducible low-rank order coefficient manipulation on the basis of introducing geometric message passing into the feature expression of the scene sound Mel-spectrogram patch semantic cascade coding vector, thereby improving the consistency of the clustering-based feature representation of the scene sound Mel-spectrogram patch semantic cascade coding vector, thereby improving the accuracy of the scene sound prediction category label obtained by inputting the scene sound Mel-spectrogram patch semantic cascade coding vector into the linear output layer.

[0084] In particular, in S4, if the voice data in the first time period does not trigger a specific scene, the compressed video data is saved. That is, after confirming that the voice data in the first time period does not trigger a specific scene, the driving recorder will compress the original video data in the time period. This process is intended to reduce the size of the video file, thereby effectively utilizing the limited storage space, while ensuring that the early recorded content will not be overwritten due to the large amount of high-definition video occupied during non-critical time periods.

[0085] In summary, the dashcam video processing method according to the embodiment of the present application is explained, firstly, the original video data collected by the vehicle camera in the first time period is obtained, and the mel-spectrogram of the original video data in the first time period is extracted, and then, the mel-spectrogram of the original video data in the first time period is fine-grained spectrum analysis is performed on the mel-spectrogram of the original video data using artificial intelligence-based speech recognition technology, so as to determine whether the speech data in the first time period triggers a specific scene, and determine whether to save the original high-definition video data or compressed video data based on the judgment result. In this way, by intelligently identifying and managing the video data of the dashcam, it can be ensured that the dashcam can completely save key information when encountering a specific scene, thereby further improving the data storage efficiency of the dashcam.

[0086] Furthermore, a driving recorder video processing system is also provided.

[0087] Figure 4 FIG. 1 is a block diagram of a driving recorder video processing system according to an embodiment of the present application. Figure 4 As shown, according to the embodiment of the present application, the dash cam video processing system 300 includes: a Mel-spectrogram feature extraction module 310, which is used to extract the Mel-spectrogram features of the speech data within the first time period to obtain a scene sound Mel-spectrogram; a scene sound Mel-spectrogram block embedding coding module 320, which is used to perform block segmentation and embedding coding on the scene sound Mel-spectrogram to obtain a sequence of scene sound Mel-spectrogram block embedding coding vectors; a context semantic association coding module 330, which is used to perform context semantic association coding on the sequence of scene sound Mel-spectrogram block embedding coding vectors to obtain a sequence of scene sound Mel-spectrogram block context semantic association coding features; a distinguishable enhancement module 340, which is used to perform sequence conditional distinguishable enhancement on the sequence of scene sound Mel-spectrogram block context semantic association coding features to obtain a sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhancement coding features; a specific scene trigger judgment module 350, which is used to determine whether the speech data within the first time period triggers a specific scene based on the sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhancement coding features.

[0088] As described above, the dash cam video processing system 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a dash cam video processing algorithm. In a possible implementation, the dash cam video processing system 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the dash cam video processing system 300 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the dash cam video processing system 300 can also be one of the many hardware modules of the wireless terminal.

[0089] Alternatively, in another example, the dash cam video processing system 300 and the wireless terminal may also be separate devices, and the dash cam video processing system 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0090] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A driving recorder video processing method, characterized in that: include: Acquire raw video data collected by the vehicle-mounted camera within a first time period; Performing video compression processing on the original video data to obtain compressed video data; If the voice data in the first time period triggers a specific scene, saving the original video data; If the voice data within the first time period does not trigger a specific scene, the compressed video data is saved, wherein if the voice data within the first time period triggers a specific scene, the original video data is saved, including: extracting the Mel frequency spectrum features of the voice data within the first time period to obtain a scene sound Mel frequency spectrum graph; performing block segmentation and embedding coding on the scene sound Mel frequency spectrum graph to obtain a sequence of scene sound Mel frequency spectrum block embedding coding vectors; performing contextual semantic association coding on the sequence of scene sound Mel frequency spectrum block embedding coding vectors to obtain a sequence of scene sound Mel frequency spectrum block contextual semantic association coding features; performing sequence conditional distinguishable enhancement on the sequence of scene sound Mel frequency spectrum block contextual semantic association coding features to obtain a sequence of scene sound Mel frequency spectrum block contextual semantic association distinguishable enhancement coding features; based on the sequence of scene sound Mel frequency spectrum block contextual semantic association distinguishable enhancement coding features, determine that the voice data within the first time period triggers a specific scene.

2. The driving recorder video processing method according to claim 1, characterized in that: The scene sound Mel-spectrogram is segmented into blocks and embedded in code to obtain a sequence of scene sound Mel-spectrogram block embedding coding vectors, including: Performing block segmentation on the scene sound Mel-spectrogram to obtain a sequence of scene sound Mel-spectrogram blocks; The sequence of scene sound Mel-spectrogram blocks is embedded and encoded to obtain a sequence of embedded coding vectors of the scene sound Mel-spectrogram blocks.

3. The driving recorder video processing method according to claim 2, characterized in that: The sequence of scene sound Mel-spectrogram blocks embedded in the coding vector is subjected to contextual semantic association coding to obtain a sequence of scene sound Mel-spectrogram block contextual semantic association coding features, including: The sequence of scene sound Mel-spectrogram block embedding coding vectors is input into the audio spectrum graph transformer network to obtain a sequence of scene sound Mel-spectrogram block context semantic association coding vectors.

4. The driving recorder video processing method according to claim 3, characterized in that: The sequence of the scene sound Mel-spectrogram block context semantic association coding features is subjected to sequence conditional distinguishable enhancement to obtain a sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhanced coding features, including: Performing field mapping on each scene sound mel-spectrogram block context semantic association coding vector in the sequence of scene sound mel-spectrogram block context semantic association coding vectors to obtain a sequence of entry scene sound mel-spectrogram block context semantic association coding vectors; Calculating the essential features of the sequence of contextual semantic association encoding vectors of the entry scene sound Mel-spectrogram block to obtain the scene sound Mel-spectrogram block semantic association field essential feature vector; Taking the scene sound Mel-spectrogram block context semantic association field essential feature vector as the conditional feature vector, each scene sound Mel-spectrogram block context semantic association coding vector in the sequence of the scene sound Mel-spectrogram block context semantic association coding vector is distinguishably enhanced to obtain a sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhancement coding vectors as the sequence of scene sound Mel-spectrogram block context semantic association distinguishable enhancement coding features.

5. The driving recorder video processing method according to claim 4, characterized in that: Calculating the essential features of the sequence of contextual semantic association encoding vectors of the entry scene sound Mel-spectrogram block to obtain the scene sound Mel-spectrogram block semantic association field essential feature vector, including: Calculating the field depth factor of each entry scene sound Mel-spectrogram block context semantic association coding vector of the sequence of entry scene sound Mel-spectrogram block context semantic association coding vectors to obtain a sequence of entry scene sound Mel-spectrogram block context semantic association field depth factors; Based on the sequence of the contextual semantic association field depth factors of the entry scene sound Mel-spectrogram block, the scene sound Mel-spectrogram block contextual semantic association field essential feature vector of the sequence of the contextual semantic association encoding vectors of the entry scene sound Mel-spectrogram block is calculated.

6. The driving recorder video processing method according to claim 5, characterized in that: Based on the sequence of the scene sound Mel-spectrogram block context semantic association field depth factors of the entry scene sound Mel-spectrogram block context semantic association field essential feature vectors of the sequence of the scene sound Mel-spectrogram block context semantic association encoding vectors of the entry scene sound Mel-spectrogram block context semantic association field, comprising: Using a softmax function, the sequence of the scene sound Mel-spectrogram block context semantic association field depth factors of the entry scene sound Mel-spectrogram block is processed to obtain a sequence of the scene sound Mel-spectrogram block context semantic association weight factors of the entry scene sound; Based on the sequence of contextual semantic association weight factors of the entry scene sound Mel-spectrogram block, the position-weighted sum of the sequence of contextual semantic association encoding vectors of the entry scene sound Mel-spectrogram block is calculated to obtain the scene sound Mel-spectrogram block contextual semantic association field essential feature vector.

7. The driving recorder video processing method according to claim 6, characterized in that: Determining whether the speech data in the first time period triggers a specific scene based on the sequence of distinguishable enhanced coding features of the scene sound mel-spectrogram block context semantic association includes: Inputting the sequence of distinguishable enhanced coding features of the scene sound Mel-spectrogram block context semantic association into a classifier to obtain a scene sound prediction category label; In response to the scene sound prediction category label belonging to the type label set of the specific scene, it is determined that the voice data in the first time period triggers the specific scene.

8. The driving recorder video processing method according to claim 7, characterized in that: The classifier uses a linear output layer to classify the sequence of distinguishable enhanced coding vectors of the scene sound Mel-spectrogram block context semantic association to obtain the scene sound prediction category label.

9. A driving recorder video processing system, characterized in that: include: A mel spectrum feature extraction module, used to extract mel spectrum features of speech data in a first time period to obtain a scene sound mel spectrum graph; A scene sound Mel-spectrogram block embedding coding module, used for performing block segmentation and embedding coding on the scene sound Mel-spectrogram to obtain a sequence of scene sound Mel-spectrogram block embedding coding vectors; A context semantic association coding module, used for performing context semantic association coding on the sequence of scene sound Mel-spectrogram blocks embedded in the coding vector to obtain a sequence of context semantic association coding features of the scene sound Mel-spectrogram blocks; A distinguishable enhancement module, used for performing sequence conditional distinguishable enhancement on the sequence of the scene sound Mel-spectrogram block contextual semantic association coding features to obtain a sequence of scene sound Mel-spectrogram block contextual semantic association distinguishable enhancement coding features; A specific scene trigger judgment module is used to determine whether the speech data in the first time period triggers a specific scene based on a sequence of distinguishable enhanced coding features based on the contextual semantic association of the scene sound mel-spectrogram block.

Images