Video perception intelligent processing system based on deep learning

Abstract

The invention provides a video perception intelligent processing system based on deep learning, and relates to the technical field of video processing, and the system comprises a video collection module, a preprocessing module, a deep learning model module, a result output module and the like. An advanced deep learning model is adopted for accurate analysis, finally, a processing result is output, the model is trained based on a specific framework and an optimization algorithm, high accuracy and processing speed are achieved, and in traffic monitoring, the vehicle and pedestrian detection accuracy is high, the product quality detection effect is excellent in industrial detection, and in security and protection monitoring, the product quality detection accuracy is high. Compared with a traditional method, the method has obvious advantages, can be widely applied to multiple fields, provides an efficient and accurate solution for video processing, promotes the intelligent development of related fields, can effectively improve the quality of an input video, provides better data for a deep learning model, and has good application prospects. And the detection and classification effects can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of video processing technology, and more specifically, particularly relates to a video perception intelligent processing system based on deep learning. Background Art

[0002] In today's digital and information age, video data has become an important carrier for information transmission and recording. However, how to quickly and accurately obtain valuable information from massive amounts of video data has become a technical problem that needs to be solved. Traditional video processing technology has gradually exposed many shortcomings in response to this challenge.

[0003] In the early days of video processing, most of them relied on simple image processing algorithms and rule-based methods. For example, in target detection, objects are often identified by setting fixed thresholds and features. This method has poor adaptability to environmental changes, the diversity of object shapes, and complex scenes. In the field of video analysis, relying on manually extracted features and manually designed classifiers to understand video content is not only inefficient, but also difficult to guarantee accuracy.

[0004] With the development of computer technology, machine learning methods have been introduced into video processing, but these methods still have limitations. Traditional machine learning algorithms usually require manual feature selection, which is not only time-consuming and laborious, but the quality and effectiveness of the selected features depend largely on human experience and prior knowledge. Moreover, these methods often have difficulty capturing the to complex relationships and deep features in the data.

[0005] At the same time, in practical applications, the demand for video processing in different fields is becoming increasingly diversified and specialized. For example, in intelligent transportation systems, vehicles and pedestrians need to be accurately detected and tracked in real time; in industrial production lines, products need to be quickly and accurately tested for quality; in the field of security monitoring, abnormal behaviors and potential threats must be discovered in a timely manner. Traditional video processing technology is difficult to meet these complex and strict requirements, and cannot achieve efficient, accurate and intelligent video processing.

[0006] In recent years, the rise of deep learning technology has brought new breakthroughs in video processing. Deep learning models can automatically learn rich and effective feature representations from a large amount of data, and have powerful pattern recognition and prediction capabilities. However, the application of deep learning to video processing still faces many challenges, such as model optimization, computing resource requirements, and real-time guarantee. In this context, the present invention proposes a video perception intelligent processing system based on deep learning, which aims to overcome the shortcomings of the existing technology and meet the higher requirements of video processing in various fields. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a video perception intelligent processing system based on deep learning to solve the above problems.

[0008] A video perception intelligent processing system based on deep learning, including a video acquisition module, a preprocessing module, a deep learning model module, and a result output module; The video acquisition module is used to obtain the original video data. It has a variety of video input interfaces and can adapt to different types of cameras, video file formats and network video stream protocols. The preprocessing module is used to perform preprocessing operations on the original video data. It includes an image color space conversion unit, an image cropping and scaling unit, an image frame rate adjustment unit, an image enhancement unit, an image denoising unit and an image format conversion unit. The image cropping and scaling unit adopts a bilinear interpolation algorithm, and the formula is: The image frame rate is adjusted using the frame rate conversion formula: NewFrameRate=OldFrameRate*Ratio, where Ratio is the frame rate adjustment ratio; The deep learning model module is used to analyze and process the preprocessed video data. It adopts a convolutional neural network model built based on a deep learning framework. The convolution layer of the model adopts convolution kernels of sizes 3×3 and 5×5, with the number of convolution kernels being 16 and 32 respectively, and the corresponding step size is configured as 1 and the padding parameter is configured as 1. The pooling layer adopts mean pooling or maximum pooling, and the number of neurons in the fully connected layer is 128. The result output module is used to output the processing results, which includes output options in multiple data formats.

[0009] Preferably, the image enhancement unit in the preprocessing module adopts histogram equalization, and its probability density function calculation formula is: in, The gray level is The number of pixels, is the total number of pixels in the image. The image denoising unit is based on median filtering, and the filter window size is 3×3. The image format conversion unit supports mutual conversion between multiple common image formats such as BMP, JPEG, and PNG.

[0010] Preferably, during the training process, the convolutional neural network model of the deep learning model module adopts a stochastic gradient descent optimization algorithm, the initial value of the learning rate is set to 0.01, and the learning rate decays to 0.1 times the original value every 10 epochs, and the training data undergoes data enhancement operations, including random rotation of ±30°, flipping, scaling 0.8-1.2 times, and adding Gaussian noise with a mean of 0 and a variance of 0.01.

[0011] Preferably, skip connections are used between the convolutional layers in the convolutional neural network model, and the connection coefficient is 0.5 to enhance feature propagation and reuse. The pooling window size of the pooling layer is 2×2, and the step size is 2; the Dropout mechanism is introduced in the fully connected layer, and the Dropout probability is set to 0.5 to prevent overfitting.

[0012] Preferably, the display unit in the result output module supports multiple resolutions such as 1920×1080, 3840×2160 and display modes such as single screen, multi-screen splicing, and split screen; the storage unit adopts a distributed storage architecture, the data compression rate reaches 70%, and supports AES-256 encrypted storage; the data transmission unit adopts wired or wireless transmission mode and complies with the TCP / IP communication protocol.

[0013] Preferably, a model optimization module is also included, which is used to optimize and update the deep learning model according to the new video data. The model pruning technology is used to set the pruning ratio to 30% to reduce the number of model parameters, or the knowledge distillation method is used to set the temperature parameters of the teacher model and the student model to 5 to transfer the knowledge of the complex model to the simpler model.

[0014] Preferably, the online learning method in the model optimization module performs small batch updates based on new data collected in real time, with the amount of data updated being 128 samples each time, while the offline learning method regularly uses a large amount of accumulated new data for full updates, with an update cycle of once a week.

[0015] Preferably, it also includes a user interaction module for receiving user instructions and parameters. It has a permission management function and can set different operation permissions according to user roles. The permissions are divided into administrators, ordinary users and visitors. It also supports user-defined task templates to facilitate the quick start of similar processing tasks.

[0016] Preferably, the operation interface in the user interaction module adopts a graphical interface, the voice command receiving unit supports voice recognition engines such as Baidu and iFLYTEK and languages ​​such as Chinese and English, and the parameter setting unit provides default parameter values ​​and parameter range prompts, and the parameter range is given in the form of a numerical interval.

[0017] Preferably, the camera in the video acquisition module supports different focal length adjustments ranging from 10-50mm, a viewing angle adjustment range of 60-120° and a resolution adjustment up to 4K. The video file supports the parsing of common encoding formats such as H.264 and H.265, and the network video stream supports RTSP and HTTP protocols.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. Significantly improve the detection accuracy: For example, the intelligent traffic monitoring system in Example 1 has a vehicle detection accuracy of more than 95%, and a pedestrian detection accuracy of more than 90%; the industrial product quality detection accuracy in Example 2 is more than 98%. Compared with the 80% vehicle detection accuracy and 75% pedestrian detection accuracy of traditional traffic monitoring in Comparative Example 1, and the 90% product detection accuracy based on simple machine learning in Comparative Example 2, the model based on deep learning of the present invention can more accurately identify and classify targets.

[0019] 2. Significantly improve the processing speed: the processing speed in Example 1 reaches 30 frames / second, the detection speed in Example 2 reaches 60 items / minute, and the system response time in Example 3 is less than 1 second. In contrast, the traditional method in Comparative Example 1 is only about 20 frames / second, and the detection speed in Comparative Example 2 is about 40 items / minute, indicating that the present invention can meet the requirements of real-time and high-speed processing.

[0020] 3. Enhance system flexibility and customization: Through the user interaction module, users can flexibly adjust parameters and set task templates according to different scenarios and needs. For example, in Examples 1-3, traffic management personnel, workers, and security personnel can customize detection targets, confidence thresholds, etc. according to specific circumstances, which improves the system's adaptability to diverse application scenarios.

[0021] 4. Achieve continuous optimization and improvement: The model optimization module can perform online or offline learning based on new data, such as the update strategy described in Examples 1-3, so that the system performance is continuously improved to adapt to data changes and new task requirements, while traditional methods such as Comparative Examples 1 and 2 lack this adaptive optimization capability.

[0022] 5. Widely applicable to multiple fields: The present invention can be effectively applied in multiple important fields such as intelligent transportation, industrial detection, security monitoring, etc., providing efficient and accurate video processing solutions for different industries and solving complex problems in various fields that are difficult to deal with with traditional methods.

[0023] 6. Improve image quality and feature extraction capabilities: Various algorithms and operations in the preprocessing module, such as image cropping, scaling, enhancement, and denoising mentioned in Examples 1-3, can effectively improve the quality of the input video and provide better data for the deep learning model, thereby helping to improve detection and classification effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the basic components of the video perception intelligent processing system in the present invention; Figure 2 It is a schematic diagram of the composition of the pre-processing module in the present invention; Figure 3 is a schematic diagram of an image cropping and scaling unit in the present invention; Figure 4 It is a schematic diagram of the deep learning model module in the present invention; Figure 5 It is a schematic diagram of the components of the pre-processing module in the present invention; Figure 6 It is a schematic diagram of the contents of the deep learning model module in the present invention; Figure 7 It is a schematic diagram of the result output module in the present invention; Figure 8 It is a schematic diagram of the model optimization module in the present invention; Fig. 9 is a schematic diagram of a user interaction module in the present invention; Fig.10 It is a schematic diagram of the video acquisition module in the present invention. DETAILED DESCRIPTION

[0025] With the rapid development of computer vision and artificial intelligence technology, video perception intelligent processing systems based on deep learning have been widely used in many fields, such as security monitoring, intelligent transportation, industrial detection, etc. The video perception intelligent processing system based on deep learning proposed in the present invention aims to provide an efficient, accurate and flexible video processing solution to meet the needs of different application scenarios. The implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0026] Example 1, intelligent traffic monitoring system: System Configuration: Video acquisition module: Install high-definition cameras at the intersections of major urban roads to collect video data with a resolution of 1920×1080 and a frame rate of 30fps.

[0027] Preprocessing module: Use the image cropping and scaling unit to crop the input video into the area of ​​interest (such as the lane) and scale it to a size suitable for the model input; use the image enhancement unit to perform histogram equalization to improve the image contrast.

[0028] Deep learning model module: Build a target detection model based on YOLOv5 to detect vehicles, pedestrians and other targets. The convolution layer of the model uses 3×3 and 5×5 convolution kernels, with 16 and 32 kernels respectively, and the number of neurons in the fully connected layer is 128.

[0029] Result output module: Displays the test results in real time on the monitoring screen, stores the data on the local server, and transmits it to the traffic management center via the network.

[0030] Model optimization module: Using online learning, the model is updated in small batches every hour based on new video data collected in real time.

[0031] User interaction module: provides an operation interface through which traffic management personnel can set detection parameters (such as detection target type, confidence threshold, etc.) and view historical detection records.

[0032] Implementation effect: The vehicle detection accuracy rate reaches over 95%, and it can accurately identify vehicles of different types, colors and postures.

[0033] The pedestrian detection accuracy rate reaches over 90%, effectively reducing false detections and missed detections.

[0034] The real-time processing speed reaches 30 frames per second, meeting the requirements of real-time monitoring.

[0035] Example 2, industrial product quality detection system: System Configuration: Video acquisition module: A high-speed industrial camera is installed on the production line to collect video data with a resolution of 3840×2160 and a frame rate of 60fps.

[0036] Preprocessing module: Use the image denoising unit to perform median filtering to remove noise in the image; use the image frame rate adjustment unit to reduce the frame rate to 30fps to reduce the amount of data.

[0037] Deep learning model module: Use a classification model based on ResNet50 to classify products as qualified or unqualified. The convolution layer of the model uses 3×3 and 5×5 convolution kernels, with 32 and 64 kernels respectively, and the number of neurons in the fully connected layer is 256.

[0038] Result output module: Displays the test results on the display screen on the production line, and sends the information of unqualified products to the control system for automatic removal.

[0039] Model optimization module: adopts offline learning method and uses accumulated new data to fully update the model every day.

[0040] User Interaction Module: Sets the parameter setting unit, where workers can adjust model parameters according to product type and quality standards.

[0041] Implementation effect: The accuracy rate of product quality inspection has reached over 98%, significantly improving the product qualification rate.

[0042] The detection speed reaches 60 pieces / minute, meeting the high-speed detection requirements of the production line.

[0043] Example 3, security monitoring system: System Configuration: Video acquisition module: Install all-round cameras in shopping malls, residential areas and other places to collect video data from different angles and resolutions.

[0044] Preprocessing module: Use the image color space conversion unit to convert the input RGB image into the HSV space to better extract features; use the image enhancement unit to perform gamma correction to enhance the image brightness.

[0045] Deep learning model module: Build a target detection and tracking model based on FasterR-CNN to identify and track multiple targets at the same time. The convolution layer of the model uses 3×3 and 5×5 convolution kernels, with 48 and 96 kernels respectively, and the number of neurons in the fully connected layer is 512.

[0046] Result output module: Displays the detection and tracking results in real time on the large screen of the monitoring center, and sends warning information to the mobile devices of security personnel.

[0047] Model optimization module: Combining online and offline learning methods, a small batch online update is performed every half day, and a full offline update is performed once a week.

[0048] User interaction module: provides a voice command receiving unit, where security personnel can issue operation commands through voice, such as viewing videos in a specific area, locking targets, etc.

[0049] Implementation effect: The accuracy rate of detecting abnormal behaviors reaches over 92%, and it can promptly detect abnormal situations such as theft and fighting.

[0050] The target tracking accuracy reaches over 90%, achieving continuous tracking of the target.

[0051] The system response time is less than 1 second, ensuring real-time performance.

[0052] Comparative Example 1: Traffic monitoring system based on traditional image processing: System Configuration: Video acquisition module: the same as in Example 1.

[0053] Preprocessing module: Use traditional image smoothing, edge detection and other methods for preprocessing.

[0054] Detection module: Use rule- and feature-based target detection algorithms, such as background difference method and frame difference method.

[0055] Result output module: the same as in Example 1.

[0056] Implementation effect: The vehicle detection accuracy is about 80%, which is significantly lower than that of Example 1.

[0057] The accuracy of pedestrian detection is about 75%, with many false detections and missed detections.

[0058] The real-time processing speed is about 20 frames per second, which is difficult to meet the requirements of high frame rate.

[0059] Comparative Example 2, industrial product quality detection system based on simple machine learning: System Configuration: Video acquisition module: the same as in Example 2.

[0060] Preprocessing module: Same as Example 2.

[0061] Detection module: Use machine learning algorithms such as support vector machines (SVM) for classification.

[0062] Result output module: the same as Example 2.

[0063] Implementation effect: The product quality detection accuracy is about 90%, which is lower than the detection accuracy of Example 2.

[0064] The inspection speed is about 40 pieces / minute, which cannot meet the requirements of high-speed production lines.

[0065] In order to more intuitively demonstrate the effect of the present invention, a series of experiments were conducted, and the experimental results were organized into the following table: It can be seen from the data in the above table that the deep learning-based video perception intelligent processing system of the present invention is significantly superior to traditional video processing methods in terms of detection accuracy, processing speed, response time, etc.

[0066] Example 1 is applied to intelligent traffic monitoring, Example 2 is applied to industrial product quality inspection, and Example 3 is applied to security monitoring. Comparative Example 1 is also traffic monitoring but adopts traditional image processing methods, and Comparative Example 2 is industrial product quality inspection but is based on simple machine learning. The embodiment adopts a deep learning-based video perception intelligent processing system, including a specific deep learning model, a preprocessing method, and an optimization module. Comparative Example 1 adopts the traditional rule- and feature-based target detection algorithm for image processing, and Comparative Example 2 adopts a simple machine learning algorithm such as a support vector machine for classification. In terms of key indicators such as detection accuracy, processing speed, and response time, the embodiment is significantly better than the comparative example. For example, the vehicle and pedestrian detection accuracy in Example 1 is higher than that in Comparative Example 1, and the product detection accuracy and detection speed in Example 2 are higher than those in Comparative Example 2. The application of Examples 1-3 in the fields of intelligent transportation, industrial detection, and security monitoring respectively demonstrates the adaptability and effectiveness of the deep learning-based video perception intelligent processing system of the present invention in different scenarios. Compared with the comparative examples, it can clearly highlight the significant improvement in detection accuracy, processing speed, etc. achieved by the deep learning technology used in the present invention, and highlights the advantages brought by technological innovation. Multiple embodiments show that the system can play a role in a variety of important fields and has wide versatility; the comparative examples verify the limitations of traditional methods from the opposite side, further highlighting the superiority of the present invention.

[0067] The deep learning-based video perception intelligent processing system proposed in the present invention has important significance and broad application prospects in today's digital and intelligent era.

[0068] Through the demonstration of multiple embodiments, we have seen the strong adaptability and outstanding performance of the system in different fields. Whether it is vehicle and pedestrian detection in intelligent traffic monitoring, high-precision classification in industrial product quality inspection, or tracking of abnormal behaviors and targets in security monitoring, the system has demonstrated its unique value. In the field of intelligent transportation, it can monitor road conditions in real time and accurately, provide strong data support for traffic management and planning, and help alleviate traffic congestion and improve traffic safety. In industrial production, it can efficiently detect product quality, reduce defective rates, and improve production efficiency and economic benefits. In terms of security monitoring, it can detect potential threats in a timely manner and ensure the safety of personnel and property.

[0069] Compared with traditional video processing methods, the advantages of the present invention are very significant. Traditional methods such as the rule- and feature-based target detection algorithm in Comparative Example 1 and the simple machine learning algorithm in Comparative Example 2 are often constrained by their own limitations when processing complex video data. These methods are difficult to automatically learn deep-level features, and have poor adaptability to environmental changes and new detection objects, resulting in unsatisfactory detection accuracy and processing speed. The deep learning technology used in the present invention can automatically learn rich feature representations from a large amount of data, thereby greatly improving the accuracy of detection and classification, and at the same time, through optimized model architecture and hardware acceleration, high-speed real-time processing is achieved.

[0070] In addition, the preprocessing module, model optimization module, and user interaction module in the present invention cooperate with each other to form a complete and powerful video processing ecosystem. The preprocessing module can effectively improve data quality and provide better input for deep learning models; the model optimization module enables the system to continuously learn and improve and adapt to new data and task requirements; the user interaction module enhances the ease of use and flexibility of the system and meets the personalized needs of different users in different scenarios. In short, the video perception intelligent processing system based on deep learning of the present invention brings innovative solutions to the field of video processing, which not only improves the effect and efficiency of existing applications, but also lays a solid foundation for the development of more intelligent video applications in the future. Its emergence will strongly promote the widespread application and in-depth development of video processing technology in various fields, and create more value for people's lives and social progress.

[0071] The embodiments of the present invention are given for the purpose of illustration and description, and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A video perception intelligent processing system based on deep learning, characterized in that: It includes video acquisition module, preprocessing module, deep learning model module and result output module; The preprocessing module includes an image color space conversion unit, an image cropping and scaling unit, an image frame rate adjustment unit, an image enhancement unit, an image denoising unit and an image format conversion unit, wherein the image cropping and scaling unit adopts a bilinear interpolation algorithm; The deep learning model module adopts a convolutional neural network model built based on a deep learning framework. The convolution layer of the model uses convolution kernels of sizes 3×3 and 5×5, with the number of convolution kernels being 16 and 32 respectively, and the corresponding step size and padding parameter are configured to be 1. The pooling layer adopts mean pooling or maximum pooling, and the number of neurons in the fully connected layer is 128.

2. A video perception intelligent processing system based on deep learning as claimed in claim 1, characterized in that: The image enhancement unit in the preprocessing module adopts histogram equalization, the image denoising unit is based on median filtering, the filter window size is 3×3, and the image format conversion unit supports mutual conversion between image formats.

3. A video perception intelligent processing system based on deep learning as claimed in claim 1, characterized in that: During the training process, the convolutional neural network model of the deep learning model module adopts a stochastic gradient descent optimization algorithm, the initial value of the learning rate is set to 0.01, and the learning rate decays to 0.1 times the original value every 10 epochs. The training data undergoes data enhancement operations, including random rotation of ±30°, flipping, scaling 0.8-1.2 times, and adding Gaussian noise with a mean of 0 and a variance of 0.

01.

4. A video perception intelligent processing system based on deep learning as claimed in claim 3, characterized in that: The convolutional layers in the convolutional neural network model use skip connections with a connection coefficient of 0.5 to enhance feature propagation and reuse. The pooling window size of the pooling layer is 2×2 and the step size is 2. The Dropout mechanism is introduced into the fully connected layer, and the Dropout probability is set to 0.5 to prevent overfitting.

5. The video perception intelligent processing system based on deep learning as claimed in claim 1, characterized in that: The storage unit adopts a distributed storage architecture, with a data compression rate of 70% and supports AES-256 encrypted storage. The data transmission unit adopts wired or wireless transmission and complies with the TCP / IP communication protocol.

6. A video perception intelligent processing system based on deep learning as claimed in claim 1, characterized in that: It also includes a model optimization module, which uses model pruning technology with the pruning ratio set to 30% to reduce the number of model parameters, or uses a knowledge distillation method with the temperature parameters of the teacher model and the student model set to 5 to transfer the knowledge of the complex model to a simpler model.

7. A video perception intelligent processing system based on deep learning as claimed in claim 6, characterized in that: The online learning method in the model optimization module performs small batch updates based on new data collected in real time, with the amount of data updated being 128 samples each time. The offline learning method regularly uses a large amount of accumulated new data to perform full updates, with an update cycle of once a week.

8. The video perception intelligent processing system based on deep learning as claimed in claim 1, characterized in that: It also includes a user interaction module, which has a permission management function. Different operation permissions can be set according to user roles. The permissions are divided into administrators, ordinary users and visitors, and it also supports user-defined task templates.

9. A video perception intelligent processing system based on deep learning as claimed in claim 8, characterized in that: The parameter setting unit in the user interaction module provides default parameter values ​​and parameter range prompts, and the parameter range is given in the form of a numerical interval.

10. The video perception intelligent processing system based on deep learning as claimed in claim 1, characterized in that: The camera in the video acquisition module supports different focal length adjustments ranging from 10-50mm, viewing angle adjustment range of 60-120° and resolution adjustment up to 4K, and the network video stream supports RTSP and HTTP protocols.

Images