Mine front-end AI video computing power terminal device

Abstract

The invention discloses a mining front-end AI video computing power terminal device comprising a video coding module based on deep learning, and the video coding module is used for improving the efficiency of a data compression algorithm under a low-bandwidth network condition; the video acquisition module is used for capturing video data; the data processing module is used for analyzing and optimizing the received video data in real time; the transmission control module is used for adjusting a video transmission strategy to reduce delay; and the feedback processing module is used for dynamically adjusting a coding parameter function based on feedback information so as to further solve the problem of video transmission delay. According to the invention, the problem of video transmission delay caused by low efficiency of a data compression algorithm under a low-bandwidth network condition can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of video computing power terminal equipment, and specifically, to a front-end AI video computing power terminal equipment for mines. Background Art

[0002] In dangerous places such as mines, a large number of monitoring equipment need to be deployed to ensure the safety of the place. Since mines are generally located in remote areas, the signal is not strong, and it is difficult to deploy high-bandwidth networks to process monitoring data like in general places. Under low-bandwidth network conditions, due to the inefficiency of data compression algorithms, video data often encounters delay problems in network transmission, which not only affects the smoothness of the video stream, but also may greatly reduce the effectiveness of some application scenarios that require real-time decision-making. Summary of the invention

[0003] The purpose of the present invention is to propose a front-end AI video computing terminal device for mines to overcome the above-mentioned shortcomings in the prior art.

[0004] To achieve the above technical objectives, the technical solution of the present invention is implemented as follows:

[0005] A front-end AI video computing terminal device for mines, comprising:

[0006] A deep learning-based video encoding module, which is used to improve the efficiency of data compression algorithms under low-bandwidth network conditions;

[0007] A video acquisition module, used for capturing video data;

[0008] A data processing module, used for real-time analysis and optimization of received video data;

[0009] A transmission control module, used to adjust the video transmission strategy to reduce delay;

[0010] The feedback processing module dynamically adjusts the encoding parameters based on the feedback information to further solve the problem of video transmission delay.

[0011] Furthermore, the deep learning-based video encoding module improves the efficiency of the data compression algorithm through the following specific steps: feature extraction based on content differences between video frames, specifically including the following sub-steps:

[0012] Use a pre-trained neural network model to analyze the pixel distribution characteristics of video frames;

[0013] Calculate the change amount of pixel points between two consecutive frames and use it as the quantization change parameter \(D\). Here, the two consecutive frames are \(F_{n1}\) and \(F_n\) respectively, that is, the formula is \(D = \|F_n - F_{n1}\|\), which is used to evaluate the similarity between frames in the video stream. \(F_n\) represents the current frame, \(F_{n1}\) represents the previous frame, and \(\|\cdot\|\) is a certain vector norm;

[0014] Dynamically adjust the quantization step size in the compression coding algorithm based on the change parameter \(D\);

[0015] Perform further encoding and compression operations on the quantized features to reduce the volume of the encoded file.

[0016] Furthermore, when the above-mentioned video coding module based on deep learning is working, the dynamic adjustment of features further includes the following four specific processes:

[0017] Analyze the activity intensity of the intra-frame region, calculate the pixel differences between different regions (such as each small block) within the frame, and set this as \(A_i\). Here, \(A_i\) refers to the activity level parameter of the \(i\)-th small block;

[0018] If it is found that \(A_i\) is greater than the pre-set threshold \(T\) (i.e., \(A_i > T\)), it indicates that the activity within the block is significant, and the encoding details need to be strengthened to maintain clarity;

[0019] In the case of \(A_i < T\), the detail retention level within the block can be appropriately reduced to optimize the overall encoding efficiency;

[0020] Finally, take differential optimization measures according to the partitions with different activity levels.

[0021] Furthermore, a more detailed evaluation method is adopted for the calculation of the intra-frame activity intensity described above:

[0022] Calculate the gradient magnitude of each pixel value in the frame to measure the motion condition within the region;

[0023] Based on the difference \(\delta(x,y) = P(x + h,y)-P(x,y)\) obtained at the pixel position \((x,y)\) and the adjacent pixel value \(h\) in the horizontal direction, where \(P\) represents the gray level or color intensity value of the pixel, evaluate the vertical direction;

[0024] When the final result \(R(x,y)\) obtained from the average value of \((\delta(x,y))^2 + (\delta(y,x))^2\) is large, it indicates frequent regional movement; otherwise, the movement is weak. The average value of \((\delta(x,y))^2 + (\delta(y,x))^2\) is used to comprehensively evaluate the activity level within the region and is then input as an important part of \(A_i\).

[0025] Furthermore, when dynamically adjusting the encoding settings based on inter-frame and intra-region activity, the specific mechanism by which the current environmental factors affect the encoding decision needs to be considered as follows:

[0026] Collect information on the current bandwidth and CPU load, and set two variables, the current bandwidth status \(B\) and the load level \(U\), for the adjustment decision basis;

[0027] When using these two dynamic variables to determine the optimal compression ratio, a function such as \(C(B,U)\) is applied to predict the possible impact level during the encoding process;

[0028] The function \(C\) reflects that the need for a conservative compression strategy increases in an environment with a smaller bandwidth limit \(B\) or a higher processor usage rate \(U\), and vice versa. The adjustment coefficients \(k_1\), \(k_2\) and their exponential power forms corresponding to the variables \(B\), \(U\) are used to quantify the impact relationship in the form of \(C(B,U)=k_1B^a\cdot k_2U^b\).

[0029] Furthermore, when adjusting the encoding parameters, more fine-grained fine-tuning must be performed based on the received feedback information to ensure the best encoding effect:

[0030] Accept feedback data from end-users, especially performance criteria such as video quality scores (e.g., QoS parameters, defined as \(S\)) and transmission delay \(Td\);

[0031] Combined with the results of the above \(C\) function, if \(C\) is large but \(S + \alpha Td < Th\) (this condition indicates that even a high compression coefficient leads to a significant decrease in the experience score, where \(Th\) is the acceptable threshold and \(\alpha\) is the delay impact weighting).

[0032] Advantages of the present invention: The present invention can solve the problem of video transmission delay caused by low efficiency of data compression algorithms under low-bandwidth network conditions. Description of the Drawings

[0033] Figure 1 It is a schematic diagram of the functional module structure of the terminal device described in the embodiment of the present invention;

[0034] Figure 2 It is a schematic diagram of a front structure of a specific application of the terminal device according to an embodiment of the present invention;

[0035] Figure 3 It is a schematic diagram of the back structure of a specific application of the terminal device according to the embodiment of the present invention;

[0036] Figure 4 It is a specific application of the terminal device described in the embodiment of the present invention. Figure 1 ;

[0037] Figure 5 It is a specific application of the terminal device described in the embodiment of the present invention. Figure 2 . DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention are described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0039] Reference Figure 1, the mine front-end AI video computing power terminal device of the present invention achieves the purpose of efficient transmission of video data under low bandwidth conditions through the following steps. First, the deep learning-based video encoding module in the front-end device provides strong video compression support for the core of the overall solution, can identify important elements and unimportant background parts in the video screen, and give priority to compressing the background or the less changing parts. This intelligent selective encoding method not only reduces the amount of data required for transmission but also improves the compression quality; at the same time, it will combine the feedback information sent by the back-end platform to continuously fine-tune the training parameters of the model to make it closer to the real application scenario. The video acquisition module follows closely. This module is usually integrated into front-end devices such as smart devices such as surveillance cameras. It can capture changes in the surrounding environment and convert them into digital signals and input them into the computing system; the information obtained at this stage will determine the effectiveness and accuracy of subsequent processing. Then the data processing module starts working: it immediately starts the analysis and optimization tasks after receiving the original video stream from the video acquisition module. During this process, the module uses a deep neural network (DNN) to identify the image content in real time and make targeted improvements to ensure the clear transmission of important scenes and the appropriate degradation of non-critical parts. For example, when monitoring crowded areas, higher resolution may be adopted for transmission, so as to facilitate more detailed information identification by security monitoring personnel; while in relatively static background or sparsely populated areas, lower precision images can be used to reduce data volume and bandwidth requirements, thereby saving network resources; in some cases, if the system determines that the picture changes are not obvious, frame interval skipping can be used to save the size of uploaded data, thereby reducing the transmission delay caused by compression. Finally, the key component, the transmission control module, comes on stage. The device can flexibly adjust the rate and delay tolerance strategy of the above video transmission according to the current network conditions; for example, when the Internet connection speed is detected to be slow or unstable, the system can automatically switch to a lower data bit rate and a more effective compilation method to ensure a smooth video playback experience, even in the case of extremely limited network bandwidth. It can also regularly detect and evaluate the effects of all the above steps based on the data feedback information mechanism mentioned above. If it is found that there are obvious quality problems in video transmission, the parameters of the video codec will be adjusted immediately until the best balance between network conditions and image quality is found. In one embodiment, a smart transportation company deployed multiple surveillance cameras equipped with the AI ​​terminal device on its city streets to assist in the operation of traffic flow management and traffic accident rapid response systems. Even when network pressure doubles during rush hour in the morning and evening, it can still achieve stable and reliable information push, significantly shortening the time window required for dispatch response personnel to arrive at the scene when an accident occurs, further ensuring the safety of people's travel.In summary, this solution, through detailed planning of each part of the entire processing flow, fully leverages the advantages of artificial intelligence video encoding to effectively reduce data congestion caused by low transmission rates, and enables surveillance videos to be delivered clearly and quickly even through weak connections. Ultimately, it achieves the construction of a video surveillance system that can still work efficiently and stably under bandwidth constraints, contributing its own modest efforts to the maintenance of modern urban infrastructure.

[0040] Next, the deep learning-based video encoding module of the present invention is described to achieve improved efficiency of the data compression algorithm through the following specific steps: feature extraction based on content differences between video frames, specifically including the following sub-steps: using a pre-trained neural network model to analyze the pixel distribution characteristics of the video frame; calculating the change in pixel points between the previous and next frames, and using it as a quantization change parameter \(D\), where the previous and next frames are \(F_{n1}\) and \(\(F_n\), respectively, that is, the formula is \(D=\|F_n-F_{n1}\|\), which is used to evaluate the similarity between frames in the video stream, \(F_n\) represents the current frame, \(F_{n1}\) represents the previous frame, and \(\|\cdot\|\) is a certain vector norm; dynamically adjusting the quantization step size in the compression coding algorithm based on the change parameter \(D\); performing further encoding compression operations on the quantized features to reduce the size of the encoded file.

[0041] First, a pre-trained neural network model is used to analyze the details of each frame of the video. Specifically, at this stage, the pre-trained model is used to identify different content characteristics in each frame, such as boundary detection and color distribution patterns. This not only reduces the redundant processing of information in more complex image areas, but also makes the compression process more focused on retaining some information that has changed significantly.

[0042] Subsequently, by calculating the difference between two consecutive frames as a measure of dynamic change \(D\), it helps to identify the extent to which adjacent frames in the video differ. In this calculation, \(D=\|F_n-F_{n1}\|\), where \(\|\cdot\|\) may choose the L1 or L2 norm. For example, in some cases, the L2 norm \(D=\sqrt{\sum(F_n-F_{n1})^2}\) can be selected. This formula means that \(D\) represents the overall measure of the difference between the two images at the pixel level. Depending on the actual situation and optimization goals, \(\|\cdot\|\) can also choose different forms or parameter settings as needed. The purpose of this setting is to quantify the visual differences between images into numerical values ​​that can be intuitively expressed through mathematical means, which is helpful for subsequent operations.

[0043] After that, the data of the video frames are cropped and simplified to different extents according to the change parameter. If the similarity of the pictures of two videos is very high, the quantization coefficient can be increased to compress the information as much as possible while ensuring the basic viewing experience, so as to achieve the purpose of reducing the memory occupancy. On the contrary, if the frame content changes greatly, this parameter should be reduced to maintain a higher restoration accuracy.

[0044] The last step is to reorganize the feature data with the adjusted information density and retention amount after the above steps, and use a higher-level coding strategy to convert it into the final storage or sharing format. In this process, while minimizing the redundant part, key information components should not be lost, so as to meet the dual requirements of both compressing the storage capacity and ensuring the video quality.

[0045] In one embodiment, a high-performance hardware device is deployed in a front-end intelligent video processor, and the above intelligent coding module is integrated. The front-end hardware processing terminal receives the real-time video input from multiple surveillance cameras installed on-site. By performing the multi-stage data optimization as described above on these pictures, the data traffic demand for uploading to the cloud center is effectively reduced, greatly saving the time-consuming and corresponding communication costs during the transmission process. At the same time, it also ensures that the client can obtain an actual playback experience with smoothness and picture quality meeting the user's requirements. Due to the existence of the front-end processor and the related algorithm mechanisms implemented by it, excellent application flexibility and economic benefits can be demonstrated in various scenarios such as security monitoring or online education and training, becoming a powerful tool to promote the popularization and application of such technologies.

[0046] Next, when the video coding module based on deep learning of the present invention is working, the dynamic adjustment of features further includes the following four specific processes: analyzing the activity intensity of the intra-frame region, calculating the pixel differences between different regions (such as each small block) within the frame, and setting this as \(A_i\), where \(A_i\) refers to the activity level parameter of the \(i\)-th small block; if it is found that \(A_i\) is greater than a pre-set threshold \(T\) (i.e., \(A_i>T\)), it indicates that the activity within the block is significant, and the coding details need to be strengthened to maintain clarity; in the case of \(A_i<T\), the detail retention level within the block can be appropriately reduced to optimize the overall coding efficiency; finally, differential optimization measures are taken according to the partitions with different activity levels.

[0047] Analyzing the activity intensity of an area within a frame means estimating the complexity of these blocks by calculating the degree of pixel change in each small block. For example, if a small block contains fast-moving or highly detailed information, the activity in this block will be more intense. By comparing the pixel values ​​in these areas one by one or using other suitable methods to quantify the amount of change within the block, an activity index \(A_i\) can be defined. In this process, \(A_i\) takes values ​​in the range of 0 to 1, and highly active blocks may have values ​​closer to 1.

[0048] For the threshold \(T\), this is a benchmark parameter set in advance according to the needs of the system and the typical input video type, which is used as a boundary condition to decide whether to add more attention to the block or allow it to simplify processing. The choice of threshold directly affects the balance between encoding quality and processing speed. Suppose the optimal \(T\) value of a system is determined to be 0.65 through repeated tests. When \(A_i>0.65\) is detected, it indicates that the small block content is dynamically rich or has a high degree of detail density, and additional computing resources are required to accurately express the block details; otherwise, that is, when \(A_i<0.65\), the block is considered to be relatively static or has fewer details. At this time, you can choose to reduce the detail encoding level, reduce the computational burden while maintaining an acceptable level of overall video quality, thereby improving compression efficiency.

[0049] The last step is to develop differentiated solutions based on the partitions of different activity levels. For example, the entire video is divided into multiple grid cells, and corresponding enhancement and simplification strategies are designed for the characteristics of each part. In one embodiment, assuming that the terminal device being processed captures a picture with extremely dynamic changes in real time, the block corresponding to the high \(A_i\) value will automatically enter the fine processing mode; and for some background or static object images with low \(A_i\) values, a simpler and more efficient compression technology may be activated, which reduces unnecessary data redundancy while ensuring the quality of the main dynamic picture, and ultimately achieves the rational use of resources and improves the video encoding effect. In this way, not only the compression process is optimized, but also a good user experience is ensured even under conditions of limited hardware resources.

[0050] Next, a more detailed evaluation method for calculating the activity intensity in the aforementioned frame is described in the present invention:

[0051] Calculate the gradient magnitude of each pixel value in the frame to measure the motion in the area;

[0052] Based on the difference \(\delta(x,y)=|P(x+h,y)-P(x,y)|\) obtained at the pixel position \((x,y)\), and the adjacent pixel value \(h\) in the horizontal direction, where \(P\) represents the grayscale level or color intensity value of the pixel, the vertical direction is evaluated;

[0053] When the final result \(R(x,y)\) obtained by the average value of \((\delta(x,y))^2+(\delta(y,x))^2\) is large, it means that the area moves frequently, otherwise the movement is weak. The average value of \((\delta(x,y))^2+(\delta(y,x))^2\) is used to comprehensively assess the activity level in the area, and then as an important part of the input of \(A_i\).

[0054] In this method, the gradient amplitude of each pixel in the frame is first calculated to detect whether there is significant motion change in each area of ​​the picture. Specifically, the derivative of each pixel is used to obtain the change between two adjacent time frames. A large gradient indicates a large motion intensity. This method can effectively distinguish between still images and videos containing moving objects.

[0055] Then, for a specified pixel position \((x,y)\), the degree of change at that location is estimated by measuring the grayscale difference between horizontal and vertical adjacent points, that is, the pixel intensity value \(\delta(x,y)\). The difference here \(\delta(x,y)\) is defined as the absolute difference between the brightness or color intensity at a position with a horizontal distance of \(h\) (typically \(1\)) and the current position. This is done to identify any state transitions that may occur in the lateral direction and lay the foundation for the next stage of calculation. The parameter \(P\) indicates the actual brightness level of the corresponding point. In practical applications, this can be the raw data directly output by the image sampler, or the grayscale information or color space transformation result after appropriate preprocessing;

[0056] After that, after obtaining the horizontal and vertical deviations, the design idea of ​​the formula \((\delta(x,y))^2+(\delta(y,x))^2\) is to square these changes to highlight the pixel positions that have changed dramatically. Finally, the average is taken to form a single motion activity index \(R(x,y)\) in the area, which is used to represent the frequency of activity in the small area. The advantage of this is that it eliminates certain influencing factors caused by changes in external illumination, leaving only signals that are closely related to the dynamics of the actual object, and simplifies the complexity of the model to make it more suitable for real-time video processing tasks. The activity is determined based on the mean. When the average is high, it can be determined that the area is more active and has more active targets. Conversely, a lower activity level means that the image is stable and does not change much. This quantitative method can be used as a reference standard for the next step of judging the input \(A_i\);

[0057] In one embodiment, the terminal device receives a video stream, selects a continuous frame in a test sample, and first captures the signs of motion of all pixel positions through gradient calculation. For example, in a traffic monitoring scenario, the busy road and the almost unchanged background on the roadside will produce a clear contrast. Afterwards, for the road part, it is assumed that the difference between the same pixel at two adjacent moments \(h=1\) is known, so the intensity contrast change \(\delta\) at the corresponding position before and after the moment is obtained. The overall motion intensity information of the region \(R\) is obtained by taking the quadratic average of these differences, and finally the sensitivity of the scene activity analysis or the tracking strategy is adjusted based on this indicator, such as responding to fast-moving objects in a timely manner while reducing the system overhead caused by misjudgment. After such operation, the noise influence under the fixed background can be effectively filtered out to highlight the action characteristics of the subject, improve the recognition accuracy and execution rate, so that the video analysis module can more flexibly respond to various real-time monitoring needs under limited resource conditions.

[0058] Next, the specific mechanism of the present invention for dynamically adjusting the encoding settings based on the activity between frames and within a region while also considering the current environmental factors affecting the encoding decision is described as follows:

[0059] First, collect information about the current bandwidth \(B\) and CPU load \(U\). These two reflect the effective throughput of the communication link and the status of the local processing capacity, respectively, and provide important basis for the subsequent adaptive coding adjustment strategy. Specifically, \(B\) is usually measured in gigabits per second (Gbps) or megabits per second (Mbps), and \(U\) is indicated by the percentage of CPU utilization (such as 20%, 60%);

[0060] Next, these two real-time monitoring data are used to determine the appropriate video compression level \(C\) and adjust it as part of the video encoding strategy. The mathematical function \(C(B,U)\) is used to predict potential encoding quality issues caused by resource constraints. Specifically, when the system adjusts the compression ratio according to actual needs to balance the relationship between performance and resources, it is necessary to consider the conservative strategy tilt of \(C\) under low bandwidth and high computing pressure conditions;

[0061] The expression form of the \(C\) function is chosen as \(C(B,U)=k_1B^a\cdot k_2U^b\). The purpose of this equation is to reflect the influence of bandwidth status \(B\) and processor utilization \(U\) on the selection of the final compression strategy. Among them, \(k_1,k_2\) are positive coefficients related to the actual application background; the coefficient exponential term \(a\) is generally taken in \([0,1]\) to indicate that the value of \(C\) increases more rapidly as the bandwidth decreases; similarly, \(b\) in the \([1,2]\) interval indicates that the influence on \(C\) becomes stronger as the processor load increases. The reason for selecting these parameters is mainly to reflect how bandwidth and CPU load linearly and nonlinearly affect the compression effect;

[0062] Specifically, in an environment where a terminal device is working, if the network bandwidth drops to 5Mbps and the server is experiencing more than 70% CPU usage, it is necessary to adjust \(k_1,k_2\) in the formula so that \(B^a\) contributes significantly to the \(C\) value and \(U^b\) improves the compression ratio, ensuring reasonable image quality and video smoothness under adverse conditions.

[0063] This solution enables the terminal to adaptively adjust video compilation parameters and optimize performance according to environmental changes. It not only reduces the risk of transmission failure and ensures that the user experience is not excessively damaged by technical bottlenecks, but also seeks the most efficient computing resource allocation and data transmission solution under limited conditions. For example, during the peak period of the live broadcast platform, the use of this mechanism to dynamically adjust parameters can avoid the problem of service quality degradation caused by the sudden increase in user access requests, and improve the reliability of the overall system and user experience satisfaction.

[0064] Next, it is described that the adjusted coding parameters of the present invention must also perform more fine-grained fine-tuning according to the received feedback information to ensure the best coding effect: receiving end-user feedback data, especially performance criteria such as video quality scores (e.g., QoS parameters, defined as \(S\)), and transmission delay \(Td\); combining the results of the above \(C\) function, if \(C\) is large but \(S + \alpha Td < Th\) (this condition indicates that even though the compression coefficient is high, it leads to a significant decrease in the experience score, where \(Th\) is the acceptable threshold and \(\alpha\) is the weighting of the delay impact).

[0065] The first step includes collecting feedback information from end-users in the actual usage scenario. This mainly includes two aspects: one is related to video transmission quality and user experience metrics, such as the video quality score parameter \(S\), that is, Quality of Service (QoS), which measures the smoothness and image clarity of the video during actual playback; the other is the transmission delay parameter \(Td\), which refers to the time interval when the user receives the video data, and generally, the smaller this time interval is, the better. These two together determine the overall video experience of the user. For example, when the score of \(S\) is 3 / 5 and the average of \(Td\) is 50 milliseconds, it is considered that the experience is average at this time and still needs improvement.

[0066] Next, the obtained data information is combined with a known calculation function \(C\) - which may represent relevant values such as video compression ratio or algorithm efficiency. This process requires a reasonable assessment of the size of \(C\). If the value of \(C\) itself is large, it means a deeper compression level, then further check the formula \(S + \alpha Td < Th\), where \(Th\) represents a preset acceptable minimum score, which is a threshold judgment parameter, generally ranging from 0 to 5, and the default can be set to 2 (depending on the specific requirements of different scenarios). And \(\alpha\) is set as a delay factor for weighting, determined according to actual application experience, and its value can range from 0 to 1, and the default can be selected as 0.2, so as to find a good compromise between the quality \(S\) and the delay \(Td\).

[0067] If the formula result indicates that although the current compression coefficient is relatively high, the actual degradation of the video experience is significant. Specifically, if it is found that \(C\) is large (assumed to be 8 / 10), and according to the example given above, \(S = 3 / 5\), \(\alpha = 0.2\), then when \(Td\geq 25\) milliseconds, \(3 / 5 + 0.2*25 = 6 / 5 = 1.2 < Th\) (assuming \(Th = 2\) here) holds, indicating that although the video file has been successfully transmitted after high compression processing, the final presented quality under this specific delay condition is lower than the user acceptance level, and the encoding algorithm needs to be reconfigured.

[0068] To make this concept more concrete, in a specific case, it can be seen that after a live streaming service provider launched an AI-accelerated video processing tool and received feedback from customers about increased lag and blurred images, etc., it immediately enabled this adaptive optimization solution to make subtle changes to the compression level of the video. The adjusted video file still ensured fast delivery while the quality was significantly improved. Through such a dynamic feedback mechanism, the overall competitiveness of the service and the user experience were ultimately improved.

[0069] The terminal device of the present invention includes a video encoding module, a video acquisition module, a data processing module, a transmission control module based on deep learning technology, and a function of dynamically adjusting encoding parameters based on feedback information. The entire design reflects the core concept of intelligent management from video acquisition to processing and transmission, improves the real-time processing and transmission efficiency of video data under low-bandwidth network conditions, effectively reduces the occurrence of latency phenomena, and thus solves the problem of video transmission delay caused by low efficiency of the data compression algorithm.

[0070] Specifically, the video acquisition module first captures the video source data to be transmitted; then the video encoding module using advanced deep learning technology is responsible for the efficient digital encoding and compression operation of the original video signal. It adaptively selects the optimal algorithm mode to convert information and condense data according to the actual environmental changes and needs, significantly improving the compression ratio and transmission rate; then the data processing module receives the compressed streaming media files, performs in-depth information processing, and makes corresponding improvements according to the actual situation; next, the transmission control module flexibly arranges the sending order and time point based on the received files to be sent according to the preset principles and the personalized needs of users, and uses the network quality assessment system to make a judgment on the congestion status on the transmission path; finally, the dynamic adjustment technology that combines the feedback mechanism of the network performance monitoring tool and the artificial intelligence algorithm can realize the intelligent adaptation process of the encoding parameters changing with the current connection environment, while reducing the bitstream bandwidth occupancy and maintaining the stability of the picture quality, so that the terminal can achieve a relatively smooth and stable transmission state even under the background of poor network speed. In summary, the optimization combination of each step can improve the overall service performance and reduce the interruption of video playback when the network bandwidth is insufficient.

[0071] like Figure 2-5 As shown, the present invention also discloses a specific application structure diagram of a front-end AI video computing power terminal device for mines, and its specific structure description is shown in the following table:

[0072] Serial number interface or indicator light description

[0073] 1 ground terminal is used for external ground lead terminal

[0074] 2 power supply DC 48V power input interface

[0075] 3-switch system power switch

[0076] 4RS485 sensor interface, can be connected to RS485 standard sensors

[0077] 5RS232 sensor interface, can connect to RS232 standard sensor 6POE CAM 12-way POE camera interface, support IEEE802.3 at / af standard power supply 7LAN Ethernet interface, can connect to the public network

[0078] 8HDMI high-definition video output interface, can be connected to the display screen

[0079] 9SIM card slot SIM card slot, supports full network access

[0080] 10TF card slot expansion storage interface, you can insert a large capacity memory card

[0081] 11 Beidou antenna Beidou antenna interface, external antenna

[0082] 124G / 5G antenna 4G / 5G antenna interface, used to connect an external antenna

[0083] 13USB 2-way USB interface, can connect to USB devices such as mouse, keyboard, mobile hard disk, etc. 14Signal status light POE indicator light 12-way POE interface signal indicator light

[0084] LAN indicator Gigabit network port signal indicator

[0085] RS485 indicator light RS485 signal indicator light

[0086] RS232 indicator light RS232 signal indicator light

[0087] Beidou signal indicator Beidou signal indicator

[0088] 5G signal indicator light 5G signal indicator light

[0089] WORK indicator light system APP working indicator light

[0090] POWER indicator light System power working indicator light

[0091] USB indicator USB power indicator

[0092] Its specifications and parameters are shown in the following table:

[0093] System Parameters

[0094] System Android 12.0

[0095] CPU octa-core 64-bit (4×Cortex-A76+4×Cortex-A55)

[0096] Main frequency 2.4GHz

[0097] Memory 8GB LPDDR4

[0098] Storage 64GB high-speed eMMC 5.1

[0099] Expandable storage

[0100] External TF card slot

[0101] Built-in 1TB SSD hard drive

[0102] Video surveillance

[0103] The maximum number of channels can connect to 12 POE cameras at the same time

[0104] PTZ support

[0105] Environmental adaptability

[0106] Working temperature -20℃~+60℃

[0107] Storage temperature -30℃~+65℃

[0108] Working humidity 10%~90%

[0109] communication

[0110] Wireless communication 4G, 5G, supporting the four major domestic operators

[0111] Wired communication supports wired Gigabit access

[0112] Beidou Positioning

[0113] Positioning accuracy (pure open area) <3m (horizontal) (no differential)

[0114] <10m (elevation) (no difference)

[0115] Speed ​​measurement accuracy <0.1m / s

[0116] First positioning time <28s (cold start) 1s (hot start) 1s (recapture)

[0117] Sensitivity -147dBm (capture), -163dBm (tracking)

[0118] Data update rate 1Hz

[0119] Power System

[0120] Power supply DC 48V

[0121] Power Standby power consumption <20W Peak power consumption 120W (connecting 12-way POE cameras)

[0122] Peripheral Interface

[0123] RS232 supports standard Modbus protocol for connecting various sensor devices

[0124] RS485 supports standard Modbus protocol for connecting various sensor devices

[0125] Video 1 HDMI 2.0 interface

[0126] USB 2-way USB interface

[0127] The methods, programs, systems, devices, etc. of the embodiments of the present invention may be executed or implemented in a single or multiple networked computers, or may be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks may be performed by remote processing devices connected via a communication network.

[0128] Those skilled in the art should understand that the embodiments of the present specification may be provided as methods, systems or computer program products. Therefore, those skilled in the art may imagine that the functional modules / units or controllers and related method steps described in the above embodiments may be implemented in software, hardware or a combination of software / hardware.

[0129] Unless explicitly stated, the actions or steps of the methods, programs, and embodiments of the present invention do not have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0130] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are merely examples of the best modes for implementing the present systems and methods. It will be appreciated by those skilled in the art that various changes may be made to the embodiments of the systems and methods described herein when implementing the present systems and / or methods without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A front-end AI video computing terminal device for mines, characterized in that: Including: A video encoding module based on deep learning, which is used to improve the efficiency of data compression algorithms under low-bandwidth network conditions; A video acquisition module, which is used to capture video data; A data processing module, which is used to perform real-time analysis and optimization on the received video data; A transmission control module, which is used to adjust the video transmission strategy to reduce latency; A feedback processing module, which has the function of dynamically adjusting encoding parameters based on feedback information to further solve the problem of video transmission latency.

2. The front-end AI video computing power terminal device for mines according to claim 1, characterized in that The video encoding module based on deep learning realizes the improvement of the efficiency of the data compression algorithm through the following specific steps: extracting features based on the content difference between video frames, specifically including the following sub-steps: Using a pre-trained neural network model to analyze the pixel distribution characteristics of video frames; Calculating the change amount of pixel points between two consecutive frames and taking it as the quantization change parameter \(D\). Here, the consecutive frames are \(F_{n1}\) and \(F_n\) respectively, that is, the formula is \(D = \|F_n - F_{n1}\|\), which is used to evaluate the similarity between frames in the video stream. \(F_n\) represents the current frame, \(F_{n1}\) represents the previous frame, and \(\|\cdot\|\) is a certain vector norm; Dynamically adjusting the quantization step size in the compression encoding algorithm based on the change parameter \(D\); Performing further encoding and compression operations on the quantized features to reduce the volume of the encoded file.

3. The front-end AI video computing terminal device for mines according to claim 2 is characterized in that: When the above-mentioned video encoding module based on deep learning works, the dynamic adjustment of features further includes the following four specific processes: Analyzing the activity intensity of the intra-frame region, calculating the pixel difference between different regions (such as each small block) within the frame, and setting this as \(A_i\). Here, \(A_i\) refers to the activity degree parameter of the \(i\)-th small block; If it is found that \(A_i\) is greater than a pre-set threshold \(T\), it means that the activity within the block is significant, and it is necessary to strengthen the encoding details to maintain clarity; In the case of \(A_i < T\), the detail retention level within this block can be appropriately reduced to optimize the overall encoding efficiency; Taking differential optimization measures according to the partitions with different activity levels.

4. The front-end AI video computing terminal device for mines according to claim 3 is characterized in that: A more detailed evaluation method is adopted for the calculation of the above-mentioned intra-frame activity intensity: Calculating the gradient magnitude of each pixel value in the frame to measure the motion condition within the region; Based on the difference \(\delta(x,y)=P(x + h,y)-P(x,y)\) obtained at the pixel position \((x,y)\) and the adjacent pixel value \(h\) in the horizontal direction, where \(P\) represents the gray level or color intensity value of the pixel, the vertical direction is evaluated; When the final result \(R(x,y)\) obtained from the average value of \((\delta(x,y))^2 + (\delta(y,x))^2\) is large, it indicates frequent regional movement; otherwise, the movement is weak. The average value of \((\delta(x,y))^2 + (\delta(y,x))^2\) is used to comprehensively evaluate the activity level within the region and is then input as an important part of \(A_i\).

5. The front-end AI video computing power terminal device for mines according to claim 4 is characterized in that: When dynamically adjusting the encoding settings based on inter-frame and intra-region activity, the specific mechanism by which the current environmental factors affect the encoding decision also needs to be considered as follows: Collect information on the current bandwidth and CPU load, and set two variables, the current bandwidth status \(B\) and the load level \(U\), for adjusting the decision-making basis; When using these two dynamic variables to determine the optimal compression ratio, apply the function \(C(B,U)\) to predict the possible impact level during the encoding process; The function \(C\) reflects that the need for a conservative compression strategy increases in an environment with a smaller bandwidth limit \(B\) or a higher processor usage rate \(U\), and vice versa. The adjustment coefficients \(k_1\) and \(k_2\) and their exponential power forms corresponding to the variables \(B\) and \(U\) are used to quantify the impact relationship in the form of \(C(B,U)=k_1B^a\cdot k_2U^b\).

6. The front-end AI video computing power terminal device for mines according to claim 5 is characterized in that: Adjusting the encoding parameters must also perform more fine-grained fine-tuning based on the received feedback information to ensure the best encoding effect: Accept feedback data from end-users, including the QoS parameter \(S\) and the transmission delay \(Td\); Combined with the results of the above \(C\) function, if \(C\) is large but \(S + \alpha Td < Th\), it means that even though the compression coefficient is high, the experience score drops significantly, where \(Th\) is the acceptable threshold and \(\alpha\) is the delay impact weighting.

Images