Video coding, compressing and transmitting method for intelligent equipment

Abstract

The invention provides a video coding and compression transmission method for intelligent equipment, which belongs to the technical field of video coding and compression transmission, realizes advanced channel quality estimation by establishing a channel prediction model enhanced by a spatial transformation network, configures layered coding parameters according to a prediction result and optimizes motion search in combination with inertial measurement unit data. An optimal coding mode is selected by an application rate distortion optimization algorithm, enhancement layer sending is adjusted according to real-time feedback by adopting a dynamic transmission scheduling strategy, and differential forward error correction protection and a time-space domain error concealment mechanism are matched; the technical problem that video transmission is unstable due to violent fluctuation of channel quality when intelligent equipment moves and works in complex electromagnetic environments such as a transformer substation is solved.

Description

Technical Field

[0001] This invention belongs to the field of video encoding, compression and transmission technology, and more specifically, relates to a method for video encoding, compression and transmission of intelligent equipment. Background Technology

[0002] In substation intelligent equipment video monitoring systems, traditional solutions employ fixed encoding parameters for video compression and transmission. The acquired video data is encoded using a standard H.264 or H.265 encoder and then transmitted wirelessly to the monitoring terminal. However, in current intelligent equipment operation scenarios, the equipment frequently moves and inspects within substations, encountering complex electromagnetic interference environments. This causes drastic time-varying characteristics in the received signal strength, signal-to-noise ratio, and packet loss rate of the wireless channel. Traditional fixed-parameter encoding schemes cannot adjust the encoding strategy in real time according to channel conditions. Some existing technologies attempt to dynamically adjust the encoding rate through feedback control mechanisms, but due to feedback delays and insufficient channel prediction capabilities, encoder parameter adjustments lag behind channel quality changes when the equipment moves at high speeds or undergoes frequent attitude changes. This results in severe video transmission stuttering, a sharp drop in image quality, or connection interruptions. In other words, existing technologies suffer from the technical problem of unstable video transmission due to drastic fluctuations in channel quality when intelligent equipment moves and operates in complex electromagnetic environments such as substations. Summary of the Invention

[0003] In view of this, the present invention provides a method for video encoding, compression and transmission of intelligent equipment, which can solve the technical problem in the prior art where unstable video transmission is caused by drastic fluctuations in channel quality when intelligent equipment moves and operates in complex electromagnetic environments such as substations.

[0004] This invention is implemented as follows: The invention provides a method for video encoding, compression, and transmission in intelligent equipment. The intelligent equipment acquires raw video frames through a video acquisition module and motion data from an inertial measurement unit (IMU). The raw video frames and historical channel state information vectors are input into an adaptive channel prediction model to obtain channel quality prediction values. Based on the channel quality prediction values, a coding layer configuration scheme is determined. The raw video frames are encoded into basic layer coding data and enhancement layer coding data. Motion compensation prediction calculations are performed using the IMU motion data to obtain predicted motion vectors. Based on the predicted motion vectors, the inter-frame motion vector search window is narrowed. A fine-search strategy is used for critical macroblocks to obtain their motion vectors, while a coarse-search strategy is used for non-critical macroblocks. A rate-distortion optimization algorithm based on dynamic programming is applied to traverse the model. The coding unit calculates the rate-distortion cost of each coding mode and selects the coding mode with the lowest rate-distortion cost as the optimal coding mode. Based on the optimal coding mode, the coding parameter configuration is determined. Intra-frame refresh macroblocks are periodically inserted into the encoded video stream according to the coding parameter configuration. Syntax element data and discrete cosine transform coefficient data are transmitted separately. Syntax element forward error correction redundancy data is added to the syntax element data, and coefficient forward error correction redundancy data is added to the discrete cosine transform coefficient data. The wireless communication module receives real-time channel quality feedback values ​​and dynamically adjusts the enhancement layer transmission strategy based on these values. When the receiver detects a packet loss event, it uses a spatiotemporal error concealment algorithm to repair damaged macroblocks and obtain repaired macroblocks. Damaged frames containing damaged macroblocks are marked as unusable reference frames, and unusable reference frames are excluded from subsequent coding processes.

[0005] Among them, the adaptive channel prediction model is a lightweight convolutional prediction network used to predict future channel quality change trends based on historical channel state information vectors.

[0006] The input layer of the adaptive channel prediction model receives historical channel state information vectors from the most recent 32 time steps. Each historical channel state information vector contains four normalized features: received signal strength indication, signal-to-noise ratio, bit error rate, and packet loss rate.

[0007] The adaptive channel prediction model includes a spatial transformation layer. The spatial transformation layer uses a geometric correction mechanism based on a spatial transformation network to perform adaptive spatial transformation on the input features. An affine transformation parameter matrix is ​​generated through a localization network, and a sampling network performs bilinear interpolation resampling on the input feature map based on the affine transformation parameter matrix. This automatically corrects feature space geometric deformation and temporal misalignment caused by device movement.

[0008] Among them, the geometric correction mechanism based on spatial transformation network realizes adaptive geometric correction of input features by introducing a learnable spatial transformation module at the front end of the network. The localization network adopts a two-layer fully connected network to learn six affine transformation parameters from the input features. The six affine transformation parameters form a 2×3 affine transformation parameter matrix, which is used to describe the geometric transformations of translation, rotation, scaling and shearing.

[0009] The adaptive channel prediction model also includes a first convolutional layer, a first pooling layer, a second convolutional layer, a fully connected layer, and an output layer. The first convolutional layer contains 16 first convolutional kernels, with a kernel size of 3×3, a kernel stride of 1, a kernel padding of 1, and the activation function of the first convolutional layer is a modified linear unit function.

[0010] The output layer contains 8 output neurons, each corresponding to a channel quality prediction value for the next 8 time steps. The output layer activation function is the Sigmoid function, which normalizes the output to the range of 0 to 1.

[0011] The historical channel state information vector is obtained by the wireless communication module collecting channel state information within a historical time period, which is the past 3.2 seconds.

[0012] The coding layer configuration scheme includes a combination of basic layer quantization parameters and enhancement layer quantization parameters. The basic layer quantization parameter is fixed at 28, while the enhancement layer quantization parameter is dynamically adjusted between 20 and 35 based on the channel quality prediction value.

[0013] The base layer encoded data uses base layer quantization parameters to encode key information, including macroblock type information, motion vector information, reference frame index information, and quantization parameter information. The enhancement layer encoded data uses enhancement layer quantization parameters to encode detail information, including high-frequency discrete cosine transform coefficients and texture detail coefficients.

[0014] The inter-frame motion vector search window range calculates the device's moving speed and direction based on the acceleration and angular velocity data in the inertial measurement unit's motion data, shifts the center of the search window to the predicted motion direction, and linearly scales the search window radius according to the device's moving speed.

[0015] Among them, the key macroblock is a macroblock that contains edge features, texture features and moving object features. The key macroblock is identified by calculating the pixel gradient variance and temporal differential energy within the macroblock. The fine search strategy is to perform a full search within the inter-frame motion vector search window with a step size of 1 pixel.

[0016] The rate-distortion cost calculation steps are as follows: calculate the mean square error between the original macroblock and the reconstructed macroblock to obtain the distortion value; divide the distortion value by the reference distortion value to obtain the normalized distortion value; calculate the number of bits required to encode the macroblock to obtain the code rate value; divide the code rate value by the reference code rate value to obtain the normalized code rate value; determine the Lagrange multiplier value through binary search according to the target code rate constraint; and the rate-distortion cost value is the sum of the normalized distortion value and the product of the Lagrange multiplier value and the normalized code rate value.

[0017] The insertion period of the intra-frame refresh macroblock is dynamically adjusted according to the current packet loss rate. When the current packet loss rate is less than 1%, an intra-frame refresh macroblock is inserted once every 30 frames. When the current packet loss rate is between 1% and 5%, an intra-frame refresh macroblock is inserted once every 15 frames. When the current packet loss rate is greater than 5%, an intra-frame refresh macroblock is inserted once every 5 frames.

[0018] The redundant data for forward error correction of syntax elements is generated using Reed-Solomon codes, with a redundancy ratio of 25%, while the redundant data for forward error correction of coefficients is generated using Reed-Solomon codes, with a redundancy ratio of 10%.

[0019] Specifically, the enhancement layer transmission strategy is dynamically adjusted based on the real-time channel quality feedback value. Specifically, when the real-time channel quality feedback value is lower than the first channel quality threshold, some enhancement layer coding data is discarded. When the real-time channel quality feedback value is lower than the second channel quality threshold, only the base layer coding data is transmitted. The first channel quality threshold is set to 0.6, and the second channel quality threshold is set to 0.3.

[0020] This invention employs an adaptive channel prediction model with an integrated spatial transformation network to proactively predict future channel quality. Based on the prediction results, hierarchical coding parameters are pre-configured, motion vector search is optimized using inertial measurement unit data, and the optimal coding mode is dynamically selected through a rate-distortion optimization algorithm. The spatial transformation network in this scheme automatically corrects the spatial geometric deformation of channel features caused by equipment movement and attitude changes, enabling the convolutional prediction network to continuously extract stable temporal features, ensuring the accuracy of channel quality prediction. The encoder adjusts its parameters accordingly, eliminating the lag inherent in traditional feedback control. Simultaneously, the hierarchical coding strategy, combined with a dynamic transmission scheduling mechanism, prioritizes data transmission of the base layer when channel quality deteriorates. Combined with forward error correction coding and error concealment algorithms, this effectively suppresses the impact of packet loss on video quality. In summary, this invention solves the technical problem mentioned in the background art of unstable video transmission caused by drastic fluctuations in channel quality when intelligent equipment operates in complex electromagnetic environments such as substations. Attached Figure Description

[0021] Figure 1 This is a comparison chart showing the computation time of the motion vector search strategy under different movement states in the embodiment.

[0022] Figure 2 This is a graph showing the change in packet loss rate at different times during the operation process in the example.

[0023] Figure 3 This is a graph showing the change in bit rate over time during the encoding and transmission process in this embodiment. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0025] This invention provides a method for video encoding, compression, and transmission of intelligent equipment, comprising the following steps: S1. The intelligent equipment acquires original video frames through the video acquisition module and acquires motion data of the inertial measurement unit through the inertial measurement unit. The original video frames and historical channel state information vectors are input into the adaptive channel prediction model to obtain the channel quality prediction value. S2. Determine the coding layer configuration scheme based on the channel quality prediction value, and encode the original video frame into basic layer coding data and enhancement layer coding data. The basic layer coding data uses basic layer quantization parameters to encode key information, and the enhancement layer coding data uses enhancement layer quantization parameters to encode detailed information. S3. Using the motion data of the inertial measurement unit, perform motion compensation prediction calculation to obtain the predicted motion vector. Based on the predicted motion vector, narrow the inter-frame motion vector search window range. Use a fine search strategy for key macroblocks to obtain the motion vector of key macroblocks, and use a coarse search strategy for non-key macroblocks to obtain the motion vector of non-key macroblocks. S4. Apply a rate-distortion optimization algorithm based on dynamic programming to traverse all coding modes of the coding unit, calculate the rate-distortion cost of each coding mode, and select the coding mode with the smallest rate-distortion cost as the optimal coding mode. Determine the coding parameter configuration based on the optimal coding mode. S5. According to the encoding parameters, periodically insert intra-frame refresh macroblocks into the encoded video stream, separate the syntax element data and discrete cosine transform coefficient data for transmission, add syntax element forward error correction redundancy data to the syntax element data, and add coefficient forward error correction redundancy data to the discrete cosine transform coefficient data. S6. The wireless communication module receives real-time channel quality feedback values ​​and dynamically adjusts the enhancement layer transmission strategy according to the real-time channel quality feedback values. When the real-time channel quality feedback value is lower than the first channel quality threshold, some of the enhancement layer coded data is discarded. When the real-time channel quality feedback value is lower than the second channel quality threshold, only the base layer coded data is transmitted. S7. When the receiver detects a packet loss event, it enables the spatiotemporal error concealment algorithm to repair the damaged macroblock and obtain the repaired macroblock. The damaged frame containing the damaged macroblock is marked as an unusable reference frame, and the unusable reference frame is excluded in the subsequent encoding process.

[0026] The adaptive channel prediction model is a lightweight convolutional prediction network used to predict future channel quality change trends based on the historical channel state information vector. The specific structure of the adaptive channel prediction model is as follows: the input layer receives the historical channel state information vectors from the most recent 32 time steps. Each historical channel state information vector contains four normalized features: received signal strength indication, signal-to-noise ratio, bit error rate, and packet loss rate. The spatial transformation layer uses a geometric correction mechanism based on a spatial transformation network to adaptively transform the input features. An affine transformation parameter matrix is ​​generated through a positioning network, and a sampling network performs bilinear interpolation resampling on the input feature map based on the affine transformation parameter matrix, automatically correcting feature space geometric deformation and temporal misalignment caused by device movement. The first convolutional layer contains 16 first convolutional kernels, each with a 3×3 size, a stride of 1, and a padding of 1. The activation function is a modified linear unit function (MRU); the first pooling layer uses max pooling with a 2×2 window and a stride of 2; the second convolutional layer contains 32 second convolutional kernels with a 3×3 kernel size, a stride of 1, and padding of 1; the activation function of the second convolutional layer is a modified linear unit function (MRU); the second pooling layer uses max pooling with a 2×2 window and a stride of 2; the fully connected layer contains 64 neurons, and the activation function of the fully connected layer is a modified linear unit function (MRU); the output layer contains 8 output neurons, each corresponding to the predicted channel quality value for the next 8 time steps; the activation function of the output layer is a sigmoid function that normalizes the output to the 0-1 range. The geometric correction mechanism based on spatial transformation networks achieves adaptive geometric correction of input features by introducing a learnable spatial transformation module at the network front end. The localization network uses a two-layer fully connected network to learn six affine transformation parameters from the input features. These six affine transformation parameters form a 2×3 affine transformation parameter matrix, used to describe geometric transformations such as translation, rotation, scaling, and shearing. The sampling network performs coordinate mapping on each position of the original feature map according to the affine transformation parameter matrix, and obtains feature values ​​at the transformed coordinate positions through bilinear interpolation, thereby generating a geometrically corrected feature map. This geometric correction mechanism based on spatial transformation networks gives the model inherent robustness to spatial transformations. When intelligent equipment undergoes posture changes or movement during operation, the channel state information collected by the equipment will produce geometric deformation and temporal misalignment in the feature space. Traditional fixed-structure convolutional networks are difficult to capture feature drift caused by such spatial changes, while spatial transformation networks automatically discover and correct geometric transformation patterns in the input data through end-to-end learning, ensuring that the feature maps received by subsequent convolutional layers maintain spatial consistency.This geometric correction capability significantly improves the accuracy and stability of channel quality prediction, especially in scenarios where intelligent equipment moves at high speeds or frequently turns. The model can continuously output reliable prediction results, providing an accurate basis for subsequent adaptive adjustment of coding parameters. Simultaneously, the geometric correction mechanism based on spatial transformation networks reduces feature extraction errors caused by spatial misalignment, lowers the risk of overfitting the model to certain postures and movement patterns in the training data, and enhances the model's generalization ability under different operating environments and movement trajectories. The introduction of the geometric correction mechanism based on spatial transformation networks enables the entire video coding compression and transmission scheme to maintain the optimal configuration of the coding parameters under the electromagnetic environment of the substation and the dynamic movement of personnel, reducing video stuttering and quality degradation caused by sudden changes in channel quality, and improving the smoothness and reliability of video transmission.

[0027] The steps for establishing the training dataset for the adaptive channel prediction model specifically include: deploying multiple intelligent devices in a substation environment to simulate operations, recording channel state information of operators under different movement speeds, posture changes, and electromagnetic interference intensities; collecting channel state information every 100 milliseconds, including the received signal strength indication value, the signal-to-noise ratio value, the bit error rate value, and the packet loss rate value; continuously collecting data for at least 60 minutes during each operation, covering movement modes such as stationary, slow walking, fast movement, and frequent turning; and organizing the collected channel state information into a sample sequence in chronological order, each... The sample includes historical data from 32 consecutive time steps as input, and channel quality values ​​corresponding to the next 8 time steps as labels. All feature values ​​are normalized: the received signal strength indicator is normalized to the range of -100 to -30, the signal-to-noise ratio is normalized to the range of 0 to 40, and the bit error rate and packet loss rate are normalized to the range of 0 to 1. Abnormal data points and invalid records caused by equipment failures during the acquisition process are removed. The dataset is divided into a training set and a validation set in an 8:2 ratio to ensure that different mobile modes and electromagnetic environment conditions are evenly distributed in the training set and the validation set.

[0028] The specific steps for training the adaptive channel prediction model include: initializing network weights using the Xavier initialization method; using mean squared error as the loss function to measure the difference between the predicted channel quality value and the actual channel quality value; selecting the adaptive moment estimation algorithm as the optimizer, with a learning rate of 0.001, a first momentum parameter of 0.9, and a second momentum parameter of 0.999; setting the batch size to 32 and the number of training epochs to 100; evaluating the model performance on the validation set after each training epoch and recording the validation set loss value; terminating training early when the validation set loss value no longer decreases after 10 consecutive training epochs, and selecting the model weights with the smallest validation set loss value as the final model; employing a data augmentation strategy during training, adding Gaussian noise to the input samples to simulate channel fluctuations, with a noise standard deviation of 0.05; and quantizing and compressing the model after training, converting floating-point weights to 8-bit integer weights to reduce model storage space and inference computation.

[0029] The historical channel state information vector is obtained by the wireless communication module collecting channel state information over a historical time period, which is the past 3.2 seconds. The channel quality prediction value is a normalized value between 0 and 1, where a value closer to 1 indicates better channel quality and a value closer to 0 indicates worse channel quality. The coding layer configuration scheme includes a combination of the base layer quantization parameters and the enhancement layer quantization parameters. The base layer quantization parameter is fixed at 28, and the enhancement layer quantization parameter is dynamically adjusted between 20 and 35 based on the channel quality prediction value. The key information includes macroblock type information, motion vector information, reference frame index information, and quantization parameter information. The detail information includes high-frequency discrete cosine transform coefficients and texture detail coefficients. The inertial measurement unit motion data includes acceleration data and angular velocity data, which are collected by the inertial measurement unit. The inter-frame motion vector search window range is calculated based on the acceleration data and angular velocity data in the inertial measurement unit motion data to determine the device's moving speed and direction. The search window center is shifted to the predicted motion direction, and the search window radius is linearly scaled according to the device's moving speed. The critical macroblock is a macroblock containing edge features, texture features, and moving object features. The critical macroblock is identified by calculating the pixel gradient variance and temporal differential energy within the macroblock. The non-critical macroblock is a macroblock that does not contain the edge features, texture features, and moving object features. The fine-grained search strategy performs a full search within the inter-frame motion vector search window with a step size of 1 pixel. The coarse-grained search strategy performs a diamond search within the inter-frame motion vector search window with a step size of 4 pixels. All encoding modes include intra-frame prediction direction mode, inter-frame partition size mode, and quantization parameter mode.

[0030] The rate-distortion cost calculation steps include: calculating the mean square error between the original macroblock and the reconstructed macroblock to obtain the distortion value; normalizing the distortion value by dividing it by a reference distortion value to obtain a normalized distortion value; calculating the number of bits required to encode the macroblock to obtain the bitrate value; normalizing the bitrate value by dividing it by the reference bitrate value to obtain a normalized bitrate value; determining the Lagrange multiplier value through binary search based on the target bitrate constraint; and finally, the rate-distortion cost value is the sum of the normalized distortion value and the products of the Lagrange multiplier value and the normalized bitrate value. The reference distortion value is a preset standard distortion benchmark value, and the reference bitrate value is a preset standard bitrate benchmark value. The encoding parameter configuration includes the prediction direction parameter, partition size parameter, and quantization parameter corresponding to the optimal encoding mode. The syntax element data includes macroblock type data, motion vector data, reference frame index data, and quantization parameter data. The discrete cosine transform coefficient data is the frequency domain coefficient data of the macroblock after discrete cosine transform and quantization.

[0031] The insertion period of the intra-frame refresh macroblock is dynamically adjusted according to the current packet loss rate. When the current packet loss rate is below 1%, the intra-frame refresh macroblock is inserted once every 30 frames; when the current packet loss rate is between 1% and 5%, the intra-frame refresh macroblock is inserted once every 15 frames; and when the current packet loss rate is above 5%, the intra-frame refresh macroblock is inserted once every 5 frames. The current packet loss rate is obtained by the wireless communication module by calculating the ratio of the number of lost packets to the total number of transmitted data packets in the most recent second. The forward error correction redundancy data of the syntax elements is generated using Reed-Solomon codes, with a redundancy ratio of 25%. The forward error correction redundancy data of the coefficients is also generated using Reed-Solomon codes, with a redundancy ratio of 10%. The real-time channel quality feedback value is obtained by the receiver feeding back to the transmitter through the wireless communication module. The first channel quality threshold is set to 0.6, and the second channel quality threshold is set to 0.3. The first and second channel quality thresholds are determined through offline testing based on the electromagnetic environment of the substation and the working distance. The spatiotemporal error concealment algorithm includes a spatial domain boundary matching algorithm and a temporal domain motion compensation algorithm. For the damaged macroblock, it first attempts to use pixel values ​​from adjacent intact macroblocks for boundary matching interpolation. If the adjacent intact macroblock is also damaged, it uses the macroblock at the corresponding position in the previous frame plus motion vector compensation for repair. Unusable reference frames are excluded from the reference frame list during subsequent encoding processes, and the encoder automatically selects the nearest intact frame as the reference frame to prevent error propagation.

[0032] As an optional implementation, this invention also provides a video encoding and compression transmission system for intelligent equipment implemented using a microcontroller. The microcontroller is communicatively connected to the video acquisition module, the inertial measurement unit, the wireless communication module, and the storage module of the intelligent equipment. The microcontroller has a readable storage medium storing program instructions, which, when executed within the microcontroller, perform the aforementioned video encoding and compression transmission method for intelligent equipment. Preferably, the microcontroller is a 32-bit microcontroller based on the ARM Cortex-M7 architecture. The video acquisition module preferably uses a CMOS image sensor chip. The inertial measurement unit preferably uses a 6-axis microelectromechanical system (MEMS) inertial measurement sensor. The wireless communication module preferably uses an 802.11ac protocol wireless network card chip. The storage module preferably uses a flash memory chip.

[0033] The specific implementation methods of the above steps are described in detail below.

[0034] The specific implementation of step S1 is that the intelligent equipment simultaneously reads motion data from the inertial measurement unit while acquiring the original video frames through the video acquisition module. The video acquisition module continuously acquires the original video frames at a rate of 30 frames per second and caches them in the video buffer. Each frame of video data contains a pixel matrix with a resolution of 1920×1080. The inertial measurement unit acquires triaxial acceleration data and triaxial angular velocity data at a sampling frequency of 100Hz. The wireless communication module records the current channel state parameters, including the received signal strength indication value, signal-to-noise ratio value, bit error rate value, and packet loss rate value, every 100 milliseconds. The most recent 32 time steps, i.e., the past 3...Channel state information within 2 seconds is organized chronologically into a historical channel state information vector. Each time step's channel state information contains four normalized feature values: the received signal strength indicator is normalized to a value range of -100 to -30; the signal-to-noise ratio is normalized to a value range of 0 to 40; and the bit error rate and packet loss rate are normalized to a value range of 0 to 1. The current original video frame and the historical channel state information vector are input together into the input layer of the adaptive channel prediction model. The adaptive channel prediction model first performs geometric correction on the input features through a spatial transformation layer. The positioning network of the spatial transformation layer uses two... The fully connected layer extracts six affine transformation parameters from the input features, forming a 2×3 affine transformation parameter matrix. This matrix describes translation, rotation, scaling, and shearing transformations. The sampling network performs coordinate mapping on each spatial location of the original feature map according to the affine transformation parameter matrix. It then uses bilinear interpolation to obtain feature values ​​at the transformed coordinate locations, generating a geometrically corrected feature map. This spatially corrected feature map is then passed through the first convolutional layer and the first pooling layer. The first convolutional layer contains 16 3×3 kernels with a stride of 1 and padding of 1. After the convolution operation, a modified linear unit activation function is used for non-linearity... The linear transformation begins with a first pooling layer using max pooling with a 2×2 window and a stride of 2 to reduce the feature map resolution. The feature map then passes through a second convolutional layer and a second pooling layer. The second convolutional layer contains 32 3×3 kernels with a stride of 1 and padding of 1, also using the modified linear unit activation function. The second pooling layer uses max pooling with a 2×2 window and a stride of 2. After these two convolutional and pooling operations, the feature map is flattened and input into a fully connected layer containing 64 neurons. The fully connected layer uses the modified linear unit activation function, and the final output layer contains 8 neurons activated by the sigmoid function. The model outputs predicted channel quality values ​​for the next eight time steps. Each predicted value is a normalized value between 0 and 1, with values ​​closer to 1 indicating better channel quality and values ​​closer to 0 indicating worse channel quality. The entire adaptive channel prediction model learns the mapping relationship between historical channel state information and future channel quality through end-to-end training. The introduction of a spatial transformation network enables the model to automatically correct for feature space geometric deformation and temporal misalignment caused by the movement of intelligent equipment, significantly improving the model's prediction accuracy and robustness in the complex electromagnetic environment of substations and under conditions of dynamic personnel movement.

[0035] The specific implementation of step S2 involves determining the coding layer configuration scheme based on the channel quality prediction value obtained in step S1. Specifically, the corresponding coding strategy is selected by judging the numerical range of the channel quality prediction value. The base layer coding always uses a fixed quantization parameter of 28 to ensure the basic quality of video transmission. The base layer coding data mainly includes key structural information such as macroblock type information, motion vector information, reference frame index information, and quantization parameter information. This information is crucial for video decoding. Using a smaller quantization parameter can retain more details and reduce quantization errors. The enhancement layer coding data includes detailed information such as high-frequency discrete cosine transform coefficients and texture detail coefficients. The enhancement layer quantization parameter is dynamically adjusted between 20 and 35 according to the channel quality prediction value. When the channel quality prediction value is higher than 0.8, the enhancement layer quantization parameter is set to 20 to retain the most detail information. When the channel quality prediction value is between 0.5 and 0.8, the enhancement layer quantization parameter is set to 28 to balance quality and bit rate. When the channel quality prediction value is lower than 0.5, the enhancement layer quantization parameter is set to 35 to reduce the coding bit rate and adapt to poor signal strength. Under the given conditions, the original video frame is first divided into 16×16 pixel macroblock units by a macroblock partitioning module. Each macroblock undergoes basic layer coding and enhancement layer coding. The basic layer encoder generates predicted macroblocks using intra-frame prediction or inter-frame prediction, calculates the residual between the original macroblock and the predicted macroblock, and performs discrete cosine transform on the residual to convert spatial domain data into frequency domain coefficients. The frequency domain coefficients are quantized using the basic layer quantization parameter 28. The quantized coefficients are then entropy-encoded to generate basic layer coded data. The enhancement layer encoder further encodes high-frequency coefficients and texture details based on the basic layer coding. These details are quantized and entropy-encoded using the enhancement layer quantization parameter to generate enhancement layer coded data. The advantage of the layered coding scheme is that when the channel quality is good, both basic layer and enhancement layer data can be transmitted simultaneously to obtain high-quality video. When the channel quality deteriorates, some or all enhancement layer data can be discarded, and only basic layer data can be retained to ensure the continuity of video transmission. This adaptive coding strategy adjusts the coding parameters in advance according to the predicted channel quality, effectively responding to the dynamic changes in channel quality in the substation environment.

[0036] The specific implementation of step S3 involves using the acceleration and angular velocity data collected by the inertial measurement unit to perform motion compensation prediction calculations. First, triaxial acceleration data is read from the inertial measurement unit. , , and triaxial angular velocity data , , The acceleration data is integrated over time to calculate the velocity components of the device along the three coordinate axes. The velocity components are then integrated again to calculate the device's displacement. Combined with angular velocity data, the device's rotation angle is calculated. Based on the displacement and rotation angle, the spatial transformation relationship of the camera's viewpoint is deduced. This spatial transformation relationship is mapped onto the image plane to obtain the predicted motion vector. This predicted motion vector reflects the displacement of image content caused by the overall movement of the device. During inter-frame motion estimation, an inter-frame motion vector search window is constructed centered on the predicted motion vector. The center position of the search window is offset according to the direction of the predicted motion vector, and the radius of the search window scales linearly with the device's movement speed. The faster the device moves, the larger the search window radius to cover a wider search range; the slower the device moves, the smaller the search window radius to reduce computation. For each macroblock to be encoded, it is first determined whether it is a critical macroblock. Critical macroblock identification is achieved by calculating the pixel gradient variance and temporal differential energy within the macroblock. The pixel gradient variance reflects the edge and texture features within the macroblock, while the temporal differential energy reflects the moving object features within the macroblock. When the pixel gradient variance exceeds a threshold of 800 or the temporal differential energy exceeds a threshold of 1200, the macroblock is considered critical. Macroblocks are marked as critical macroblocks. A fine-grained search strategy is used for critical macroblocks, performing a full search within the inter-frame motion vector search window with a step size of 1 pixel. The full search traverses every integer pixel position within the search window to calculate the matching cost, and selects the position with the minimum matching cost as the critical macroblock motion vector. For non-critical macroblocks, a coarse-grained search strategy is used, performing a diamond search within the inter-frame motion vector search window with a step size of 4 pixels. The diamond search first calculates the matching cost at diamond pattern positions around the search center, selects the position with the minimum cost as the new search center, and repeats this process until the search center no longer moves. Compared with the full search, the diamond search significantly reduces the number of search points, thereby reducing computational complexity. The principle of using inertial measurement unit data to narrow the search window range for motion compensation prediction is that the overall motion of the device has a similar global motion effect on all macroblocks. Predicting this global motion through inertial measurement unit data can focus the search on the vicinity of the predicted position, avoiding blind searching within the entire reference frame. At the same time, different search strategies are used for critical macroblocks and non-critical macroblocks, reducing the computational overhead of non-critical areas while ensuring the coding quality of critical areas, thus achieving a balance between coding efficiency and coding quality.

[0037] The specific implementation of step S4 involves applying a rate-distortion optimization algorithm based on dynamic programming to traverse all coding modes of the coding unit and select the optimal coding mode. All coding modes of the coding unit include intra-frame prediction direction mode, inter-frame partition size mode, and quantization parameter mode. Intra-frame prediction direction mode covers multiple directions such as horizontal prediction, vertical prediction, DC prediction, and diagonal prediction. Inter-frame partition size mode includes multiple partition sizes such as 16×16, 16×8, 8×16, and 8×8. The quantization parameter mode selects either the base layer quantization parameter or the enhancement layer quantization parameter according to the coding level. For each combination of coding modes, its rate-distortion cost is calculated. The calculation of the rate-distortion cost first involves... The original macroblock is reconstructed through prediction and transform quantization to obtain a reconstructed macroblock. The sum of squares of the differences between each pixel in the original macroblock and the reconstructed macroblock is calculated to obtain the mean square error as the distortion value. The distortion value is normalized by dividing it by a reference distortion value of 256. The number of bits required to encode this macroblock, including the number of bits for prediction mode, motion vectors, residual coefficients, and syntax elements, is calculated to obtain the bitrate value. The bitrate value is normalized by dividing it by a reference bitrate value of 512. Based on the target bitrate constraint, the Lagrange multiplier value is determined through binary search. The binary search repeatedly selects the midpoint value within the candidate interval of the Lagrange multiplier value to calculate the corresponding total bitrate. If the calculation... If the total bitrate is higher than the target bitrate, the Lagrange multiplier value is increased to increase the bitrate penalty; if the calculated total bitrate is lower than the target bitrate, the Lagrange multiplier value is decreased to reduce the bitrate penalty. This process is repeated until the Lagrange multiplier value that makes the total bitrate closest to the target bitrate is found. The rate-distortion cost is calculated as the sum of the normalized distortion value, the Lagrange multiplier value, and the normalized bitrate value. This cost considers both coding distortion and coding bitrate. The Lagrange multiplier value controls the trade-off between distortion and bitrate. The rate-distortion cost is calculated for each combination of coding modes. The coding mode with the smallest rate-distortion cost is selected as the optimal coding mode. The coding parameter configuration is determined based on the optimal coding mode, including... The application of dynamic programming algorithms in the hierarchical decision-making process of coding units is reflected in the prediction of direction parameters, partition size parameters, and quantization parameters. For coding units that can be further divided, the algorithm recursively calculates the optimal coding cost of the sub-coding units after division and the cost of direct coding without division, and selects the scheme with the smaller total cost. This bottom-up decision-making method ensures global optimality. The rate-distortion optimization algorithm based on dynamic programming traverses all possible combinations of coding modes and selects based on rate-distortion cost, minimizing coding distortion under a given bitrate constraint. Compared with traditional heuristic mode selection methods, this algorithm can find the theoretically optimal coding parameter configuration, significantly improving coding efficiency.

[0038] The specific implementation of step S5 involves periodically inserting intra-frame refresh macroblocks into the encoded video stream according to the encoding parameter configuration, and performing separate transmission and forward error correction encoding on the syntax element data and discrete cosine transform coefficient data. The insertion period of the intra-frame refresh macroblock is dynamically adjusted according to the current packet loss rate. The wireless communication module calculates the current packet loss rate by counting the ratio of the number of lost packets to the total number of transmitted data packets in the most recent second. When the current packet loss rate is less than 1%, the video transmission quality is good and the error propagation risk is low, so an intra-frame refresh macroblock is inserted every 30 frames. When the current packet loss rate is between 1% and 5%, the video transmission experiences a certain degree of packet loss, so an intra-frame refresh macroblock is inserted every 15 frames to limit errors. Error propagation range: When the current packet loss rate exceeds 5%, video transmission quality deteriorates and the risk of error propagation increases significantly. Therefore, an intra-frame refresh macroblock is inserted every 5 frames to quickly restore video quality. The intra-frame refresh macroblock is encoded independently of the reference frame using intra-frame coding mode. When a frame at the receiving end experiences packet loss or corruption, the intra-frame refresh macroblock can serve as a new starting point to prevent errors from propagating to subsequent frames. Before transmission, the encoded video stream separates the syntax element data from the discrete cosine transform coefficient data. The syntax element data includes macroblock type data, motion vector data, reference frame index data, and quantization parameter data. These data are crucial for the correctness of video decoding. The discrete cosine transform coefficient data is the result of the macroblock being encoded using the discrete cosine transform... The discrete cosine transform and quantized frequency domain coefficients, which primarily affect the visual quality of the video, are used to generate forward error correction redundancy data for the syntax element data. The redundancy ratio is set to 25%, meaning 1 byte of redundant data is added for every 4 bytes of syntax element data. Reed-Solomon codes are error-correcting codes based on finite field operations, capable of detecting and correcting a certain number of erroneous bytes at the receiver. A 25% redundancy ratio can correct up to 12.5% ​​of errors. Similarly, Reed-Solomon codes are used to generate forward error correction redundancy data for the discrete cosine transform coefficient data, with a redundancy ratio set to 10%, meaning 1 byte of redundant data is added for every 10 bytes of coefficient data. Since the coefficient data is crucial for decoding... The impact of accuracy is relatively small compared to the syntax element data, but the data volume is large. Therefore, a low redundancy ratio is adopted to control the total bit rate. The syntax element data and discrete cosine transform coefficient data, along with their respective forward error correction redundancy data, are encapsulated into independent data packets for transmission. The advantage of separate transmission is that differentiated protection strategies can be adopted for data of different importance. Stronger error correction capability is provided for critical syntax element data, while moderate error correction capability is provided for secondary coefficient data. Redundancy overhead is reduced while ensuring decoding correctness. Forward error correction technology enables the receiver to autonomously correct transmission errors without requesting retransmission, reducing retransmission delay and improving the real-time performance of video transmission.

[0039] The specific implementation of step S6 involves the wireless communication module receiving a real-time channel quality feedback value and dynamically adjusting the enhancement layer transmission strategy based on this feedback value. The receiving end periodically feeds back the current channel quality status to the transmitting end through the wireless communication module. The calculation method for the real-time channel quality feedback value is the same as that for the channel quality prediction value in step S1, both being normalized values ​​between 0 and 1. After receiving the real-time channel quality feedback value, the transmitting end compares it with a preset channel quality threshold to determine the transmission strategy. The first channel quality threshold is set to 0.6 to represent a medium channel quality level, and the second channel quality threshold is set to 0.3 to represent a poor channel quality level. These two thresholds are determined through offline testing in a substation environment. During the test, the video transmission performance under different channel quality levels is recorded, including stuttering rate, average latency, and video quality score. The channel quality value that makes the performance indicators reach an acceptable level is selected as the threshold. When the real-time channel quality feedback value is higher than the first channel quality threshold of 0.6, the channel condition is good, and the transmitting end simultaneously transmits the base layer coded data and all enhancement layer coded data to provide the highest video quality. When the real-time channel quality feedback value is lower than the first channel quality threshold of 0.6 but higher than the first channel quality threshold of 0.6, the channel condition is good, and the transmitting end transmits the base layer coded data and all enhancement layer coded data simultaneously to provide the highest video quality. When the second channel quality threshold is 0.3, the channel condition is generally good. The transmitter discards some enhancement layer coded data and only transmits the basic layer coded data and high-priority enhancement layer coded data. The high-priority enhancement layer coded data includes low-frequency discrete cosine transform coefficients and key texture coefficients. These coefficients have a significant impact on video quality. By discarding secondary enhancement layer data such as high-frequency coefficients, the transmission bitrate can be reduced to adapt to the current channel capacity. When the real-time channel quality feedback value is lower than the second channel quality threshold of 0.3, the channel condition is poor. The transmitter only transmits the basic layer coded data and completely discards the enhancement layer coded data. The basic layer coded data contains the minimum information required to decode the video, ensuring that the video transmission can still maintain basic continuity and watchability even under very poor channel quality. The principle of dynamically adjusting the enhancement layer transmission strategy is to adaptively change the transmission content according to the real-time channel feedback. When the channel capacity is sufficient, the complete data is transmitted to obtain high quality. When the channel capacity is limited, the transmission of key data is prioritized, sacrificing some video quality to maintain transmission stability. This adaptive transmission strategy can better cope with the rapid fluctuations in channel quality in the substation environment and reduce video stuttering and interruption.

[0040] The specific implementation of step S7 is as follows: When the receiving end detects a packet loss event, it activates a spatiotemporal error concealment algorithm to repair the damaged macroblock. The receiving end identifies the lost data packet by checking the data packet sequence number, and determines the damaged macroblock based on the frame to which the lost data packet belongs and the macroblock location. The spatiotemporal error concealment algorithm first attempts to use a spatial domain boundary matching algorithm for repair. The spatial domain boundary matching algorithm extracts the boundary pixel values ​​from the adjacent intact macroblocks of the damaged macroblock, calculates the matching error between each adjacent macroblock and the boundary of the damaged macroblock, selects the adjacent macroblock with the smallest boundary matching error, and then uses the internal pixel values ​​of that adjacent macroblock to repair the damaged macroblock. The algorithm fills damaged macroblocks with interpolation using bilinear or bicubic interpolation to ensure a smooth transition between the generated pixel values ​​and surrounding pixels. If adjacent macroblocks are also damaged and spatial domain repair is not possible, the algorithm switches to temporal motion compensation. This algorithm extracts a macroblock from the previous frame corresponding to the damaged macroblock as a reference macroblock. If motion vector information for this macroblock exists, motion vector compensation is applied to the reference macroblock to obtain a predicted macroblock for the current frame, which replaces the damaged macroblock. If no motion vector information exists, the macroblock from the previous frame is directly copied. The spatiotemporal error concealment algorithm repairs damaged macroblocks by comprehensively utilizing spatial and temporal correlations. Although the repaired macroblocks cannot completely restore the original content, they can maintain the visual continuity of the video and reduce abruptness during viewing. After the repair process is completed, the receiver marks the damaged frame containing the damaged macroblock as an unusable reference frame. Unusable reference frames are excluded from the reference frame list in subsequent encoding processes. When performing inter-frame prediction, the encoder automatically skips unusable reference frames and selects the nearest intact frame as the reference frame. The purpose of this mechanism is to avoid error propagation. Since the repaired macroblock in the damaged frame is not the actual original content, if subsequent frames use the damaged frame as a reference for motion compensation prediction, the repair error will propagate to subsequent frames, leading to error accumulation. By excluding unusable reference frames, the error propagation path can be cut off, ensuring that the encoding quality of subsequent frames is not affected by the damaged frame. When a new intra-frame refresh macroblock or a complete intra-frame encoded frame is received, the error concealment state is reset, and the encoder resumes the normal reference frame selection strategy. The combination of the spatiotemporal error concealment algorithm and the reference frame management mechanism effectively improves the error resistance and viewing experience of video transmission in packet loss environments.

[0041] It should be noted that the key technical ideas of this invention include adaptive channel prediction based on spatial transformation networks, motion compensation prediction based on inertial measurement units, and hierarchical coding and dynamic transmission strategies. Adaptive channel prediction based on spatial transformation networks introduces a learnable geometric correction module at the front end of the convolutional neural network, enabling the model to automatically identify and correct feature space deformation and temporal misalignment caused by the movement of intelligent equipment. Traditional fixed-structure convolutional networks experience a decrease in feature extraction capability and prediction accuracy when the device's posture changes. However, spatial transformation networks adaptively adjust the spatial alignment of feature maps through end-to-end learning, ensuring that subsequent convolutional layers always receive spatially consistent feature inputs. This significantly improves the model's prediction robustness and generalization ability in dynamic moving scenarios, providing a reliable basis for the forward-looking adjustment of coding parameters. Motion compensation prediction based on inertial measurement units uses data from the device's own motion sensors to predict the global motion trend of image content, focusing the motion estimation search on the vicinity of the prediction location. Compared to traditional global search methods, this significantly reduces the search range and computational complexity. Furthermore, by distinguishing between critical and non-critical macroblocks and employing differentiated search strategies, the computational overhead of non-critical regions is further reduced while ensuring the coding quality of important regions, achieving an optimized balance between coding efficiency and coding quality. The layered coding and dynamic transmission strategy divides video data into a base layer and an enhancement layer. It adaptively selects the transmission content based on real-time channel quality feedback. Compared to fixed-rate transmission, this approach allows for flexible adjustment of the transmission strategy when channel quality fluctuates. When channel quality is good, it transmits complete data to provide high-quality video; when channel quality deteriorates, it prioritizes base layer data transmission to maintain video continuity, avoiding transmission interruptions and video stuttering caused by channel congestion. The synergistic effect of these three key technologies is reflected in the spatial transformation network providing accurate channel quality predictions, which serve as a basis for decision-making regarding coding parameter adjustment and transmission strategy selection. Motion compensation prediction based on inertial measurement units reduces coding computational complexity, enabling the system to operate in real-time on resource-constrained intelligent equipment. The layered coding and dynamic transmission strategy flexibly adjusts the transmission content based on predicted channel quality and real-time feedback. Together, these three elements constitute a complete closed-loop system from channel prediction to coding optimization to transmission adaptation. Compared to traditional methods, this significantly improves the stability, smoothness, and reliability of video transmission in the complex electromagnetic environment of substations and under conditions of dynamic personnel movement.

[0042] It should be noted that this invention also solves the following technical problem: the problem of increased encoding latency due to excessive computational complexity in motion vector search for intelligent equipment in high-speed moving scenarios. Traditional video encoders use a fixed-size search window to perform a full search algorithm with the same precision on all macroblocks. When the equipment moves rapidly, the search range needs to be expanded to cover large motion vectors, resulting in a quadratic increase in the number of search points and a significant increase in encoder processing latency, making it difficult to meet real-time transmission requirements. This invention uses acceleration and angular velocity data from the inertial measurement unit to calculate the equipment's moving speed and direction, shifts the center of the search window to the predicted motion direction, and dynamically adjusts the window radius according to the moving speed, effectively narrowing the search range. At the same time, it identifies key macroblocks containing important visual features by calculating macroblock gradient variance and temporal differential energy. A fine full search with a 1-pixel step size is used for key macroblocks to ensure encoding quality, while a coarse diamond-shaped search with a 4-pixel step size is used for non-key macroblocks in flat areas to reduce computation. This differentiated search strategy significantly reduces the computational complexity of motion estimation while ensuring the encoding accuracy of key areas, enabling the encoder to maintain low processing latency even when the equipment moves at high speed, meeting the timeliness requirements of real-time video transmission.

[0043] Furthermore, this invention also solves the technical problem of video decoding error propagation caused by packet loss in wireless channels. Traditional video coding uses inter-frame prediction technology to improve compression efficiency. However, damage to the reference frame can propagate to subsequent frames through motion compensation mechanisms, causing large-area image distortion that is difficult to recover. This invention dynamically adjusts the insertion period of intra-frame refresh macroblocks according to the current packet loss rate. When the packet loss rate is low, it inserts once every 30 frames; when the packet loss rate rises to 1% to 5%, it shortens to once every 15 frames; and when the packet loss rate exceeds 5%, it further shortens to once every 5 frames. This ensures that even if packet loss occurs, error propagation can be blocked within a limited number of frames using intra-frame coded macroblocks. At the same time, it separates the syntax element data and discrete cosine transform coefficient data for transmission and applies differentiated forward error correction protection. It adds 25% redundancy to the syntax element data carrying key structural information such as macroblock type and motion vectors. De Solomon coding adds only 10% redundancy to high-frequency coefficient data, prioritizing the correct parsing of the video structure by the decoder. Combined with a spatiotemporal error concealment algorithm, when a damaged macroblock is detected, boundary matching interpolation is performed using the pixel values ​​of adjacent intact macroblocks, or motion vector compensation is added to the corresponding position in the previous frame for repair. Frames containing damaged macroblocks are marked as unusable reference frames and excluded from the subsequent encoding reference list. This multi-layered protection mechanism effectively limits the spread of decoding errors caused by packet loss in the temporal dimension, significantly improving the robustness of video transmission in high packet loss rate channel environments.

[0044] Specifically, the principle of this invention is as follows: The fundamental reason why this invention can solve the technical problem lies in establishing a complete closed-loop control mechanism from channel state prediction to coding parameter configuration. The spatial transformation network uses a learnable affine transformation parameter matrix to geometrically correct historical channel state characteristics, eliminating feature space distortion caused by equipment movement. This enables the lightweight convolutional network to accurately capture the changing trend of channel quality from aligned time-series data, achieving reliable prediction of the channel state for the next eight time steps. The encoder adjusts the quantization parameters and hierarchical configuration in advance based on the predicted values. Combined with motion information provided by the inertial measurement unit, it narrows the search window and distinguishes between critical and non-critical macroblocks using a differentiated search strategy. This ensures coding accuracy in critical areas while reducing computational complexity. The rate-distortion optimization algorithm traverses all coding modes and unifies the evaluation criteria through normalization, ensuring that the selected coding mode achieves optimal quality and rate balance under the current channel constraints. The dynamic transmission scheduling module flexibly discards enhancement layer data or switches to basic layer-only transmission mode based on real-time feedback. Combined with the separate transmission of syntax elements and transform coefficients and differentiated forward error correction protection, it ensures the reliable delivery of critical information to the greatest extent. Even if packet loss occurs, it can be quickly repaired and error propagation blocked through the spatiotemporal domain error hiding algorithm. The synergistic effect of the entire mechanism enables the video transmission system to maintain stable transmission performance even when the channel quality fluctuates drastically.

[0045] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0046] The specific implementation of step S1 is as follows: the intelligent equipment acquires original video frames through the video acquisition module and collects motion data of the inertial measurement unit through the inertial measurement unit. The original video frames and historical channel state information vectors are input into the adaptive channel prediction model to obtain channel quality prediction values. The historical channel state information vector is obtained by the wireless communication module collecting channel state information within a historical time period, which is the past 3.2 seconds. The channel state information at each time step includes four normalized features: received signal strength indication value, signal-to-noise ratio value, bit error rate value, and packet loss rate value. The normalization formula for the received signal strength indication value is expressed as follows: ; In the formula, This is the normalized received signal strength indication value, ranging from 0 to 1; This is the original received signal strength indication value, in units of ; This is the lower limit of the received signal strength indication value, in units of... The value is -100. ; This is the upper limit of the received signal strength indicator value, in units of The value is -30 The normalization formula for the signal-to-noise ratio is expressed as follows: ; In the formula, This is the normalized signal-to-noise ratio value, ranging from 0 to 1; This is the original signal-to-noise ratio value, in units of ; For reference signal-to-noise ratio values, the unit is... The value is 40. The bit error rate and packet loss rate values ​​themselves range from 0 to 1, requiring no further normalization. In the adaptive channel prediction model, the spatial transform layer generates an affine transform parameter matrix through a localization network. The affine transform parameter matrix is ​​described as follows: ; In the formula, Let be the affine transformation parameter matrix, which is a 2x3 matrix; The rotation, scaling, and shearing parameters are dimensionless, with empirical values ​​ranging from 0.8 to 1.2. The translation parameter is in pixels, with an empirical range of -50 to 50 pixels. The localization network uses a two-layer fully connected structure to learn the six affine transformation parameters from the input features. The sampling network performs coordinate mapping on each position of the original feature map according to the affine transformation parameter matrix. The formula for calculating the transformed coordinate position is as follows: ; In the formula, This represents the x-coordinate position in the original feature map, in pixels; This represents the ordinate position in the original feature map, in pixels; This represents the transformed x-coordinate position, in pixels. The vertical coordinate is the transformed position in pixels. The sampling network obtains feature values ​​at the transformed coordinate position through bilinear interpolation to generate a geometrically corrected feature map. The adaptive channel prediction model outputs channel quality prediction values ​​for the next 8 time steps. Each prediction value is a normalized value between 0 and 1. The closer the value is to 1, the better the channel quality, and the closer the value is to 0, the worse the channel quality.

[0047] The specific implementation of step S2 is to determine the coding layer configuration scheme based on the channel quality prediction value. The quantization parameter of the base layer is fixed at 28, and the quantization parameter of the enhancement layer is dynamically adjusted according to the channel quality prediction value. The adjustment formula is expressed as follows: ; In the formula, The quantization parameters for the enhancement layer range from 20 to 35. The maximum quantization parameter is set to 35. The minimum quantization parameter is set to 20. This is the predicted channel quality value, ranging from 0 to 1; The normalization coefficient is 1. The original video frame is divided into 16×16 pixel macroblock units by the macroblock partitioning module. Each macroblock is encoded using a base layer and an enhancement layer. The base layer encoded data uses base layer quantization parameters to encode key information, and the enhancement layer encoded data uses enhancement layer quantization parameters to encode detail information.

[0048] The specific implementation of step S3 involves using the motion data from the inertial measurement unit to perform motion compensation prediction calculations to obtain the predicted motion vector. First, the device velocity component is calculated based on the triaxial acceleration data collected by the inertial measurement unit. The velocity component calculation formula is expressed as follows: ; ; ; In the formula, for time Normalized velocity components along the coordinate axes, dimensionless; for time Normalized velocity components along the coordinate axes, dimensionless; for time Normalized velocity components along the coordinate axes, dimensionless; for time Normalized velocity components along the coordinate axes, dimensionless; for time Normalized velocity components along the coordinate axes, dimensionless; for time Normalized velocity components along the coordinate axes, dimensionless; for time Axis acceleration data, in units of ; for time Axis acceleration data, in units of ; for time Axis acceleration data, in units of ; The sampling time interval is in seconds and has a value of 0.01 seconds. For reference speed, the unit is... The value is 10. The displacement of the equipment is calculated based on the velocity component. The formula for calculating the displacement is as follows: ; ; In the formula, for Time-image plane Normalized displacement in direction, dimensionless; for Time-image plane Normalized displacement in direction, dimensionless; for Time-image plane The normalized displacement in the direction, dimensionless; for Time-image plane The normalized displacement in the direction, dimensionless; The reference displacement is in meters, with a value of 1 meter. The predicted motion vector is obtained by mapping the displacement onto the image plane. The formula for calculating the predicted motion vector is as follows: ; In the formula, The predicted motion vector is a 2x1 column vector; To predict the lateral component of the motion vector, in pixels; The vertical component of the predicted motion vector is shown in pixels. The focal length of the camera on the image plane The projection of the direction, in pixels, with an empirical value of 1000 pixels; The focal length of the camera on the image plane The projection of the direction, in pixels, is empirically set to 1000 pixels. The search window radius scales linearly with the device's movement speed; the formula for calculating the search window radius is as follows: ; In the formula, The radius of the search window, in pixels; The radius of the basic search window is in pixels and has a value of 16 pixels. The scaling factor is dimensionless and has a value of 2. Normalized velocity, unit: The value is 5 Key macroblock identification is achieved by calculating the pixel gradient variance and temporal difference energy within the macroblock. The formula for calculating the pixel gradient variance is as follows: ; In the formula, The normalized pixel gradient variance is dimensionless. This represents the total number of pixels within the macroblock, with a value of 256. For the first The gradient magnitude of each pixel, in the same unit as the pixel value; This is the average gradient magnitude of all pixels within the macroblock, with the same unit as the pixel value. The calculation formula is: ; The reference gradient magnitude, with the same unit as the pixel value, is set to 100. The formula for calculating the temporal domain differential energy is as follows: ; In the formula, The normalized time-domain difference energy is dimensionless. For the current frame number The grayscale value of each pixel; The position corresponding to the previous frame The grayscale value of each pixel; This is a reference difference value, with the same unit as the pixel value, and a value of 50. When... Exceeding the normalization threshold of 8 or When the normalization threshold of 12 is exceeded, the macroblock is marked as a critical macroblock. For critical macroblocks, a fine search strategy is used to perform a full search within the inter-frame motion vector search window with a step size of 1 pixel. For non-critical macroblocks, a coarse search strategy is used to perform a diamond search within the inter-frame motion vector search window with a step size of 4 pixels.

[0049] The specific implementation of step S4 is to apply a rate-distortion optimization algorithm based on dynamic programming to traverse all coding modes of the coding unit. The rate-distortion cost calculation first calculates the mean square error between the original macroblock and the reconstructed macroblock to obtain the distortion value. The distortion value calculation formula is expressed as follows: ; In the formula, This represents the mean squared error distortion value. For the original macroblock Each pixel value; To rebuild macroblock number Each pixel value. The formula for calculating the normalized distortion value is as follows: ; In the formula, The normalized distortion value is dimensionless. For reference distortion values, the units are... The value is the same, taking the value of 256. The encoding rate is obtained by statistically analyzing the number of bits required for encoding. The formula for calculating the normalized rate is as follows: ; In the formula, This is the normalized bitrate value, which is dimensionless. The number of bits required to encode a macroblock, in bits; This is a reference bitrate value, in bits, with a value of 512 bits. The rate-distortion cost calculation formula is as follows: ; In the formula, The value of the rate distortion is dimensionless; The Lagrange multiplier value is dimensionless and determined by binary search based on the target bit rate constraint; its empirical range is 0.1 to 10. The rate-distortion cost is calculated for each combination of coding modes, and then selected... The smallest encoding pattern is taken as the optimal encoding pattern. The encoding parameter configuration, including prediction direction parameter, partition size parameter, and quantization parameter, is determined based on the optimal encoding pattern.

[0050] The specific implementation of step S5 involves periodically inserting intra-frame refresh macroblocks into the encoded video stream according to the encoding parameters. The formula for calculating the current packet loss rate is as follows: ; In the formula, The current packet loss rate is dimensionless. This represents the number of packets lost in the last second, in packets. This represents the total number of data packets sent in the last second, expressed in packets. The intra-frame refresh macroblock insertion period is dynamically adjusted based on the current packet loss rate. When the value is below 0.01, the intra-frame refresh macroblock is inserted once every 30 frames. When the value is between 0.01 and 0.05, the intra-frame refresh macroblock is inserted once every 15 frames. When the value is higher than 0.05, the intra-frame refresh macroblock is inserted once every 5 frames. The syntax element data is generated by using Reed-Solomon codes to generate syntax element forward error correction redundancy data with a redundancy ratio of 25%. The discrete cosine transform coefficient data is generated by using Reed-Solomon codes to generate coefficient forward error correction redundancy data with a redundancy ratio of 10%.

[0051] The specific implementation of step S6 is that the wireless communication module receives the real-time channel quality feedback value and dynamically adjusts the enhancement layer transmission strategy according to the real-time channel quality feedback value. When the real-time channel quality feedback value is lower than the first channel quality threshold, part of the enhancement layer coded data is discarded. When the real-time channel quality feedback value is lower than the second channel quality threshold, only the base layer coded data is transmitted. The first channel quality threshold is set to 0.6 and the second channel quality threshold is set to 0.3.

[0052] The specific implementation of step S7 is as follows: when the receiving end detects a packet loss event, it enables the spatiotemporal domain error concealment algorithm to repair the damaged macroblock, and the spatial domain boundary matching algorithm calculates the boundary matching error between each adjacent macroblock and the damaged macroblock. The formula for calculating the boundary matching error is as follows: ; In the formula, The normalized boundary matching error is dimensionless. This represents the number of boundary pixels, with a value of 16. To damage macroblock boundaries Each pixel value; For the boundary of adjacent macroblocks Each pixel value; This is a reference pixel value, in the same unit as the pixel value, and is set to 128. Select. The smallest adjacent macroblock is interpolated and filled. If the adjacent macroblock is also damaged, the temporal motion compensation algorithm is used. The temporal motion compensation is based on the macroblock at the corresponding position in the previous frame plus motion vector compensation for repair. The repaired macroblock marks the damaged frame containing the damaged macroblock as an unusable reference frame. The unusable reference frame is excluded in the subsequent encoding process to avoid error propagation.

[0053] To better understand and implement this invention, a specific application scenario of the invention is provided below as Example 2: To verify the effectiveness of the invention, technicians built a numerical simulation analysis environment to simulate an internal inspection operation scenario of the main equipment in an ultra-high voltage substation. Real-time video acquisition and transmission were performed using an intelligent safety helmet. The intelligent safety helmet integrates a high-definition camera, a 6-axis inertial measurement unit, an 802.11ac wireless communication module, and a lighting laser system. Workers wearing this equipment performed inspection tasks inside a 500kV GIS switchgear. The simulation environment was set with an electromagnetic interference intensity of 85 to 120 volts per meter within the substation. The work area covered a GIS equipment compartment with a length of 45 meters and a width of 32 meters. The wireless access point was deployed in the center of the equipment compartment, with a signal coverage radius of 30 meters.

[0054] Technicians first established a training dataset for the adaptive channel prediction model. In a simulation environment, they simulated workers moving at different speeds, including a stationary observation phase lasting 180 to 300 seconds, a slow walking speed of 0.3 to 0.8 meters per second, a normal walking speed of 0.9 to 1.5 meters per second, and a fast moving speed of 1.6 to 2.2 meters per second. During the data acquisition process, channel state information, including received signal strength indication, signal-to-noise ratio, bit error rate, and packet loss rate, was recorded every 100 milliseconds. Channel data was continuously collected for 12 consecutive work tasks, each lasting 60 to 90 minutes, resulting in a total of 58,420 samples, covering the channel characteristics of workers under different movement patterns and posture changes. The statistical data of the channel state information are shown in Table 1.

[0055] Table 1 Channel State Information Statistics

[0056] The training dataset was divided into a training set of 46,736 samples and a validation set of 11,684 samples in an 8:2 ratio. The network weights were initialized using the Xavier initialization method, with mean squared error as the loss function. The optimizer employed the adaptive moment estimation algorithm, with a learning rate of 0.001, a first momentum parameter of 0.9, and a second momentum parameter of 0.999. The batch size was set to 32, and the number of training epochs was set to 100. During training, the validation set loss reached its minimum of 0.0042 in the 68th epoch. Subsequently, the validation set loss did not decrease for the next 10 epochs, triggering an early termination mechanism. The model weights from the 68th epoch were ultimately selected as the adaptive channel prediction model. After training, the model was quantized and compressed, converting floating-point weights to 8-bit integer weights. The model file size was compressed from 2.3MB to 0.6MB, and the inference computation time was reduced from 12 milliseconds to 3 milliseconds.

[0057] Technicians simulated a typical inspection scenario. The operator entered from the equipment compartment entrance, first performing a visual inspection of the equipment from a stationary position for 120 seconds. Then, they moved longitudinally along the equipment at a speed of 0.5 meters per second for detailed observation for 180 seconds. Next, they moved laterally to the other side of the equipment at a speed of 1.2 meters per second for 90 seconds. Finally, they returned to the entrance position at a speed of 0.8 meters per second for 150 seconds. The entire operation lasted 540 seconds. The smart safety helmet camera captured video at a frame rate of 30 frames per second, with a resolution of 1920×1080 pixels, accumulating 16,200 raw video frames. The inertial measurement unit simultaneously collected acceleration and angular velocity data at a sampling frequency of 100Hz, accumulating 54,000 sets of inertial measurement data.

[0058] The adaptive channel prediction model predicts channel quality in real time for the next eight time steps based on historical channel state information vectors. Technicians extracted the predicted channel quality values ​​at four typical moments during the operation and compared them with the actual measured values, as shown in Table 2.

[0059] Table 2 Comparison of predicted and actual channel quality values

[0060] The coding layer configuration scheme is determined based on the channel quality prediction value. The base layer quantization parameter is fixed at 28, while the enhancement layer quantization parameter is dynamically adjusted according to the channel quality prediction value. When the channel quality prediction value is 0.88, the enhancement layer quantization parameter is set to 20, at which point the enhancement layer coded data contains rich high-frequency detail information. When the channel quality prediction value decreases to 0.62, the enhancement layer quantization parameter is adjusted to 26, appropriately reducing the amount of detail information encoded. When the channel quality prediction value decreases to 0.35, the enhancement layer quantization parameter is adjusted to 32, significantly reducing the amount of enhancement layer coded data. When the channel quality prediction value rises back to 0.58, the enhancement layer quantization parameter is adjusted to 27, restoring some detail information encoding.

[0061] Technicians used motion data from the inertial measurement unit (IMU) to perform motion compensation prediction calculations. At the 300-second mark, the worker moved laterally at a speed of 1.2 meters per second. The IMU measured a lateral acceleration of 0.15 meters per second squared, a longitudinal acceleration of 0.02 meters per second squared, and a yaw rate of 0.08 radians per second. Based on these data, a predicted motion vector was calculated, with a lateral component of 38 pixels and a longitudinal component of 5 pixels. The center of the inter-frame motion vector search window was shifted to the predicted motion direction, and the search window radius was linearly scaled to 64 pixels according to the device's movement speed. A fine-grained search strategy was used for critical macroblocks containing edge and texture features, performing a full search with a step size of 1 pixel to obtain the motion vectors of the critical macroblocks. A coarse-grained search strategy was used for non-critical macroblocks in flat areas, performing a diamond search with a step size of 4 pixels to obtain the motion vectors of the non-critical macroblocks. The computation time of the motion vector search strategy under different movement states is as follows: Figure 1 As shown.

[0062] Technicians applied a rate-distortion optimization algorithm based on dynamic programming to traverse all coding modes of a coding unit. For a 16×16 macroblock, they traversed 9 intra-frame prediction direction modes, 7 inter-frame partition size modes, and 5 quantization parameter modes, resulting in a total of 315 coding mode combinations. The rate-distortion cost of each coding mode was calculated, and the coding mode with the lowest rate-distortion cost was selected as the optimal coding mode. In a critical macroblock at time 300, after traversing all coding modes, the intra-frame vertical prediction mode combined with a 16×8 partition size and a quantization parameter of 26 yielded the lowest rate-distortion cost of 1.32, and therefore this combination was selected as the optimal coding mode. The coding parameter configuration, including prediction direction parameters, partition size parameters, and quantization parameters, was determined based on the optimal coding mode.

[0063] Intra-frame refresh macroblocks are periodically inserted into the encoded video stream, with the insertion period dynamically adjusted based on the current packet loss rate. Technicians statistically analyze the changes in packet loss rate at different time points during the simulation, such as... Figure 2 As shown. During the stationary observation phase (0-120 seconds), the packet loss rate remained between 0.002 and 0.003, with an intra-frame refresh macroblock inserted every 30 frames. During the slow-moving phase (120-300 seconds), the packet loss rate increased to between 0.010 and 0.015, with an intra-frame refresh macroblock inserted every 15 frames. During the normal-moving phase (300-390 seconds), the packet loss rate increased to between 0.025 and 0.038, with an intra-frame refresh macroblock inserted every 15 frames. During the fast-moving phase (390-480 seconds), the packet loss rate increased sharply to between 0.060 and 0.092, with an intra-frame refresh macroblock inserted every 5 frames. During the slow-moving return phase (480-540 seconds), the packet loss rate dropped back to between 0.012 and 0.018, with an intra-frame refresh macroblock inserted every 15 frames.

[0064] Technicians transmit syntax element data and discrete cosine transform coefficient data separately. Forward error correction redundancy data generated by Reed-Solomon codes is added to the syntax element data, with a redundancy ratio of 25%. Forward error correction redundancy data generated by Reed-Solomon codes is added to the discrete cosine transform coefficient data, with a redundancy ratio of 10%. The wireless communication module receives real-time channel quality feedback values ​​from the receiver and dynamically adjusts the enhancement layer transmission strategy based on these values. At 390 seconds, the real-time channel quality feedback value drops to 0.55, below the first channel quality threshold of 0.6, and the system automatically discards 20% of the enhancement layer coded data, prioritizing the transmission of basic layer coded data. At 420 seconds, the real-time channel quality feedback value further decreases to 0.28, below the second channel quality threshold of 0.3, and the system stops transmitting all enhancement layer coded data, transmitting only basic layer coded data. After 480 seconds, the real-time channel quality feedback value rises back to 0.58, and the system resumes transmitting enhancement layer coded data.

[0065] At 405 seconds, the receiver detected three consecutive packet loss events, involving five corrupted macroblocks. The system employed a spatiotemporal error concealment algorithm to repair the corrupted macroblocks. First, it attempted boundary matching interpolation using pixel values ​​from adjacent intact macroblocks. If adjacent macroblocks were also corrupted, it used the macroblock at the corresponding position in the previous frame plus motion vector compensation for repair. Of the five corrupted macroblocks, three were successfully repaired using the spatial domain boundary matching algorithm, and two were repaired using the temporal domain motion compensation algorithm. The system marked frame 12150, containing the corrupted macroblocks, as an unusable reference frame, excluding it in subsequent encoding processes and automatically selecting frame 12148 as the reference frame to prevent error propagation. The total number of unusable reference frames during the entire operation was 18, accounting for 0.11% of the total frames. The bitrate changes during encoding and transmission are as follows: Figure 3 As shown.

[0066] Technical personnel analyzed the simulation test results and found that the adaptive channel prediction model adopted in this invention, by introducing a geometric correction mechanism based on a spatial transformation network, can automatically correct feature space geometric deformation and temporal misalignment caused by equipment movement. This allows the model to continuously output reliable channel quality prediction results even in scenarios involving high-speed movement or frequent turning of personnel, providing an accurate basis for adaptive adjustment of coding parameters. Traditional fixed-structure convolutional networks struggle to capture feature drift caused by spatial changes, leading to a significant increase in channel quality prediction errors during rapid movement, which in turn causes improper coding parameter configuration and video quality degradation. This invention automatically discovers and corrects geometric transformation patterns in the input data through end-to-end learning of the spatial transformation network, ensuring spatial consistency of the feature maps received by subsequent convolutional layers and significantly improving the accuracy and stability of channel quality prediction. Simultaneously, the layered coding strategy employed in this invention encodes video into a base layer and an enhancement layer. The transmission strategy of the enhancement layer is dynamically adjusted based on real-time channel quality feedback values. When channel quality is poor, priority is given to transmitting the base layer; when channel quality is good, the enhancement layer is transmitted to improve video quality, achieving adaptive matching between bitrate and channel conditions. Traditional fixed-rate coding schemes cannot cope with the drastic fluctuations in channel quality caused by electromagnetic interference in substations, easily leading to video stuttering and quality degradation. This invention uses motion compensation prediction based on inertial measurement unit motion data, narrowing the inter-frame motion vector search window and employing differentiated search strategies for critical and non-critical macroblocks, significantly reducing the computational complexity of motion estimation while maintaining coding quality. Traditional full-search algorithms use the same search strategy for all macroblocks, resulting in enormous computational demands and failing to meet the real-time coding requirements of the limited computing resources of smart safety helmets. This invention achieves an optimal balance between coding efficiency and video quality by traversing all coding modes and selecting the optimal mode using a rate-distortion optimization algorithm. Furthermore, this invention effectively suppresses error propagation caused by packet loss by periodically inserting intra-frame refresh macroblocks and employing forward error correction coding, combined with a spatiotemporal error concealment algorithm and an unavailable reference frame exclusion mechanism, improving the robustness and reliability of video transmission. Traditional video coding schemes are prone to error propagation and diffusion under packet loss conditions, leading to severe video quality degradation or even complete decoding failure.

[0067] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent equipment video encoding compression transmission, characterized in that, The intelligent equipment collects original video frames through a video acquisition module and collects inertial measurement unit motion data through an inertial measurement unit, inputs the original video frames and historical channel state information vectors into an adaptive channel prediction model to obtain channel quality prediction values, determines a coding level configuration scheme according to the channel quality prediction values, encodes the original video frames into base layer coding data and enhancement layer coding data, calculates a predicted motion vector by using the inertial measurement unit motion data for motion compensation prediction, narrows down the inter-frame motion vector search window range according to the predicted motion vector, adopts a fine search strategy to obtain key macroblock motion vectors and adopts a coarse search strategy to obtain non-key macroblock motion vectors, applies a rate-distortion optimization algorithm based on dynamic programming to traverse all coding modes of a coding unit, calculates the rate-distortion cost value of each coding mode and selects the coding mode with the minimum rate-distortion cost value as the optimal coding mode, determines coding parameter configurations according to the optimal coding mode, periodically inserts intra-refresh macroblocks in the coded video stream according to the coding parameter configurations, separates syntax element data from discrete cosine transform coefficient data for transmission, adds syntax element forward error correction redundancy data to the syntax element data and adds coefficient forward error correction redundancy data to the discrete cosine transform coefficient data, a wireless communication module receives real-time channel quality feedback values, dynamically adjusts the enhancement layer transmission strategy according to the real-time channel quality feedback values, and when a packet loss event is detected at the receiving end, a spatio-temporal error concealment algorithm is enabled to repair damaged macroblocks to obtain repaired macroblocks, and a damaged frame containing a damaged macroblock is marked as an unusable reference frame, and the subsequent encoding process excludes the unusable reference frame.

2. The method of claim 1, wherein, The adaptive channel prediction model is a lightweight convolutional prediction network, which is used to predict the future channel quality trend according to the historical channel state information vectors.

3. The method of claim 2, wherein, The input layer of the adaptive channel prediction model receives the historical channel state information vectors of the last 32 time steps, and each historical channel state information vector contains 4 normalized features including received signal strength indication values, signal-to-noise ratio values, bit error rate values and packet loss rate values.

4. The method of claim 3, wherein, The adaptive channel prediction model includes a spatial transformation layer, which adopts a geometric correction mechanism based on a spatial transformation network to perform adaptive spatial transformation on the input features, generates an affine transformation parameter matrix through a positioning network, and resamples the input feature maps according to the affine transformation parameter matrix through a sampling network to automatically correct the feature space geometric deformation and time sequence misalignment caused by device movement.

5. The method of claim 4, wherein, The geometric correction mechanism based on the spatial transformation network realizes adaptive geometric correction of the input features by introducing a learnable spatial transformation module at the front end of the network, and the positioning network adopts a two-layer fully connected network to learn 6 affine transformation parameters of the affine transformation from the input features. The 6 affine transformation parameters form a 2x3 affine transformation parameter matrix, which is used to describe the geometric transformation of translation, rotation, scaling and shearing.

6. The method of claim 5, wherein, The adaptive channel prediction model further comprises a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a full connection layer and an output layer.

7. The method of claim 6, wherein, The output layer comprises 8 output neurons corresponding to channel quality prediction values of future 8 time steps, and an output layer activation function is a Sigmoid function to normalize the output to the interval of 0 to 1.

8. The method of claim 7, wherein, The historical channel state information vector is obtained by the wireless communication module collecting channel state information in a historical time period, and the historical time period is 3.2 seconds in the past.

9. The method of claim 8, wherein, The coding level configuration scheme comprises a combination of a base layer quantization parameter and an enhancement layer quantization parameter, the base layer quantization parameter is fixed as 28, and the enhancement layer quantization parameter is dynamically adjusted between 20 and 35 according to the channel quality prediction value.

10. The method of claim 9, wherein, The base layer coding data encodes key information using the base layer quantization parameter, and the key information comprises macroblock type information, motion vector information, reference frame index information and quantization parameter information; the enhancement layer coding data encodes detail information using the enhancement layer quantization parameter, and the detail information comprises high frequency discrete cosine transform coefficients and texture detail coefficients.

Images