A wireless video transmission method, device, apparatus and storage medium

Abstract

The application belongs to the technical field of video transmission, and provides a wireless video transmission method, device, equipment and storable medium. The method comprises the following steps: based on Rician fading, a channel model between a transmitting end and a receiving end is established; based on the performance of the channel model under different weather conditions, different modulation schemes are established; during video transmission, data points describing channel states are detected, and the best modulation scheme is adaptively selected based on the K-means clustering algorithm; based on structural similarity, the similarity between a current frame and a previous frame image in a video stream to be transmitted is calculated, and the current frame image with a similarity higher than a first threshold to the previous frame image is deleted; after the receiving end receives the video stream, all frame images in the video stream are played; the application can solve the problems of slow image transmission and communication obstruction between the application and a remote controller in the prior art.

Description

Technical Field

[0001] This invention relates to the field of video transmission technology, and in particular to a wireless video transmission method, apparatus, device, and storage medium. Background Technology

[0002] In video transmission that does not rely on wired network signals, such as surveillance video signals and drone live video signals with special requirements, the communication between the transmitting and receiving ends can be affected by various factors, resulting in image display delays. For example, changes in communication conditions due to rainfall or geographical changes (urban to rural areas), or excessively high resolution and frame rate of the video stream, can lead to slow video image transmission under limited communication conditions, ultimately resulting in display delays. This is detrimental to live streaming and results in a poor product experience. Summary of the Invention

[0003] In view of the above technical problems, the present invention provides a wireless video transmission method, apparatus, device and storage medium to solve the problems of slow image transmission and communication obstruction between the transmitter and receiver in the prior art.

[0004] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0005] According to one aspect of the present invention, a wireless video transmission method is disclosed, the method comprising:

[0006] Based on Ricean fading, a channel model is established between the transmitter and receiver. The channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor that weights the different channel matrices. The channel matrix is ​​determined by the number, spacing, angle of arrival, and angle of reception of the transmitting and receiving antennas.

[0007] Based on the performance of the channel model under different weather conditions, different modulation schemes are established. During video transmission, data points describing the channel state are detected. These data points include at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographical location. The data points are grouped based on the K-means clustering algorithm, with each group representing a set of similar performance. Different modulation schemes are associated with the cluster heads of each group. When the channel state changes, the optimal modulation scheme is adaptively selected based on the K-means clustering algorithm.

[0008] Communication is performed between the transmitter and receiver according to the currently selected optimal modulation scheme. Based on structural similarity, the similarity between the current frame and the previous frame in the video stream to be transmitted is calculated. When transmitting images, the first frame is sent as a reference frame. In subsequent transmission processes, the current frame image with a similarity higher than the first threshold with the previous frame image is deleted, and the deleted image is replaced with a marker code.

[0009] After receiving the video stream, the receiving end plays all the frame images in the video stream. When the marker code is encountered, the frame image preceding the marker code is used to replace the current frame image.

[0010] Furthermore, during the video stream transmission, the method further includes:

[0011] When transmitting the video stream, a neural network-based encoder encodes the images in the video stream, such that the high-dimensional pixel matrix of the image is encoded into a low-dimensional representation, and the marker code is skipped.

[0012] After the low-dimensional representation is transmitted to the receiving end, it is decoded by the decoder of the neural network to obtain the reconstructed image. During training, the neural network undergoes iterative parameter adjustment using an extended Kalman filter.

[0013] Furthermore, the neural network undergoes iterative parameter adjustment via an extended Kalman filter during training, including:

[0014] The state vector of the extended Kalman filter is defined as the weight and bias of each layer of the neural network;

[0015] Based on the forward propagation evolution process of the state vector, establish the state transition equation;

[0016] Based on the state transition equation, estimate the predicted state and prediction error covariance of the extended Kalman filter;

[0017] Based on the predicted state and the prediction error covariance, the Kalman gain is determined, and the Kalman gain is used to update the state vector and error covariance of the neural network parameters.

[0018] In each iteration, the network parameters are updated by the extended Kalman filter based on the difference between the output of the neural network and the target output, until a predetermined number of iterations or an error threshold is reached.

[0019] Furthermore, during the video stream transmission, the method further includes:

[0020] The video stream is transmitted frame by frame. For each frame of the video stream, it is first encoded into a multi-row pixel matrix at the transmitting end, and each row is sent in real time after being encoded. The receiving end combines and displays each row of pixels received in real time.

[0021] Furthermore, during the video stream transmission, the method further includes:

[0022] When the similarity between the current frame image and the previous frame image is lower than the second threshold, the difference value of the pixels between the current frame image and the previous frame image is calculated, the difference value is mapped back to the corresponding position in the current frame image, the difference frame image is extracted and formed, and the start and end pixel coordinates are saved.

[0023] The difference frame image is transmitted in place of the current frame image. After the receiving end receives the difference frame and its corresponding pixel coordinates, it maps the difference frame to the previous frame image according to the pixel coordinates to form the current frame image.

[0024] Furthermore, the adaptive selection of the optimal modulation scheme based on the K-means clustering algorithm includes:

[0025] For each data point, calculate the Euclidean distance between each cluster head and the data point, and assign it to the nearest cluster;

[0026] Update the position of each cluster head and calculate the mean of all the data points assigned to it;

[0027] Repeat the operation until the change in the cluster head is within the threshold range or reaches the preset number of iterations, determine the position of the cluster head, and calculate its vertical distance from the performance standard;

[0028] Select the modulation scheme corresponding to the cluster head with the smallest distance.

[0029] Furthermore, the modulation scheme includes binary phase shift keying, quadrature phase shift keying, 16-quadrature amplitude modulation, and 256-QAM.

[0030] According to a second aspect of this disclosure, a wireless video transmission device is provided, comprising: a modeling module for establishing a channel model between a transmitter and a receiver based on Ricean fading, the channel model including a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor weighted for different channel matrices, the channel matrix being determined by the number, spacing, angle of arrival, and angle of reception of the transmitting and receiving antennas; and an adaptive scheduling module for establishing different modulation schemes based on the performance of the channel model under different weather conditions, detecting data points describing the channel state during video transmission, the data points including at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographical location, and grouping the data points based on a K-means clustering algorithm, each group representing a set with similar performance. The system associates different modulation schemes with the cluster head of each group. When the channel state changes, it adaptively selects the optimal modulation scheme based on the K-means clustering algorithm. The transmitting module communicates between the transmitter and receiver according to the currently selected optimal modulation scheme. Based on structural similarity, it calculates the similarity between the current frame and the previous frame in the video stream to be transmitted. During image transmission, it sends the first frame as a reference frame. In subsequent transmissions, it deletes the current frame image whose similarity to the previous frame image is higher than a first threshold and replaces the deleted image with a marker code. The receiving end playback module plays all the frame images in the video stream after the receiver receives the video stream. When the marker code is encountered, it replaces the current frame image with the previous frame image of the marker code.

[0031] According to a third aspect of this disclosure, a wireless video transmission device is provided, comprising: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to implement the wireless video transmission method as described above.

[0032] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the wireless video transmission method as described above.

[0033] The technical solution disclosed herein has the following beneficial effects:

[0034] By establishing a channel model to analyze the communication performance of the transmitter under different weather conditions, different modulation schemes are established accordingly. K-means clustering is used to intelligently select the modulation scheme to adapt to different communication conditions, so that the transmitter and receiver can maintain the best possible rate and avoid image transmission delay. At the same time, based on the set video stream processing method, the deletion of similar frames can significantly reduce the amount of data transmission, greatly reduce the communication pressure, and further avoid image transmission delay. Attached Figure Description

[0035] Figure 1 This is a flowchart of a wireless video transmission method as described in the embodiments of this specification;

[0036] Figure 2 This is a structural block diagram of the wireless video transmission device in the embodiments of this specification;

[0037] Figure 3 This refers to the terminal device of the wireless video transmission method in the embodiments of this specification;

[0038] Figure 4 This is a computer-readable storage medium for the wireless video transmission method described in the embodiments of this specification. Detailed Implementation

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure may be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., may be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0040] Furthermore, the accompanying drawings are merely illustrative of this disclosure. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0041] like Figure 1 As shown in the embodiments of this specification, a wireless video transmission method is provided. The execution subject of this method can be a server, a remote controller, etc. Specifically, the method may include the following steps S101 to S105:

[0042] In step S101, a channel model between the transmitter and receiver is established based on Ricean fading. The channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor that weights the different channel matrices. The channel matrix is ​​determined by the number, spacing, angle of arrival, and angle of reception of the transmitting and receiving antennas.

[0043] The channel model can be simplified as follows:

[0044]

[0045] The channel model uses a complex fading coefficient H, which can be decomposed into contributions from both straight-path and non-straight-path components, H. LOS The channel matrix representing a straight path, as seen in rural scenarios with few obstacles, H NLOS The channel matrix, represented as a non-linear path, is suitable for urban environments with many obstacles, allowing for the prediction of signal fluctuations and stability in different scenarios. The channel matrix has dimensions NR*NT, where NR and NT represent the number of receiving and transmitting antennas, respectively, and k is the Ricean fading factor. in and These represent the power of the straight path and the non-straight path, respectively. The Ricean fading factor k is used to quantify the relative power levels of the straight path and the non-straight path. In a multiple-input multiple-output channel, assuming the channel is stationary, the channel parameters will not change over a period of time. Therefore, H... LOS The specific form can be the geometric layout (number and spacing) of the transmitting and receiving antennas, and the mathematical formulas for how the angle of arrival and the angle of reception of the signal affect the rectilinear propagation of the channel, exemplarily:

[0046]

[0047] Where, d T and d R AOA represents the spacing between the transmitting and receiving antennas, respectively. R and AOD T These represent the angle of arrival and the angle of reception, respectively. H NLOS It follows a Rayleigh fading distribution, with the same dimension as a straight path, where Rayleigh fading is a type of Ricean fading.

[0048] In step S102, based on the performance of the channel model under different weather conditions, different modulation schemes are established. During video transmission, data points describing the channel state are detected. These data points include at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographical location. The data points are grouped based on the K-means clustering algorithm, with each group representing a set of similar performance. Different modulation schemes are associated with the cluster heads of each group. When the channel state changes, the optimal modulation scheme is adaptively selected based on the K-means clustering algorithm.

[0049] After obtaining the channel model, it can be used to understand and predict the signal propagation characteristics in video transmission, especially under different geographical and environmental conditions. This allows for the development of different modulation schemes to adapt to varying weather conditions and geographical environments. Modulation is the process of converting digital signals into wireless signals; different modulation schemes can provide different data transmission rates and error rate performance. In this step, the optimal modulation scheme is selected based on changes in channel state. Specifically, during video transmission, key channel state data points are collected, including signal-to-noise ratio (SNR), channel capacity, bit error rate (BER), signal strength, and geographical location. Then, the collected data points are grouped using a K-means clustering algorithm, dividing them into K groups. Data points within each group are similar in characteristics, while data points between groups differ significantly in characteristics. Each group represents a set of data points with similar performance. Different modulation schemes are associated with the center (cluster head) of each group. Thus, each group corresponds to a specific modulation scheme. When the channel state changes, based on the evaluation results of the K-means clustering algorithm, the modulation scheme that best matches the current channel state is adaptively selected, ensuring that video transmission maintains optimal performance under different conditions.

[0050] In step S103, communication is performed between the transmitter and receiver according to the currently selected optimal modulation scheme. Based on structural similarity, the similarity between the current frame and the previous frame in the video stream to be transmitted is calculated. When transmitting images, the first frame is sent as a reference frame. In subsequent transmission processes, the current frame image with a similarity higher than the first threshold with the previous frame image is deleted, and the deleted image is replaced with a marker code.

[0051] Structural similarity (SSIM) is used to quantify the visual similarity between consecutive frames in a video stream captured by the transmitter. It considers the brightness, contrast, and structural information of the images. Specifically, structural similarity includes: brightness comparison: calculating the average difference in brightness values ​​of corresponding pixels in two images; contrast comparison: calculating the difference in contrast (i.e., standard deviation) of corresponding pixels in two images; and structural comparison: calculating the covariance of corresponding pixels in two images, which reflects the similarity of object structures in the images. Its formula is expressed as:

[0052]

[0053] μ x and μ y These are the mean brightness values ​​of the two images, σ y and σ x These are the standard deviations of brightness for the two images, σ and σ'. xyThe covariance of the two images is represented by c1 and c2, which are small constants used to maintain stability and prevent the denominator from being zero. The SSIM result is between 0 and 1; a value of 1 indicates that the two images are identical, while a value of 0 indicates that the images are completely different. At the beginning of video stream transmission, the first frame is sent as a reference frame. This reference frame serves as the basis for comparing subsequent frames, detecting their similarity. Frames with excessively high similarity are removed from the video stream's transmission path, reducing image transmission load and improving efficiency. Removed frames are replaced with a marker code, allowing the receiver to identify similarity between the current and previous frames, thus saving bandwidth. Although some frames are deleted in this step, the overall visual quality of the video is maintained due to the high similarity threshold. The receiver can reconstruct the video stream using the reference frame and the remaining frames.

[0054] In addition, although some frames in the video stream are deleted, the original video is still stored in the non-volatile memory at the transmitter, and users can still use these original videos later.

[0055] In step S104, after the receiving end receives the video stream, all frame images in the video stream are played. When the marker code is encountered, the previous frame image of the marker code is used to replace the current frame image.

[0056] In this process, the receiving end receives each frame of the video stream. When a flag is encountered, the receiving end does not display a blank or error frame, but instead uses the previous frame to replace the currently displayed frame, thus maintaining the continuity of video playback. Since the human eye is not very sensitive to small changes between consecutive frames, this method can reduce the amount of data transmitted without significantly degrading visual quality. In particular, with the development of technology, current live video transmissions are mostly shot and transmitted at 720P resolution and 60-120 frames per second, while the human eye can generally perceive a frame rate of 60. Therefore, even if some frames are deleted, users will not perceive the difference when watching the video. This method can greatly reduce the latency of image transmission in video transmission and improve smoothness.

[0057] In one embodiment, the method further includes the following when transmitting the video stream:

[0058] When transmitting the video stream, the encoder based on the neural network encodes the images in the video stream, so that the high-dimensional pixel matrix of the image is encoded into a low-dimensional representation, and the marker code is skipped; after the low-dimensional representation is transmitted to the receiving end, it is decoded by the decoder of the neural network to obtain the reconstructed image. The neural network is trained by iteratively adjusting the parameters through an extended Kalman filter.

[0059] The encoding process begins with a high-dimensional pixel matrix containing the color information of each pixel in the image. The encoder extracts image features through a series of layers (such as convolutional layers, pooling layers, or fully connected layers), enabling the identification of important patterns and structures in the image. Simultaneously, the encoder uses pooling operations to progressively reduce the dimensionality of the data, removing redundant information and retaining the most important features, outputting a low-dimensional representation. This low-dimensional representation is a much smaller dataset than the original pixel matrix, reducing data transmission pressure and latency. At the receiving end, after receiving the low-dimensional representation, decoding—the reverse of encoding—is performed. Upsampling (or transposed convolution) operations are used to progressively increase the data dimensionality, thereby gradually restoring the spatial resolution of the image. During upsampling, the decoder uses the feature maps learned by the encoder to reconstruct image details, including visual features such as color, texture, and shape. Based on this, efficient image encoding and reconstruction are achieved during video stream transmission. The decoder and encoder employ an extended Kalman filter to iteratively adjust network parameters (weights and biases) to optimize encoding and decoding performance, minimize prediction errors, and improve convergence speed.

[0060] Specifically, during training, the neural network undergoes iterative parameter adjustment using an extended Kalman filter, including: defining the state vector of the extended Kalman filter as the weights and biases of each layer of the neural network; establishing a state transition equation based on the forward propagation evolution of the state vector; estimating the predicted state and prediction error covariance of the extended Kalman filter based on the state transition equation; determining the Kalman gain based on the predicted state and prediction error covariance, and using the Kalman gain to update the state vector and error covariance of the neural network parameters; and updating the network parameters through the extended Kalman filter in each iteration based on the difference between the output of the neural network and the target output, until a predetermined number of iterations or an error threshold is reached.

[0061] As an explanation, neural networks typically contain nonlinear activation functions. Kalman filters handle the nonlinearity of the entire network by linearizing these nonlinear functions. Specifically, they predict the state of the network parameters at the next time step through the state transition equation, estimating the uncertainty of the predicted state, i.e., the prediction error covariance. This helps to understand the reliability of the prediction. Then, based on the prediction error covariance and the observation error covariance, the Kalman filter calculates a Kalman gain to determine the weight of the observation data in the state update. Using the Kalman gain, the network parameters are updated to make the predicted output closer to the target output.

[0062] In another implementation, unlike the neural network encoding and decoding described above, the method further includes the following when transmitting the video stream:

[0063] The video stream is transmitted frame by frame. For each frame of the video stream, it is first encoded into a multi-row pixel matrix at the transmitting end, and each row is sent in real time after being encoded. The receiving end combines and displays each row of pixels received in real time.

[0064] In this method, the video stream is broken down into individual frames, and each frame is transmitted independently. This facilitates the processing of real-time video streams. Each frame is encoded into a multi-row pixel matrix at the transmitting end, meaning the image is divided into multiple horizontal stripes or rows. Each row contains a portion of the image's pixel data. After each row of pixels is encoded, it is immediately sent to the receiving end, ensuring rapid data transmission and reducing latency. At the receiving end, each row of pixel data is decoded and combined in real time to reconstruct and display each frame, improving video smoothness. As each row of pixel data is received and combined, the receiving end can gradually construct a complete frame image and continuously display the video stream. In this embodiment, because the data is sent immediately after preparation, rather than waiting for the entire frame to be encoded, this line-by-line encoding and transmission method reduces transmission latency from the transmitting end to the receiving end. If a row of data is lost or corrupted during transmission, it will only affect a small portion of the video frame, not the entire frame, thus improving transmission fault tolerance. Furthermore, this method dynamically adjusts the amount of data per row of pixels according to available bandwidth to adapt to different network conditions, achieving low latency and high real-time performance.

[0065] In one embodiment, when transmitting the video stream, the method further includes: when the similarity between the current frame image and its previous frame image is lower than a second threshold, calculating the pixel difference value between the current frame image and the previous frame image, mapping the difference value back to the corresponding position in the current frame image, extracting and forming a difference frame image, and saving the start and end pixel coordinates; transmitting the difference frame image instead of the current frame image; and after the receiving end receives the difference frame and its corresponding pixel coordinates, mapping the difference frame to the previous frame image according to the pixel coordinates to form the current frame image.

[0066] Due to the setting of the second threshold, when there are significant differences between frames, a pixel-level comparison is performed between the current frame and the previous frame. The difference value between each corresponding pixel is calculated, and the calculated difference value is mapped back to the corresponding position in the current frame to form a new image, namely the difference frame image. The difference frame image highlights the areas that have changed since the previous frame. At this time, the coordinates of the start and end pixels in the difference frame are recorded and saved. These coordinates identify the position of the changed area in the image. Then, the difference frame image and its corresponding pixel coordinates are transmitted to the receiving end, instead of transmitting the entire current frame image, which can significantly reduce the amount of data transmitted. After receiving the difference frame and the corresponding pixel coordinates, the receiving end maps the changes in the difference frame back to the corresponding position in the previous frame image based on these coordinates, reconstructs and fuses them to form the current frame image, thereby maintaining the continuity and integrity of the video stream, optimizing the transmission of video data, reducing the amount of data generated by sending the entire frame, and preserving important visual changes in the video.

[0067] In one implementation, the adaptive selection of the optimal modulation scheme based on the K-means clustering algorithm includes: for each data point, calculating the Euclidean distance between each cluster head and the data point, and assigning it to the nearest cluster; updating the position of each cluster head, and calculating the mean of all the assigned data points for it; repeating the operation until the change of the cluster head is within a threshold range or reaches a preset number of iterations, determining the position of the cluster head, and calculating its vertical distance to the performance standard; and selecting the modulation scheme corresponding to the cluster head with the smallest distance.

[0068] The modulation schemes include binary phase-shift keying (BPSK), quadrature phase-shift keying (QPSK), 16-QAM, and 256-QAM. BPSK is the simplest modulation scheme, offering the lowest spectral efficiency but the highest noise immunity. QPSK offers even higher spectral efficiency but is more sensitive to noise than BPSK. 16-QAM offers higher data transmission rates but requires better channel conditions. 256-QAM offers the highest spectral efficiency but has the most stringent channel requirements and is highly sensitive to noise and interference.

[0069] Based on the same line of thought, such as Figure 2As shown, an exemplary embodiment of this disclosure also provides a wireless video transmission device, including: a modeling module 201, used to establish a channel model between the transmitter and receiver based on Ricean fading, the channel model including a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor weighted for different channel matrices, the channel matrix being determined by the number, spacing, angle of arrival, and angle of reception of the transmitting and receiving antennas; and an adaptive scheduling module 202, used to establish different modulation schemes based on the performance of the channel model under different weather conditions, detect data points describing the channel state during video transmission, the data points including at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographical location, and group the data points based on a K-means clustering algorithm, each group representing devices with similar performance. The set of different modulation schemes is associated with the cluster head of each group. When the channel state changes, the optimal modulation scheme is adaptively selected based on the K-means clustering algorithm. The transmitting module 203 is used to communicate between the transmitting end and the receiving end according to the currently selected optimal modulation scheme, and calculate the similarity between the current frame and the previous frame in the video stream to be transmitted based on structural similarity. When transmitting images, the first frame is sent as a reference frame. In subsequent transmission processes, the current frame image with a similarity higher than a first threshold with the previous frame image is deleted, and the deleted image is replaced with a marker code. The receiving end playback module 204 is used to play all the frame images in the video stream after the receiving end receives the video stream. When the marker code is encountered, the current frame image is replaced with the previous frame image of the marker code.

[0070] The aforementioned device analyzes the communication performance of the transmitter under different weather conditions by establishing a channel model, thereby establishing different corresponding modulation schemes. K-means clustering is used to intelligently select the modulation scheme to adapt to different communication conditions, so that the transmitter and receiver can maintain the best possible speed and avoid image transmission delay. At the same time, based on the set video stream processing method, the deletion of similar frames can significantly reduce the amount of data transmission, greatly reduce the communication pressure, and further avoid image transmission delay.

[0071] The specific details of each module in the above-mentioned device have been described in detail in the method section of the implementation. For any undisclosed details, please refer to the implementation content of the method section, and therefore will not be repeated here.

[0072] Following the same line of thought, this specification also provides a wireless video transmission device, such as... Figure 3 As shown.

[0073] The wireless video transmission device can be the terminal device or server provided in the above embodiments.

[0074] Wireless video transmission devices can vary significantly due to differences in configuration or performance. They may include one or more processors 301 and memory 302, with memory 302 storing one or more application programs or data. Memory 302 may include readable media in the form of volatile memory cells, such as random access memory (RAM) and / or cache memory, and may further include read-only memory cells. The application programs stored in memory 302 may include one or more program modules (not shown in the figures), including but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Furthermore, processor 301 may be configured to communicate with memory 302 to execute a series of computer-executable instructions stored in memory 302 on the video privacy protection device. The video privacy protection device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more I / O interfaces (input / output interfaces) 305, one or more external devices 306 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a receiving end to interact with the device, and / or with any device that enables the device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed through I / O interfaces 305. Furthermore, the device can also communicate with one or more networks (e.g., local area networks (LANs)) through wired or wireless interfaces 304.

[0075] Specifically, in this embodiment, the video privacy protection device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the video privacy protection device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0076] Based on Ricean fading, a channel model is established between the transmitter and receiver. The channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor that weights the different channel matrices. The channel matrix is ​​determined by the number, spacing, angle of arrival, and angle of reception of the transmitting and receiving antennas.

[0077] Based on the performance of the channel model under different weather conditions, different modulation schemes are established. During video transmission, data points describing the channel state are detected. These data points include at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographical location. The data points are grouped based on the K-means clustering algorithm, with each group representing a set of similar performance. Different modulation schemes are associated with the cluster heads of each group. When the channel state changes, the optimal modulation scheme is adaptively selected based on the K-means clustering algorithm.

[0078] Communication is performed between the transmitter and receiver according to the currently selected optimal modulation scheme. Based on structural similarity, the similarity between the current frame and the previous frame in the video stream to be transmitted is calculated. When transmitting images, the first frame is sent as a reference frame. In subsequent transmission processes, the current frame image with a similarity higher than the first threshold with the previous frame image is deleted, and the deleted image is replaced with a marker code.

[0079] After receiving the video stream, the receiving end plays all the frame images in the video stream. When the marker code is encountered, the frame image preceding the marker code is used to replace the current frame image.

[0080] Based on the same idea, exemplary embodiments of this disclosure also provide a computer-readable storage medium storing a program product capable of implementing the methods described above. In some possible embodiments, various aspects of this disclosure can also be implemented as a program product including program code, which, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to the various exemplary embodiments of this disclosure.

[0081] refer to Figure 4 As shown, a program product 400 for implementing the above-described method according to an exemplary embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0082] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0083] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0084] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0085] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the receiving computing device, partially on the receiving computing device, as a standalone software package, partially on the receiving computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the receiving computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0086] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the exemplary embodiments of this disclosure.

[0087] Furthermore, the above figures are merely illustrative representations of the processes included in the methods according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0088] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0089] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A method of wireless video transmission, characterized by, The method comprises: Based on the Rician fading, a channel model between the transmitting end and the receiving end is established, the channel model comprises a channel matrix of a straight path, a channel matrix of a non-straight path, a Rician fading factor for weighting different channel matrices, the channel matrix is determined by the number, spacing, angle of arrival, and angle of reception of the transmitting antenna and the receiving antenna; Based on the performance of the channel model under different weather conditions, different modulation schemes are established, data points describing the channel state are detected during video transmission, the data points comprise signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographical position, the data points are grouped based on the K-means clustering algorithm, each group represents a set with similar performance, and different modulation schemes are associated with cluster heads of each group; When the channel state changes, the best modulation scheme is adaptively selected based on the K-means clustering algorithm, comprising: for each data point, calculating the Euclidean distance between each cluster head and the data point, and assigning to the nearest cluster; updating the position of each cluster head and calculating the mean of all data points assigned thereto; repeating the operation until the change of the cluster head is within a threshold range or a preset number of iterations is reached, determining the position of the cluster head, calculating the perpendicular distance between the cluster head and the performance standard; selecting the modulation scheme corresponding to the cluster head with the smallest distance; According to the currently selected best modulation scheme, communication is carried out between the transmitting end and the receiving end, and based on structural similarity, the similarity between the current frame and the previous frame image in the video stream to be transmitted is calculated; during image transmission, the first frame image is sent as a reference frame, in the subsequent transmission process, the current frame image with a similarity higher than a first threshold to the previous frame image is deleted and replaced by a marker code; when the similarity between the current frame image and the previous frame image is lower than a second threshold, the difference value of the pixels between the current frame image and the previous frame image is calculated, the difference value is mapped back to the corresponding position of the current frame image, a difference frame image is extracted, and the starting and ending pixel coordinates are saved; the difference frame image is transmitted instead of the current frame image, and after the receiving end receives the difference frame image and the corresponding pixel coordinates, the difference frame image is mapped to the previous frame image according to the pixel coordinates to form the current frame image; After the receiving end receives the video stream, all frame images in the video stream are played, and when the marker code is encountered, the previous frame image of the marker code is used instead of the current frame image.

2. The wireless video transmission method of claim 1, wherein, The modulation schemes comprise binary phase shift keying, quadrature phase shift keying, 16 quadrature amplitude modulation, and 256-QAM.

3. The wireless video transmission method of claim 1, wherein, The method further comprises, when transmitting the video stream, encoding images in the video stream based on a neural network encoder, so that a high-dimensional pixel matrix of the images is encoded into a low-dimensional representation, and the marker code is skipped; after transmitting the low-dimensional representation to a receiving end, decoding the low-dimensional representation through a decoder of the neural network to obtain reconstructed images, the neural network is iteratively adjusted in parameters through an extended Kalman filter during training; wherein the neural network is iteratively adjusted in parameters through an extended Kalman filter during training, comprising: defining a state vector of the extended Kalman filter as weights and biases of each layer of the neural network; establishing a state transition equation according to a forward propagation evolution process of the state vector; estimating a predicted state and a predicted error covariance of the extended Kalman filter according to the state transition equation; determining a Kalman gain according to the predicted state and the predicted error covariance, and using the Kalman gain to update the state vector and the error covariance of the parameters of the neural network; in each iteration, the network parameters are updated through the extended Kalman filter according to the difference between the output of the neural network and the target output, until a predetermined number of iterations or an error threshold is reached.

4. A wireless video transmission apparatus, characterized by comprising: Comprise: A channel modeling module is configured to establish a channel model between a transmitting end and a receiving end based on a Rician fading, the channel model comprising a channel matrix of a straight path, a channel matrix of a non-straight path, and a Rician fading factor for weighting different channel matrices, the channel matrix being determined by the number, spacing, angle of arrival, and angle of reception of transmitting antennas and receiving antennas; An adaptive scheduling module is configured to establish different modulation schemes based on the performance of the channel model under different weather conditions, detect data points describing channel states during video transmission, the data points comprising signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographic location, group the data points based on a K-means clustering algorithm, each group representing a set of similar performance, and associate different modulation schemes with cluster heads of each group; When the channel state changes, the best modulation scheme is adaptively selected based on the K-means clustering algorithm, comprising: for each data point, calculating the Euclidean distance between each cluster head and the data point, and assigning it to the nearest cluster; updating the position of each cluster head and calculating the mean of all data points assigned to it; repeating the operation until the change of the cluster head is within a threshold range or a preset number of iterations is reached, determining the position of the cluster head, calculating the perpendicular distance between the cluster head and the performance standard; selecting the modulation scheme corresponding to the cluster head with the smallest distance; The sending module is configured to communicate between the transmitting end and the receiving end according to the currently selected optimal modulation scheme, and calculate the similarity between the current frame and the previous frame image in the video stream to be transmitted based on the structural similarity; during image transmission, the first frame image is sent as a reference frame, and in the subsequent sending process, the current frame image with a similarity higher than a first threshold to the previous frame image is deleted and replaced by a marker code; when the similarity between the current frame image and the previous frame image is lower than a second threshold, the difference value of the pixels between the current frame image and the previous frame image is calculated, the difference value is mapped back to the corresponding position of the current frame image, a difference frame image is extracted, and the starting and ending pixel coordinates are saved; the difference frame image is transmitted instead of the current frame image, and after the receiving end receives the difference frame image and the corresponding pixel coordinates, the difference frame image is mapped to the previous frame image according to the pixel coordinates to form the current frame image; The receiving end playing module is configured to play all frame images in the video stream after the receiving end receives the video stream, and replace the current frame image with the previous frame image of the marker code when the marker code is encountered.

5. A wireless video transmission device, characterized by, Comprise: a processor; and a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the wireless video transmission method of any one of claims 1-2.

6. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the wireless video transmission method of any one of claims 1 to 2. The computer program is executed by the processor to implement the wireless video transmission method of any one of claims 1 to 2.

Images