Wireless video transmission method, device and equipment and storable medium

Abstract

The invention belongs to the technical field of video transmission, and provides a wireless video transmission method and device, equipment and a storable medium. The method comprises the following steps: establishing a channel model between a transmitting end and a receiving end based on Rician fading; establishing different modulation schemes based on the performance of the channel model under different weather conditions, detecting data points describing a channel state during video transmission, and adaptively selecting an optimal modulation scheme based on the K-means clustering algorithm; based on the structural similarity, calculating the similarity between the current frame and the previous frame of image in the to-be-transmitted video stream, and deleting the current frame of image of which the similarity with the previous frame of image is higher than a first threshold value; after a receiving end receives the video stream, playing all frame images in the video stream; according to the invention, the problems of slow image transmission and blocked communication with the remote controller in the prior art can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of video transmission, and in particular to a wireless video transmission method, device, equipment and storable medium. Background Art

[0002] In video transmission that does not rely on wired network signals, such as surveillance video signals with special needs and drone video live broadcast signals, the communication between the transmitter and the receiver will be affected by various factors, resulting in image display delays. For example, rainfall, geographical changes (urban and rural), etc. may cause changes in communication conditions, or the resolution and frame rate of the video stream may be too large. Under limited communication conditions, video images are transmitted slowly, which ultimately leads to display delays. This is not conducive to live broadcasting and the product experience is not good. Summary of the invention

[0003] In view of the above technical problems, the present invention provides a wireless video transmission method, device, equipment and storable medium to solve the problems of slow image transmission and obstructed communication between the transmitter and the receiver in the prior art.

[0004] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0005] According to one aspect of the present invention, a wireless video transmission method is disclosed, the method comprising:

[0006] Based on Ricean fading, a channel model between the transmitting end and the receiving end is established. The channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor for weighting different channel matrices. The channel matrix is ​​determined by the number, spacing, arrival angle, and reception angle of the transmitting antenna and the receiving antenna.

[0007] Based on the performance of the channel model under different weather conditions, different modulation schemes are established, data points describing the channel state are detected during video transmission, the data points at least including signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographic location, the data points are grouped based on a K-means clustering algorithm, each group represents a set with similar performance, different modulation schemes are associated with a cluster head of each group, and when the channel state changes, the best modulation scheme is adaptively selected based on the K-means clustering algorithm;

[0008] The communication is performed between the transmitting end and the receiving end according to the currently selected best modulation scheme, and based on the structural similarity, the similarity between the current frame and the previous frame image in the video stream to be transmitted is calculated, and when the image is transmitted, the first frame image is sent as a reference frame, and in the subsequent transmission process, the current frame image whose similarity with the previous frame image is higher than a first threshold is deleted, and the deleted image is replaced by a marker code;

[0009] After receiving the video stream, the receiving end plays all the frame images in the video stream, and when encountering the marker code, replaces the current frame image with the frame image before the marker code.

[0010] Furthermore, when performing the video stream transmission, the method further includes:

[0011] When transmitting the video stream, encoding the image in the video stream based on the neural network encoder so that the high-dimensional pixel matrix of the image is encoded into a low-dimensional representation and the marker code is skipped;

[0012] After the low-dimensional representation is transmitted to the receiving end, it is decoded by the decoder of the neural network to obtain a reconstructed image. During the training, the neural network is iteratively adjusted in parameters through an extended Kalman filter.

[0013] Furthermore, the neural network is subjected to iterative adjustment of parameters through an extended Kalman filter during training, including:

[0014] Defining the state vector of the extended Kalman filter as the weights and biases of each layer of the neural network;

[0015] Establishing a state transfer equation according to the forward propagation evolution process of the state vector;

[0016] estimating a predicted state and a prediction error covariance of the extended Kalman filter according to the state transfer equation;

[0017] Determine a Kalman gain according to the predicted state and the predicted error covariance, and use the Kalman gain to update the state vector and error covariance of the parameters of the neural network;

[0018] In each iteration, the network parameters are updated by the extended Kalman filter according to the difference between the output of the neural network and the target output until a predetermined number of iterations or error threshold is reached.

[0019] Furthermore, when performing the video stream transmission, the method further includes:

[0020] The video stream is transmitted frame by frame, and each frame image in the video stream is first encoded into a plurality of rows of pixel matrices at the transmitting end, and each row is sent in real time after being encoded, and the receiving end combines and displays each row of pixels received in real time.

[0021] Furthermore, when performing the video stream transmission, the method further includes:

[0022] When the similarity between the current frame image and the previous frame image is lower than a second threshold, the difference value between the pixels of the current frame image and the previous frame image is calculated, the difference value is mapped back to the corresponding position of the current frame image, a difference frame image is extracted, and the starting and ending pixel coordinates are saved;

[0023] The difference frame image is used to replace the current frame image for transmission. After the receiving end receives the difference frame and the corresponding pixel coordinates, the difference frame is mapped to the previous frame image according to the pixel coordinates to form the current frame image.

[0024] Further, the adaptively selecting the best modulation scheme based on the K-means clustering algorithm includes:

[0025] For each of the data points, the Euclidean distance between each cluster head and the data point is calculated and assigned to the nearest cluster;

[0026] Update the position of each cluster head and calculate the mean of all the data points assigned to it;

[0027] Repeat the operation until the change of the cluster head is within the threshold range or reaches the preset number of iterations, determine the location of the cluster head, and calculate the vertical distance between it and the performance standard;

[0028] The modulation scheme corresponding to the cluster head with the smallest distance is selected.

[0029] Furthermore, the modulation schemes include binary phase shift keying, quadrature phase shift keying, 16-quadrature amplitude modulation and 256-QAM.

[0030] According to a second aspect of the present disclosure, a wireless video transmission device is provided, including: a modeling module, which is used to establish a channel model between a transmitter and a receiver based on Ricean fading, wherein the channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor for weighting different channel matrices, and the channel matrix is ​​determined by the number, spacing, arrival angle, and reception angle of the transmitting antenna and the receiving antenna; an adaptive scheduling module, which is used to establish different modulation schemes based on the performance of the channel model under different weather conditions, detect data points describing the channel state during video transmission, wherein the data points include at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographic location, and group the data points based on the K-means clustering algorithm, and each group represents a collection with similar performance. , associating different modulation schemes with the cluster head of each group, and adaptively selecting the best modulation scheme based on the K-means clustering algorithm when the channel state changes; a sending module, used to communicate between the transmitting end and the receiving end according to the currently selected best modulation scheme, and based on the structural similarity, calculate the similarity between the current frame and the previous frame image in the video stream to be transmitted, and send the first frame image as a reference frame when transmitting the image, and in the subsequent sending process, delete the current frame image whose similarity with the previous frame image is higher than the first threshold, and replace the deleted image with a marker code; a receiving end playback module, used to play all frame images in the video stream after the receiving end receives the video stream, and when encountering the marker code, replace the current frame image with the previous frame image of the marker code.

[0031] According to a third aspect of the present disclosure, there is provided a wireless video transmission device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor implements the wireless video transmission method as described above;

[0032] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, storing a computer program, wherein the computer program implements the wireless video transmission method as described above when executed by a processor.

[0033] The technical solution disclosed in this disclosure has the following beneficial effects:

[0034] By establishing a channel model to analyze the communication performance of the transmitter under different weather conditions, we can establish corresponding different modulation schemes, and use K-means clustering to intelligently select modulation schemes to adapt to different communication conditions, so that the transmitter and the receiver can maintain the best rate as much as possible to avoid image transmission delays. At the same time, based on the set video stream processing method, the deletion of similar frames can greatly reduce the amount of data transmission, greatly reduce communication pressure, and further avoid image transmission delays. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of a wireless video transmission method in an embodiment of this specification;

[0036] Figure 2 This is a structural block diagram of a wireless video transmission device in an embodiment of this specification;

[0037] Figure 3 It is a terminal device of the wireless video transmission method in the embodiment of this specification;

[0038] Figure 4 A computer-readable storage medium for the wireless video transmission method in the embodiments of this specification. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0040] In addition, the accompanying drawings are only schematic illustrations of the present disclosure. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0041] like Figure 1 As shown, the embodiment of this specification provides a wireless video transmission method, and the execution subject of the method can be a server, a remote controller, etc. The method can specifically include the following steps S101 to S105:

[0042] In step S101, based on Ricean fading, a channel model between the transmitter and the receiver is established. The channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor for weighting different channel matrices. The channel matrix is ​​determined by the number, spacing, arrival angle, and reception angle of the transmitting antenna and the receiving antenna.

[0043] Among them, the channel model can be simplified as:

[0044]

[0045] The channel model is represented by a complex fading coefficient H, which can be decomposed into the contributions of the straight path and the non-straight path. LOS The channel matrix of the straight path is shown in the rural scene with few obstacles. NLOS The channel matrix is ​​represented as a non-linear path, which is used in urban environments with many obstacles. This can be used to predict the fluctuation and stability of signals in different scenarios. The dimension of the channel matrix is ​​NR*NT, where NR and NT represent the number of receiving antennas and transmitting antennas respectively, and k is the Rice fading factor. in and are the power of the straight path and the non-straight path respectively. The Rice fading factor k is used to quantify the relative power level of the straight path and the non-straight path. In a multiple-input multiple-output channel, assuming that the channel is stationary, the channel parameters will not change over a period of time, then H LOS The specific form of can be a mathematical formula of how the geometric layout (number and spacing) of the transmitting antenna and the receiving antenna, the arrival angle and the reception angle of the signal affect the straight-line propagation of the channel, exemplary:

[0046]

[0047] Among them, d T and d R Respectively represent the distance between the transmitting antenna and the receiving antenna, AOA R and AOD T Denote the arrival angle and reception angle respectively. NLOS It obeys the Rayleigh fading distribution, and its dimension is the same as that of the straight line path, where Rayleigh fading is a type of Ricean fading.

[0048] In step S102, different modulation schemes are established based on the performance of the channel model under different weather conditions, and data points describing the channel status are detected during video transmission, wherein the data points include at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographic location. The data points are grouped based on the K-means clustering algorithm, and each group represents a set with similar performance. Different modulation schemes are associated with the cluster head of each group. When the channel status changes, the best modulation scheme is adaptively selected based on the K-means clustering algorithm.

[0049] After the channel model is obtained, it can be used to understand and predict the signal propagation characteristics in video transmission, especially under different geographical and environmental conditions, and then different modulation schemes can be established to adapt to different weather conditions and geographical environments. Modulation is the process of converting digital signals into wireless signals. Different modulation schemes can provide different data transmission rates and error rate performance. In this step, the best modulation scheme is selected according to the change of channel state. Specifically: during the video transmission process, key channel state data points are collected, including signal-to-noise ratio (SNR), channel capacity, bit error rate (BER), signal strength, geographical location, etc., and then the collected data points are grouped using the K-means clustering algorithm to divide the data points into K groups. The data points in each group are similar in characteristics, while the data points between groups are quite different in characteristics. Each group represents a set of data points with similar performance. Different modulation schemes are associated with the center (cluster head) of each group. In this way, each group corresponds to a specific modulation scheme. When the channel state changes, based on the evaluation results of the K-means clustering algorithm, the modulation scheme that best matches the current channel state is adaptively selected to ensure that video transmission can maintain the best performance under different conditions.

[0050] In step S103, communication is performed between the transmitter and the receiver according to the currently selected optimal modulation scheme, and based on the structural similarity, the similarity between the current frame and the previous frame image in the video stream to be transmitted is calculated. When transmitting the image, the first frame image is sent as a reference frame. In the subsequent sending process, the current frame image whose similarity with the previous frame image is higher than the first threshold is deleted, and the deleted image is replaced by a marker code.

[0051] Among them, structural similarity (SSIM) can be used to quantify the visual similarity between consecutive frames in the video stream captured by the transmitter, which takes into account the brightness, contrast and structural information of the image. Specifically, structural similarity includes: brightness comparison: calculating the average difference in the brightness values ​​of the corresponding pixels of the two images; contrast comparison: calculating the difference in the contrast (i.e., standard deviation) of the corresponding pixels of the two images; structural comparison: calculating the covariance of the corresponding pixels of the two images, which reflects the similarity of the structure of the objects in the images. Its formula is expressed as:

[0052]

[0053] μ x and μ y are the brightness mean of the two images, σ y and σ x are the brightness standard deviations of the two images, σ xy is the covariance of the two images, c 1 and c 2is a small constant used to maintain stability and prevent the denominator from being zero. The result of SSIM is between 0 and 1. When it is 1, it means that the two images are exactly the same, and when it is 0, it means that the images are completely different. At the beginning of the video stream transmission, the first frame image is sent as a reference frame. The reference frame is the basis for comparison of subsequent frames and is used to detect the similarity between subsequent frame images and it. The current frame image with too high similarity will be eliminated from the transmission path of the video stream, reducing the pressure of image transmission and improving transmission efficiency. The eliminated frame image is replaced with a marker code, so that the receiving end can know that this frame is similar to the previous frame, thereby saving transmission bandwidth. Although some frames are deleted in this step, the overall visual quality of the video is maintained due to the setting of a high similarity threshold. The receiving end can rely on the reference frame and the undeleted frame image to reconstruct the video stream.

[0054] In addition, although some frame images in the video stream are deleted, the captured original video is still stored in the non-volatile memory of the transmitter, and the user can still use the original video later.

[0055] In step S104, after receiving the video stream, the receiving end plays all the frame images in the video stream, and when encountering the marker code, replaces the current frame image with the frame image before the marker code.

[0056] Among them, the receiving end receives each frame of the image in the video stream. When encountering a marker code, the receiving end does not display a blank or error frame, but uses the previous frame of the image to replace the current frame to be displayed, thereby maintaining the continuity of the video playback. Since the human eye is not very sensitive to small changes between consecutive frames, this method can reduce the amount of data transmitted without significantly reducing the visual quality. In particular, with the development of technology, the existing live video transmission is mostly shot and transmitted at 720P resolution and 60-120 frame rate, while the human eye can generally recognize a frame rate of 60. Therefore, in this way, even if some frames are deleted, the user will not notice the difference when watching the video. Through this method, the delay of image transmission in video transmission can be greatly reduced and the fluency can be improved.

[0057] In one embodiment, when performing the video stream transmission, the method further includes:

[0058] When transmitting the video stream, an encoder based on a neural network encodes the image in the video stream so that the high-dimensional pixel matrix of the image is encoded into a low-dimensional representation and the marking code is skipped; after the low-dimensional representation is transmitted to the receiving end, it is decoded by the decoder of the neural network to obtain a reconstructed image. During training, the neural network iteratively adjusts the parameters through an extended Kalman filter.

[0059] The encoding process starts with a high-dimensional pixel matrix, which contains the color information of each pixel of the image. The encoder extracts the features of the image through a series of layers (such as convolutional layers, pooling layers or fully connected layers, etc.), so that the important patterns and structures in the image can be identified. While extracting the features, the encoder gradually reduces the dimension of the data through pooling operations, removes redundant information and retains the most important features, and outputs a low-dimensional representation. The low-dimensional representation is a data set that is much smaller than the original pixel matrix, reducing the pressure of data transmission and reducing transmission delay. At the receiving end, after receiving the low-dimensional representation of the data set, decoding is performed, which is the inverse process of encoding, such as using upsampling (or transposed convolution) operations to gradually increase the dimension of the data, thereby gradually restoring the spatial resolution of the image. In the upsampling process, the decoder uses the feature mapping learned by the encoder to reconstruct the details of the image, including visual features such as color, texture and shape. Based on this, efficient image encoding and reconstruction are achieved during video stream transmission. The above decoder and encoder use extended Kalman filters to iteratively adjust network parameters (weights and biases) to optimize the performance of encoding and decoding, minimize prediction errors, and improve convergence speed.

[0060] Specifically, the neural network is subjected to iterative parameter adjustment through an extended Kalman filter during training, including: defining the state vector of the extended Kalman filter as the weight and bias of each layer of the neural network; establishing a state transfer equation according to the forward propagation evolution process of the state vector; estimating the predicted state and predicted error covariance of the extended Kalman filter according to the state transfer equation; determining the Kalman gain according to the predicted state and the predicted error covariance, and using the Kalman gain to update the state vector and error covariance of the parameters of the neural network; in each iteration, updating the network parameters through the extended Kalman filter according to the difference between the output of the neural network and the target output, until a predetermined number of iterations or an error threshold is reached.

[0061] As an explanation, neural networks usually contain nonlinear activation functions. The Kalman filter handles the nonlinear characteristics of the entire network by linearizing these nonlinear functions. Specifically, it predicts the state of the network parameters at the next moment through the state transfer equation and estimates the uncertainty of the predicted state, namely the prediction error covariance, which helps to understand the reliability of the prediction. Then, based on the prediction error covariance and the observation error covariance, the Kalman filter calculates a Kalman gain to determine the weight of the observed data in the state update. Using the Kalman gain, the parameters of the network are updated to make the predicted output closer to the target output.

[0062] In another embodiment, different from the above-mentioned neural network encoding and decoding, when performing the video stream transmission, the method further includes:

[0063] The video stream is transmitted frame by frame, and each frame image in the video stream is first encoded into a plurality of rows of pixel matrices at the transmitting end, and each row is sent in real time after being encoded, and the receiving end combines and displays each row of pixels received in real time.

[0064] The video stream is decomposed into separate frames, and each frame is transmitted independently, which helps to process real-time video streams. Each frame of the image is encoded into a plurality of rows of pixel matrices at the transmitting end, which means that the image is divided into multiple horizontal strips or rows, and each row contains a portion of the pixel data of the image. After each row of pixels is encoded, the row of data is immediately sent to the receiving end, which ensures the rapid transmission of data and helps to reduce delays. Each row of pixel data received by the receiving end is decoded and combined in real time so that each frame of the image can be reconstructed and displayed in time, which improves the smoothness of the video. With the reception and combination of each row of pixel data, the receiving end can gradually construct a complete frame image and continuously display the video stream. In this embodiment, since the data is sent immediately after it is ready, instead of waiting for the entire frame to be encoded, this method of encoding and transmitting line by line can reduce the transmission delay from the transmitting end to the receiving end. If a row of data is lost or damaged during transmission, it will only affect a small part of the video frame, not the entire frame, thereby improving the fault tolerance of the transmission. In addition, the amount of data per row of pixels is dynamically adjusted according to the available bandwidth to adapt to different network conditions, so as to achieve low latency and high real-time performance.

[0065] In one embodiment, when performing the video stream transmission, the method also includes: when the similarity between the current frame image and its previous frame image is lower than a second threshold, calculating the difference value of the pixels between the current frame image and the previous frame image, mapping the difference value back to the corresponding position of the current frame image, extracting the difference frame image, and saving the starting and ending pixel coordinates; transmitting the difference frame image instead of the current frame image, and after the receiving end receives the difference frame and the corresponding pixel coordinates, mapping the difference frame to the previous frame image according to the pixel coordinates to form the current frame image.

[0066] Among them, due to the setting of the second threshold, when there is a significant difference between frames, the current frame is compared with the previous frame at the pixel level, the difference value between each corresponding pixel is calculated, and the calculated difference value is mapped back to the corresponding position of the current frame to form a new image, namely the difference frame image. The difference frame image highlights the area that has changed since the previous frame. At this time, the coordinates of the start and end pixels in the difference frame are recorded and saved. These coordinates identify the position of the changed area in the image, and then the difference frame image and its corresponding pixel coordinates are transmitted to the receiving end instead of transmitting the entire current frame image, which can significantly reduce the amount of transmitted data. After receiving the difference frame and the corresponding pixel coordinates, the receiving end maps the changes in the difference frame back to the corresponding position of the previous frame image according to these coordinates, and reconstructs and fuses to form the current frame image, thereby maintaining the continuity and integrity of the video stream, optimizing the transmission of video data, reducing the amount of data generated by sending the entire frame, and retaining important visual changes in the video.

[0067] In one embodiment, the adaptive selection of the best modulation scheme based on the K-means clustering algorithm includes: for each of the data points, calculating the Euclidean distance between each cluster head and the data point, and assigning it to the nearest cluster; updating the position of each cluster head, and calculating the mean of all the assigned data points for it; repeating the operation until the change of the cluster head is within a threshold range or reaches a preset number of iterations, determining the position of the cluster head, and calculating the vertical distance between it and the performance standard; and selecting the modulation scheme corresponding to the cluster head with the smallest distance.

[0068] The modulation schemes include binary phase shift keying, quadrature phase shift keying, 16-quadrature amplitude modulation and 256-QAM. Binary phase shift keying (BPSK) is the simplest modulation scheme, providing the lowest spectral efficiency but the highest noise immunity. Quadrature phase shift keying (QPSK) can provide higher spectral efficiency but is more sensitive to noise than BPSK. 16-quadrature amplitude modulation (16-QAM) can provide higher data transmission rates but requires better channel conditions. 256-QAM can provide the highest spectral efficiency but has the highest requirements on channel conditions and is very sensitive to noise and interference.

[0069] Based on the same idea, Figure 2As shown, an exemplary embodiment of the present disclosure also provides a wireless video transmission device, including: a modeling module 201, which is used to establish a channel model between a transmitter and a receiver based on Rice fading, wherein the channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Rice fading factor for weighting different channel matrices, and the channel matrix is ​​determined by the number, spacing, arrival angle, and reception angle of the transmitting antenna and the receiving antenna; an adaptive scheduling module 202, which is used to establish different modulation schemes based on the performance of the channel model under different weather conditions, detect data points describing the channel state during video transmission, wherein the data points include at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographic location, and group the data points based on the K-means clustering algorithm, and each group represents a group with similar performance. a set of modulation schemes, associating different modulation schemes with the cluster head of each group, and adaptively selecting the best modulation scheme based on the K-means clustering algorithm when the channel state changes; a sending module 203, used to communicate between the transmitting end and the receiving end according to the currently selected best modulation scheme, and calculate the similarity between the current frame and the previous frame image in the video stream to be transmitted based on the structural similarity, and when transmitting the image, send the first frame image as a reference frame, and in the subsequent sending process, delete the current frame image whose similarity with the previous frame image is higher than the first threshold, and replace the deleted image with a marker code; a receiving end playback module 204, used to play all frame images in the video stream after the receiving end receives the video stream, and when encountering the marker code, replace the current frame image with the previous frame image of the marker code.

[0070] The above-mentioned device analyzes the communication performance of the transmitter under different weather conditions by establishing a channel model, thereby establishing corresponding different modulation schemes, and adopts K-means clustering to intelligently select modulation schemes to adapt to different communication conditions, so that the transmitter and the receiver can maintain the best rate as much as possible to avoid image transmission delays; at the same time, based on the set video stream processing method, the deletion of similar frames can greatly reduce the data transmission volume, greatly reduce the communication pressure, and further avoid image transmission delays.

[0071] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0072] Based on the same idea, the embodiment of this specification also provides a wireless video transmission device, such as Figure 3 shown.

[0073] The wireless video transmission device may be the terminal device or the server provided in the above embodiments.

[0074] The wireless video transmission device may have relatively large differences due to different configurations or performances, and may include one or more processors 301 and memory 302, and the memory 302 may store one or more storage applications or data. Among them, the memory 502 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache storage unit, and may further include a read-only storage unit. The application stored in the memory 302 may include one or more program modules (not shown in the figure), such program modules include but are not limited to: an operating system, one or more applications, other program modules and program data, each of these examples or some combination may include the implementation of a network environment. Furthermore, the processor 301 can be configured to communicate with the memory 302 to execute a series of computer executable instructions in the memory 302 on the video privacy protection device. The video privacy protection device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more I / O interfaces (input and output interfaces) 305, one or more external devices 306 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a receiving end to interact with the device, and / or communicate with any device that enables the device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through the I / O interface 305. In addition, the device may also communicate with one or more networks (e.g., a local area network (LAN)) through the wired or wireless interface 304.

[0075] Specifically, in this embodiment, the video privacy protection device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions in the video privacy protection device, and the one or more programs are configured to be executed by one or more processors, including computer executable instructions for performing the following:

[0076] Based on Ricean fading, a channel model between the transmitting end and the receiving end is established. The channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor for weighting different channel matrices. The channel matrix is ​​determined by the number, spacing, arrival angle, and reception angle of the transmitting antenna and the receiving antenna.

[0077] Based on the performance of the channel model under different weather conditions, different modulation schemes are established, data points describing the channel state are detected during video transmission, the data points at least including signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographic location, the data points are grouped based on a K-means clustering algorithm, each group represents a set with similar performance, different modulation schemes are associated with a cluster head of each group, and when the channel state changes, the best modulation scheme is adaptively selected based on the K-means clustering algorithm;

[0078] The communication is performed between the transmitting end and the receiving end according to the currently selected best modulation scheme, and based on the structural similarity, the similarity between the current frame and the previous frame image in the video stream to be transmitted is calculated, and when the image is transmitted, the first frame image is sent as a reference frame, and in the subsequent transmission process, the current frame image whose similarity with the previous frame image is higher than a first threshold is deleted, and the deleted image is replaced by a marker code;

[0079] After receiving the video stream, the receiving end plays all the frame images in the video stream, and when encountering the marker code, replaces the current frame image with the frame image before the marker code.

[0080] Based on the same idea, the exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0081] refer to Figure 4 As shown, a program product 400 for implementing the above method according to an exemplary embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0082] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0083] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0084] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0085] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the receiving end computing device, partially on the receiving end device, as a separate software package, partially on the receiving end computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the receiving end computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0086] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary implementation of the present disclosure.

[0087] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiment of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0088] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0089] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A wireless video transmission method, characterized in that: The method comprises: Based on Ricean fading, a channel model between the transmitting end and the receiving end is established. The channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Ricean fading factor for weighting different channel matrices. The channel matrix is ​​determined by the number, spacing, arrival angle, and reception angle of the transmitting antenna and the receiving antenna. Based on the performance of the channel model under different weather conditions, different modulation schemes are established, data points describing the channel state are detected during video transmission, the data points at least including signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographic location, the data points are grouped based on a K-means clustering algorithm, each group represents a set with similar performance, different modulation schemes are associated with a cluster head of each group, and when the channel state changes, the best modulation scheme is adaptively selected based on the K-means clustering algorithm; The communication is performed between the transmitting end and the receiving end according to the currently selected best modulation scheme, and based on the structural similarity, the similarity between the current frame and the previous frame image in the video stream to be transmitted is calculated, and when the image is transmitted, the first frame image is sent as a reference frame, and in the subsequent transmission process, the current frame image whose similarity with the previous frame image is higher than a first threshold is deleted, and the deleted image is replaced by a marker code; After receiving the video stream, the receiving end plays all the frame images in the video stream, and when encountering the marker code, replaces the current frame image with the frame image before the marker code.

2. A wireless video transmission method according to claim 1, characterized in that: When performing the video stream transmission, the method further includes: When transmitting the video stream, encoding the image in the video stream based on the neural network encoder so that the high-dimensional pixel matrix of the image is encoded into a low-dimensional representation and the marker code is skipped; After the low-dimensional representation is transmitted to the receiving end, it is decoded by the decoder of the neural network to obtain a reconstructed image. During the training, the neural network is iteratively adjusted in parameters through an extended Kalman filter.

3. A wireless video transmission method according to claim 2, characterized in that: The neural network is trained by iteratively adjusting parameters through an extended Kalman filter, including: Defining the state vector of the extended Kalman filter as the weights and biases of each layer of the neural network; Establishing a state transfer equation according to the forward propagation evolution process of the state vector; estimating a predicted state and a prediction error covariance of the extended Kalman filter according to the state transfer equation; Determine a Kalman gain according to the predicted state and the predicted error covariance, and use the Kalman gain to update the state vector and error covariance of the parameters of the neural network; In each iteration, the network parameters are updated by the extended Kalman filter according to the difference between the output of the neural network and the target output until a predetermined number of iterations or error threshold is reached.

4. A wireless video transmission method according to claim 1, characterized in that: When performing the video stream transmission, the method further includes: The video stream is transmitted frame by frame, and each frame image in the video stream is first encoded into a plurality of rows of pixel matrices at the transmitting end, and each row is sent in real time after being encoded, and the receiving end combines and displays each row of pixels received in real time.

5. A wireless video transmission method according to claim 1, characterized in that: When performing the video stream transmission, the method further includes: When the similarity between the current frame image and the previous frame image is lower than a second threshold, the difference value between the pixels of the current frame image and the previous frame image is calculated, the difference value is mapped back to the corresponding position of the current frame image, a difference frame image is extracted, and the starting and ending pixel coordinates are saved; The difference frame image is used to replace the current frame image for transmission. After the receiving end receives the difference frame and the corresponding pixel coordinates, the difference frame is mapped to the previous frame image according to the pixel coordinates to form the current frame image.

6. A wireless video transmission method according to claim 1, characterized in that: The adaptively selecting the best modulation scheme based on the K-means clustering algorithm includes: For each of the data points, the Euclidean distance between each cluster head and the data point is calculated and assigned to the nearest cluster; Update the position of each cluster head and calculate the mean of all the data points assigned to it; Repeat the operation until the change of the cluster head is within the threshold range or reaches the preset number of iterations, determine the location of the cluster head, and calculate the vertical distance between it and the performance standard; The modulation scheme corresponding to the cluster head with the smallest distance is selected.

7. A wireless video transmission method according to claim 1, characterized in that: The modulation schemes include binary phase shift keying, quadrature phase shift keying, 16-quadrature amplitude modulation, and 256-QAM.

8. A wireless video transmission device, characterized in that: include: A channel modeling module is used to establish a channel model between the transmitter and the receiver based on Rice fading. The channel model includes a channel matrix for a straight path, a channel matrix for a non-straight path, and a Rice fading factor for weighting different channel matrices. The channel matrix is ​​determined by the number, spacing, arrival angle, and reception angle of the transmitting antenna and the receiving antenna. An adaptive scheduling module, for establishing different modulation schemes based on the performance of the channel model under different weather conditions, detecting data points describing the channel state during video transmission, wherein the data points include at least signal-to-noise ratio, channel capacity, bit error rate, signal strength, and geographic location, grouping the data points based on a K-means clustering algorithm, wherein each group represents a set with similar performance, associating different modulation schemes with a cluster head of each group, and adaptively selecting the best modulation scheme based on the K-means clustering algorithm when the channel state changes; A sending module, used for communicating between a transmitting end and a receiving end according to a currently selected optimal modulation scheme, and calculating the similarity between a current frame and a previous frame of an image in a video stream to be transmitted based on structural similarity, sending a first frame of an image as a reference frame when transmitting an image, deleting a current frame of an image whose similarity with a previous frame of an image is higher than a first threshold value in a subsequent sending process, and replacing the deleted image with a marker code; The receiving end playing module is used to play all the frame images in the video stream after the receiving end receives the video stream, and when encountering the marking code, replace the current frame image with the frame image before the marking code.

9. A wireless video transmission device, characterized in that: include: processor; and a memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the wireless video transmission method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the wireless video transmission method according to any one of claims 1 to 7 is implemented.

Images