Video transmission method and device, equipment, storage medium and program product

Abstract

The invention relates to a video transmission method and device, equipment, a storage medium and a program product. The method comprises the following steps: under the condition that the network quality of a wireless network accessed by video sending equipment is smaller than a quality threshold value, acquiring channel state information of a wireless signal transmitted between the video sending equipment and video receiving equipment according to a preset sampling frequency; determining a current scene of the video sending device according to the information similarity between different channel state information; sending video data to video receiving equipment by adopting a video data sending scheme matched with the current scene; the video data sending scheme comprises a scheme for sending the video data based on the video picture change data, or a scheme for sending the video data by adjusting the sending number of B frames in a basic picture group GOP. According to the method, the video data is sent based on the video picture change data and the adjustment of the B frame sending quantity, so that the video transmission data quantity can be reduced, and the definition and fluency of the video are considered.

Description

Technical Field

[0001] This application relates to the field of data transmission technology, and in particular to a video transmission method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of the Internet, the demand for high-definition video transmission has exploded. However, severe channel fading, high packet loss rates, and latency jitter in weak wireless network environments (such as subway tunnels, remote areas, and the outskirts of densely populated cities) seriously restrict the availability of high-definition video services.

[0003] In traditional technologies, the severe imbalance between ultra-high-definition video bitrate and weak network bandwidth causes video transmitting devices to actively reduce the resolution of the transmitted video in weak network environments in order to reduce the amount of data transmitted between the video transmitting device and the video receiving device. This is to avoid video playback stuttering on the video receiving device, resulting in the video receiving device only being able to play low-resolution video. Furthermore, when the channel change delay (200-500 milliseconds) far exceeds the human eye's perceptible stuttering threshold (100 milliseconds), it will cause poor smoothness such as screen freezing and mosaic.

[0004] Therefore, ensuring the clarity and smoothness of video playback in weak network environments has become an urgent problem to be solved. Summary of the Invention

[0005] Therefore, it is necessary to provide a video transmission method, apparatus, device, storage medium, and program product to address the aforementioned technical problems.

[0006] In a first aspect, this application provides a video transmission method, including:

[0007] When the network quality of the wireless network accessed by the video transmitting device is less than the quality threshold, multiple channel state information of the wireless signal transmitted between the video transmitting device and the video receiving device are obtained according to the preset sampling frequency.

[0008] The current scene of the video transmission device is determined based on the information similarity between multiple channel state information.

[0009] A video data transmission scheme matching the current scenario is adopted to send video data to the video receiving device. The video data transmission scheme includes a scheme that transmits video data based on changes in video frame data, or a scheme that transmits video data by adjusting the number of B frames sent in the basic GOP. The basic GOP is the GOP used when the network quality of the wireless network accessed by the video transmitting device is not less than the quality threshold.

[0010] In one embodiment, determining the current scene of the video transmitting device based on the information similarity between multiple channel state information includes: determining the information similarity between multiple channel state information based on the parameter values ​​of each state parameter in different channel state information; determining the current scene of the video transmitting device as a static scene if the information similarity is greater than a similarity threshold; and determining the fluctuation index corresponding to the video transmitting device based on the fluctuation index if the information similarity is less than or equal to the similarity threshold.

[0011] In one embodiment, each state parameter includes phase offset and amplitude attenuation; determining the fluctuation index corresponding to the video transmitting device based on the preset sampling frequency and the parameter values ​​of each state parameter in different channel state information includes: determining a first rate of change of phase offset based on the preset sampling frequency and the parameter values ​​of phase offset in different channel state information; and determining a second rate of change of amplitude attenuation based on the preset sampling frequency and the parameter values ​​of amplitude attenuation in different channel state information; and determining the fluctuation index corresponding to the video transmitting device based on the first rate of change and the second rate of change.

[0012] In one embodiment, determining the current scenario based on the volatility index includes: determining the current scenario as a first dynamic scenario if the volatility index is less than a first volatility threshold; determining the current scenario as a second dynamic scenario if the volatility index is not less than the first volatility threshold and is less than a second volatility threshold; determining the current scenario as a third dynamic scenario if the volatility index is not less than the second volatility threshold and is less than a third volatility threshold; and determining the current scenario as a fourth dynamic scenario if the volatility index is not less than the third volatility threshold.

[0013] In one embodiment, the fluctuation index corresponding to the video transmission device is determined based on the preset sampling frequency and the parameter values ​​of each state parameter in the channel state information. This includes: determining the cause of the difference between different channel state information based on the difference information between different channel state information; if the cause of the difference is the cause of target intrusion, determining the fluctuation index corresponding to the video transmission device based on the preset sampling frequency and the parameter values ​​of each state parameter in different channel state information; and if the cause of the difference is environmental interference or an unknown cause, determining the current scene as a static scene.

[0014] In one embodiment, when the current scene is a static scene, a video data transmission scheme matching the current scene is adopted to send video data to the video receiving device, including: sending a substitute frame data packet to the video receiving device at a preset transmission frequency; wherein, the substitute frame data packet includes visual attribute adjustment parameters, which are determined based on the video image change data between the current video frame of the video transmitting device and the reference frame previously sent to the video receiving device, and the reference frame is an I-frame or a P-frame; and sending an I-frame to the video receiving device at specified intervals.

[0015] In one embodiment, if the cause of the difference between multiple channel state information is unknown, the method further includes: sending P frames to a video receiving device; and sending I frames to the video receiving device for each first number of P frames sent to the video receiving device.

[0016] In one embodiment, a video data transmission scheme matching the current scene is adopted to transmit video data to the video receiving device, including any of the following: when the current scene is a first dynamic scene, video frames are transmitted to the video receiving device sequentially based on the order of various types of video frames in a first target GOP; wherein the first target GOP is obtained by adding B-frames to a basic GOP; when the current scene is a second dynamic scene, video frames are transmitted to the video receiving device sequentially based on the order of various types of video frames in a second target GOP; wherein the second target GOP is obtained by replacing B-frames in the basic GOP with P-frames; when the current scene is a third dynamic scene, I-frames and P-frames are transmitted to the video receiving device sequentially, skipping B-frames in the basic GOP, based on the order of various types of video frames in the basic GOP; when the current scene is a fourth dynamic scene, dynamically encoded frames are transmitted to the video receiving device sequentially; wherein the dynamically encoded frame is a video frame obtained by encoding a changing image region, the changing image region being determined based on the video frame change data between the current video frame of the video transmitting device and the reference frame previously transmitted to the video receiving device, the reference frame being an I-frame or a P-frame.

[0017] Secondly, this application also provides a video transmission device, comprising:

[0018] The acquisition module is used to acquire multiple channel state information of the wireless signal transmitted between the video transmitting device and the video receiving device according to a preset sampling frequency when the network quality of the wireless network accessed by the video transmitting device is less than a quality threshold.

[0019] The determination module is used to determine the current scene of the video transmission device based on the information similarity between multiple different channel state information.

[0020] The sending module is used to send video data to the video receiving device using a video data sending scheme that matches the current scene. The video data sending scheme includes a scheme that sends video data based on changes in video frame data, or a scheme that sends video data by adjusting the number of B frames sent in the basic GOP. The basic GOP is the GOP used when the network quality of the wireless network accessed by the video sending device is not less than the quality threshold.

[0021] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the various method embodiments provided in the first aspect above:

[0022] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the various method embodiments provided in the first aspect above.

[0023] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the various method embodiments provided in the first aspect above.

[0024] In the aforementioned video transmission methods, apparatus, devices, storage media, and program products, when the network quality of the wireless network accessed by the video transmitting device is less than a quality threshold, the current scenario of the video transmitting device is determined based on the information similarity between different channel state information between the video transmitting device and the video receiving device. A video data transmission scheme matching the current scenario is then adopted to transmit video data to the video receiving device. This video data transmission scheme includes schemes based on video frame change data or schemes that transmit video data by adjusting the number of B-frames transmitted in a Group of Picture Frames (GOP). In this way, on the one hand, transmitting video data based on video frame change data and adjusting the number of B-frame transmissions effectively reduces the amount of data transmitted compared to directly transmitting video frames, without reducing the clarity of the transmitted video frames, thus ensuring the clarity of the video played by the video receiving device. On the other hand, it allows for flexible adjustment of the amount of data transmitted between the video transmitting device and the video receiving device according to the current scenario of the video transmitting device, effectively reducing the bit rate and video playback stuttering, thereby balancing the smoothness and clarity of the video data and achieving high-definition video transmission in weak wireless network environments. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a diagram illustrating the application environment of the video transmission method in the embodiments of this application;

[0027] Figure 2 This is a flowchart illustrating a video transmission method according to an embodiment of this application;

[0028] Figure 3A This is a flowchart illustrating the scene determination method in the embodiments of this application;

[0029] Figure 3B This is a time sequence diagram of an information similarity comparison process in an embodiment of this application;

[0030] Figure 4A This is a schematic diagram of the scene judgment process of the video sending device in the embodiments of this application;

[0031] Figure 4B This is a schematic diagram of the transmission frame sequence in an embodiment of this application;

[0032] Figure 4C This is a flowchart illustrating the video transmission strategy under unknown circumstances in the embodiments of this application;

[0033] Figure 5A This is a schematic diagram of a method for determining the cause of difference in an embodiment of this application;

[0034] Figure 5B This is a schematic diagram of the overall process of the video transmission method in the embodiments of this application;

[0035] Figure 6 This is a schematic diagram of the structure of a video transmission device according to an embodiment of this application;

[0036] Figure 7 This is an internal structural diagram of a computer device according to an embodiment of this application.

[0037] Figure 8 This is an internal structure diagram of another computer device in an embodiment of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0039] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments or any combination of multiple embodiments.

[0040] In traditional technologies, the severe imbalance between ultra-high-definition video bitrate and weak network bandwidth causes video transmitting devices to actively reduce the resolution of the transmitted video in weak network environments. This reduces the amount of data transmitted with the video receiving device, thus preventing playback stuttering. Consequently, the receiving device can only play low-resolution video, and when channel abrupt changes in latency (200–500 milliseconds) far exceed the human eye's perceptible stuttering threshold (100 milliseconds), it can cause issues such as image freezing and pixelation, resulting in poor smoothness. Therefore, ensuring both clarity and smoothness of video playback in weak network environments has become a pressing problem to be solved.

[0041] To address the aforementioned problems, embodiments of this application provide a video transmission method that can be applied to, for example... Figure 1 In the application environment shown, the video receiving device 102 communicates with the video sending device 104 via a network. A data storage system can store data that the video sending device 104 needs to process, such as video data. The data storage system can be integrated into the video sending device 104 or placed in the cloud or on other network servers. The video sending device 104 can be a computer device with video transmission capabilities; this computer device can be a terminal, such as a camera, camcorder, or mobile phone, or a server, such as the backend server of an image acquisition device. The video receiving device 102 can be a computer device with video display capabilities; this computer device can be a terminal, such as various personal computers, laptops, smartphones, tablets, cameras, or other video acquisition devices, or a server. The servers acting as the video sending device 104 or video receiving device 102 can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud services.

[0042] In one exemplary embodiment, such as Figure 2As shown, a video transmission method is provided, which is applied to... Figure 1 Taking the video transmission device 104 as an example, the following steps are included:

[0043] S201, when the network quality of the wireless network accessed by the video transmitting device is less than the quality threshold, multiple channel status information of the wireless signal transmitted between the video transmitting device and the video receiving device are obtained according to the preset sampling frequency.

[0044] Channel State Information (CSI) describes the channel characteristics experienced by a wireless signal along its propagation path from the transmitter (video transmitting device) to the receiver (video receiving device). It can include amplitude attenuation, phase shift, and Doppler shift. Amplitude attenuation characterizes how much the signal strength weakens; phase shift characterizes the change in the signal's phase; and Doppler shift characterizes the frequency change caused by the relative motion between the transmitter and receiver. Optionally, the above channel state information can be represented in matrix form, called a CSI matrix.

[0045] The video transmitting device monitors the network quality of the accessed wireless network in real time. Optionally, network quality can be measured individually using quality indicators such as signal strength, bandwidth, latency, packet loss rate, throughput, and channel utilization. Correspondingly, the quality threshold is the threshold value for the corresponding quality indicator. For example, if the bandwidth is below 100 Mbps and the network quality is determined to be below the quality threshold, channel state information is obtained according to a preset sampling frequency. Similarly, if the latency is greater than 50 milliseconds and the network quality is determined to be below the quality threshold, channel state information is obtained according to the preset sampling frequency. Alternatively, network quality can be comprehensively measured using multiple quality indicators. For example, signal strength, bandwidth, and packet loss rate can be normalized and then weighted and summed, with the result used as the network quality. Or, bandwidth, latency, packet loss rate, and throughput can be normalized and averaged, with the result used as the network quality. Furthermore, the aforementioned quality thresholds and preset sampling frequencies can be set according to actual needs; for example, the video transmitting device can obtain signal state information once per second.

[0046] Optionally, the wireless signal transmitted between the video transmitting device and the video receiving device is a video stream obtained by the video transmitting device from the video acquisition area. Each video frame in this video stream has corresponding channel state information. Therefore, the preset sampling frequency can be set according to the motion of various targets within the video acquisition area of ​​the video transmitting device. It is understandable that when the motion of various targets within the video acquisition area is high (e.g., high speed), in order to ensure data transmission between the video transmitting device and the video receiving device, the video transmitting device needs to obtain the channel state information with the video receiving device in a timely manner to determine the video data transmission scheme based on the channel state information and avoid affecting data transmission. Therefore, when the motion of various targets within the video acquisition area is high, the preset sampling frequency also needs to be set higher. For example, the preset sampling frequency can be the video stream frame rate, i.e., acquiring the channel state information frame by frame. Conversely, when the motion of various targets within the video acquisition area is low (e.g., low speed), the preset sampling frequency can be appropriately reduced. For example, channel state information can be acquired every preset number of frames, and the channel state information acquired each time includes the channel state information of the preset number of video frames generated after the previous acquisition. The preset quantity can be set based on empirical values, experimental values ​​from multiple trials, and the needs of actual applications.

[0047] For ease of description, when the network quality of the wireless network accessed by the video transmitting device is less than the quality threshold, it can be referred to as the video transmitting device being in a weak network environment. In the following text, "the video transmitting device being in a weak network environment" refers to the wireless network accessed by the video transmitting device having a network quality less than the quality threshold.

[0048] In one optional embodiment, the channel state information may be obtained by the video transmitting device according to a preset sampling frequency based on the data transmission situation between the video transmitting device and the video receiving device, or it may be obtained by the video receiving device according to a preset sampling frequency based on the data transmission situation between the video transmitting device and the video receiving device and fed back to the video transmitting device.

[0049] S202, determine the current scene of the video transmission device based on the information similarity between multiple channel state information;

[0050] Typically, channel state information can be represented by a matrix or vector. Therefore, the information similarity between multiple channel state information can be determined by determining the similarity between vectors (matrices). For example, cosine similarity can be used to determine the information similarity between different channel state information. No specific limitation is made here.

[0051] Optionally, if the channel state information is acquired twice and only one channel state information is acquired each time, the information similarity between the two channel state information can be directly determined; or, if the channel state information is acquired twice and multiple channel state information is acquired each time, the average value of the multiple channel state information acquired each time can be determined, and the information similarity between the two average values ​​can be determined.

[0052] Optionally, if different channel state information is acquired more than twice, the acquired channel state information can be divided into two groups according to the acquisition time from earliest to latest. The information similarity between the two groups of channel state information is then obtained by comparing their similarity. For example, if different channel state information is acquired four times, the first two acquired channel state information are divided into one group, and the last two acquired channel state information are divided into another group, and the information similarity between the two groups is obtained by comparing their similarity.

[0053] In one embodiment, the number of times channel state information is acquired, and the method for grouping the channel state information when the number of acquisitions is greater than two, can be set according to the movement of various targets within the video capture area of ​​the video transmitting device. Specifically, when the movement speed of various targets within the video capture area is high, in order to quickly obtain the information similarity between multiple channel state information, the number of acquisitions of channel state information can be set to two, so as to quickly and timely determine the subsequent video data transmission plan when the video transmitting device is detected to be in a weak network state; when the movement speed of various targets within the video capture area is low, in order to improve the accuracy of information similarity, the number of acquisitions of channel state information can be set to greater than two. Optionally, when the number of acquisitions of channel state information is even, the channel state information acquired multiple times can be divided into two information groups evenly, while when the number of acquisitions of channel state information is odd, it can be divided according to the order of acquisition time from earliest to latest. For example, if the number of acquisitions of channel state information is 5, the channel state information acquired the first three times can be divided into one group, and the channel state information acquired the last three times can be divided into another group, wherein the channel state information acquired the third time is divided into two groups. The number of times the channel state information is acquired, and the method for grouping the channel state information when the number of acquisitions is greater than two, can be set based on empirical values, experimental values ​​from multiple trials, and the needs of actual applications. No specific limitations are imposed on these settings.

[0054] Optionally, in order to compare the similarity between two sets of channel state information, the average value of multiple channel state information contained in each set of channel state information can be determined, and the information similarity between the two obtained average values ​​can be determined as the information similarity between multiple channel state information; or, the similarity between two channel state information with the same order in the two sets of channel state information can be determined in turn to obtain multiple similarities, and the average value of the above multiple similarities can be determined as the information similarity between multiple channel state information.

[0055] It should be noted that the above embodiments are examples of specific implementation methods for determining the information similarity between multiple channel state information and are not limitations. Any specific implementation method that can determine the information similarity between multiple channel state information is within the protection scope of this application.

[0056] The current scene of the video transmitting device can be determined based on the similarity of the obtained information. This can be achieved by defining different similarity intervals. For example, if the information similarity is greater than or equal to a similarity threshold, the current scene is determined to be a static scene, while if the information similarity is less than the similarity threshold, the current scene is determined to be a dynamic scene. Furthermore, more similarity intervals can be defined to achieve more detailed scene classification.

[0057] Typically, the video footage transmitted from a video transmitting device to a video receiving device is generated by the transmitting device capturing video within its covered video capture area. The movement of people, vehicles, and other targets within this capture area determines the frequency and magnitude of changes in the content of the different video frames captured by the transmitting device. When the movement of various targets within the video capture area is low, such as in nighttime scenes, the changes in the content of the different video frames captured by the transmitting device are slow and infrequent. In some cases, the content of the different video frames captured by the transmitting device can be considered constant; therefore, this scenario can be considered a static scene for the video transmitting device. Conversely, when the movement of various targets within the video capture area is high, such as at road intersections during rush hour, the changes in the content of the different video frames captured by the transmitting device are rapid and significant. Therefore, this scenario can be considered a dynamic scene for the video transmitting device. Furthermore, the movement of various targets within the video capture area in dynamic scenes can be further subdivided to classify the dynamic scenes of the video transmitting device into different levels, such as low-dynamic scenes, medium-dynamic scenes, and high-dynamic scenes.

[0058] S203 uses a video data transmission scheme that matches the current scenario to send video data to the video receiving device.

[0059] The video data transmission schemes include schemes that transmit video data based on changes in video frames, or schemes that transmit video data by adjusting the number of B frames transmitted in a basic GOP (Group of Pictures). The basic GOP is the GOP used when the network quality of the wireless network accessed by the video transmission device is not less than a quality threshold.

[0060] Depending on the current scenario of the video transmitting device, the following two video data transmission schemes can be used to send video data to the video receiving device:

[0061] Option 1: Send video data based on changes in video frame.

[0062] In situations where the video transmitting device is in a weak network environment, to reduce data transmission volume, the data source device can determine the video data to be sent to the video receiving device based on video frame change data, instead of sending the entire video frame. This allows the video receiving device to analyze the video frame change data between the current frame to be displayed and the received previous reference frame, and then, based on this change data and the previous reference frame (i.e., I-frame or P-frame), reconstruct and display the current frame, ensuring normal video playback while effectively reducing data transmission and stuttering.

[0063] Option 2: Adjust the number of B frames sent in the basic frame group (GOP) to transmit video data.

[0064] A Group of Pictures (GOP) is a collection of consecutive frames in a video sequence. It defines a relatively independent coding unit structure. When transmitting video data, video transmitting devices divide a long video into multiple consecutive GOPs for compression. A GOP typically begins with a keyframe and contains the following frame types:

[0065] I-frame (intra-coded frame, keyframe): It contains complete picture information itself, so it can present a complete and independent picture without relying on any other frames. It is similar to a complete Joint Photographic Experts Group (JPEG) picture. It is the starting frame and random access point of a GOP and has a relatively low compression ratio.

[0066] P-frame (forward predictive coded frame): It relies on the previous I-frame or P-frame for encoding to present a complete picture. It only stores the part that differs from the previous reference frame (i.e., I-frame or P-frame), and the compression ratio is higher than that of I-frame.

[0067] B-frames (bidirectional predictive coded frames): rely on preceding and following reference frames for encoding and decoding, storing the differences between the frames and the preceding and following reference frames. They have the highest compression ratio, but also the largest encoding and decoding latency (requiring the arrival of subsequent reference frames).

[0068] In this embodiment, the base GOP is the GOP used when the network quality of the wireless network accessed by the video transmitting device is not less than the quality threshold. The base GOP starts with a keyframe (I-frame). In this embodiment, the base GOP is mainly described as [IBBPBBPBB]. It should be noted that the number of frames, the number of B-frames, and the order of P-frames in the base GOP may differ for different video transmitting devices. The above base GOP is only an example and is not specifically limited here.

[0069] In the aforementioned video transmission method, when the network quality of the wireless network accessed by the video transmitting device is less than a quality threshold, the current scenario of the video transmitting device is determined based on the information similarity between different channel state information between the video transmitting device and the video receiving device. A video data transmission scheme matching the current scenario is then adopted to send video data to the video receiving device. This video data transmission scheme includes a scheme based on video frame change data or a scheme by adjusting the number of B-frames sent in the basic frame group (GOP). On the one hand, transmitting video data based on video frame change data and adjusting the number of B-frames sent effectively reduces the amount of data transmitted compared to directly sending video frames, without reducing the clarity of the transmitted video frames, thus ensuring the clarity of the video played by the receiving device. On the other hand, it allows for flexible adjustment of the amount of data transmitted between the video transmitting and receiving devices according to the current scenario of the video transmitting device, effectively reducing the bit rate and video playback stuttering, thereby balancing the smoothness and clarity of the video data and achieving high-definition video transmission in weak wireless network environments.

[0070] Based on the above embodiments, in an exemplary embodiment, the determination of the current scenario is further refined. Optionally, such as... Figure 3A As shown, the following steps may be included:

[0071] S301, determine the information similarity between multiple channel state information based on the parameter values ​​of each state parameter in different channel state information.

[0072] When the network quality of the wireless network accessed by the video transmitting device is less than a quality threshold, the video transmitting device acquires channel state information according to a preset sampling frequency. After acquiring multiple channel state information sets, the information similarity between them can be determined. For example, if the channel state information is represented in the form of a CSI matrix, the cosine similarity of multiple CSI matrices can be determined as the information similarity.

[0073] It is understandable that if the information similarity between multiple channel state information is high, it indicates that the image changes of the corresponding video frames are small. Therefore, the current scene of the video transmitting device can be determined by comparing the information similarity and a similarity threshold.

[0074] S302, if the information similarity is greater than the similarity threshold, determine that the current scene of the video sending device is a static scene.

[0075] The similarity threshold can be set based on empirical values, experimental results from multiple trials, and the needs of actual applications, such as 0.98, without specific limitations. For example, when the information similarity is greater than 0.98, the current scene of the video sending device is determined to be a static scene.

[0076] like Figure 3B The diagram shown is a timing diagram of an information similarity comparison process in an embodiment of this application. Taking a similarity threshold of 0.98 as an example, when the number of channel state information items is two, the timing diagram may include the following steps:

[0077] Step 1: Obtain channel state information 1;

[0078] Step 2: Obtain channel state information 2 and determine the information similarity between channel state information 2 and channel state information 1;

[0079] Step 3: If the information similarity is greater than 0.98, the current scene of the video sending device is determined to be a static scene.

[0080] In an exemplary embodiment, when the current scene is a static scene, a video data transmission scheme matching the current scene is used to send video data to the video receiving device. This includes sending a substitute frame data packet to the video receiving device at a preset transmission frequency; and sending an I-frame to the video receiving device at specified intervals. The substitute frame data packet includes visual attribute adjustment parameters, which are determined based on video frame change data between the current video frame of the video transmitting device and a reference frame previously sent to the video receiving device. The reference frame is either an I-frame or a P-frame.

[0081] Visual attributes are attributes used to describe the basic characteristics of visual phenomena. They can include at least one of attributes such as brightness, contrast, saturation, color temperature, and hue. Visual attribute adjustment parameters are used to adjust the corresponding visual attributes. For example, when the visual attribute is brightness, the visual attribute adjustment parameter can be to reduce the brightness by 30%.

[0082] In this embodiment, when the current scene is static, the video data transmission scheme is based on video image change data. Because it is a static scene, the changes in the image are very small, even negligible. However, due to factors such as lighting, the visual attributes of the video image may change, for example, the brightness may increase. After receiving the frame data packet, the video receiving device can parse the visual attribute adjustment parameters and then adjust the visual attributes of the previously acquired keyframe according to the visual attribute adjustment parameters to restore the video image to be displayed. Since the continuously displayed multiple frames are all obtained by adjusting the visual attributes of the previous keyframe according to the visual attribute adjustment parameters, the content of the images in the continuously displayed multiple frames does not change.

[0083] Specifically, a substitute frame data packet can be determined based on the video frame change data between the current video frame of the video transmitting device and the reference frame last sent to the video receiving device, according to a preset transmission frequency, and then sent to the video receiving device. For example, one substitute frame data packet can be sent per second. This substitute frame data packet can also be called a CSI substitute frame.

[0084] Optionally, the structure of the frame data packet can be found in Table 1:

[0085] Table 1. Structure of a Frame Data Packet

[0086]

[0087] Among them, CSI source information is channel state information, and Cyclic Redundancy Check (CRC) is used for data integrity verification. It should be noted that the structure of the frame data packets in Table 1 is only for illustrative purposes, and can be set according to actual needs; no specific limitations are made here.

[0088] Understandably, if the video transmitting device remains in a static scene, it will continuously send frame data packets to the video receiving device. Over time, the background of the video captured by the transmitting device will change. If only frame data packets containing visual attribute adjustment parameters are sent, the changes in the background cannot be conveyed to the receiving device, leading to accumulated errors in the video displayed. Therefore, to eliminate these accumulated errors and ensure the accuracy of the displayed video, an I-frame needs to be sent to the receiving device at specified intervals as the background of the video. The receiving device can then combine the visual attribute adjustment parameters with the I-frame to reconstruct a new video image. The specified interval can be set according to requirements. When the accuracy and integrity of the video image are highly critical, the specified interval can be set lower, for example, sending one I-frame every 6 hours. When the smoothness of the video image is less critical, the specified interval can be set higher, for example, sending one I-frame every 24 hours. Alternatively, if it is determined through observation that the video image is prone to change within a certain time period, the I-frame can be set to be sent to the receiving device after that time period. Optionally, to ensure the accuracy of the video footage, an I-frame can be forcibly sent every 24 hours to eliminate accumulated errors.

[0089] Optionally, in static scenarios, in conventional technologies, the frame structure of video frames sent from a video transmitting device to a video receiving device is: |I|B|B|P|B|B|P|B|B|... However, in this embodiment, the frame structure of video frames sent from the video transmitting device to the video receiving device is: I-frame + substitute frame data packet, such as |I|CSI|CSI|CSI|CSI|CSI|CSI|CSI|CSI|...|I|. In conventional technologies, one GOP is sent every 2 seconds, while in this application, one CSI substitute frame (e.g., 32 bytes) is sent per second.

[0090] As can be seen from the above comparison, in static scenarios, using CSI substitute frames for video data transmission can effectively reduce the amount of data transmitted, greatly save bitrate, avoid unnecessary complex calculations and redundant frames, and achieve the transmission of high-definition video streams in weak network environments due to the smaller data size of the substitute frame data packets.

[0091] S303, when the information similarity is less than or equal to the similarity threshold, determine the fluctuation index corresponding to the video transmission device based on the preset sampling frequency and the parameter values ​​of each state parameter in different channel state information, and determine the current scene based on the fluctuation index.

[0092] If the information similarity is less than or equal to the similarity threshold, for example, when the information similarity is less than or equal to 0.98, it indicates that the current scene has changed significantly from the previously captured video scene, and the current scene of the video transmitting device is not a static scene. In this case, the fluctuation index corresponding to the video transmitting device is determined based on the preset sampling frequency and the parameter values ​​of each state parameter in the channel state information. Then, the current scene is determined based on the fluctuation index.

[0093] The state parameters may include phase shift, amplitude attenuation, and Doppler frequency shift. The fluctuation index can be obtained by weighted summation of the state parameters, for example, by weighted summation of phase shift and amplitude attenuation, or by weighted summation of amplitude attenuation and Doppler frequency shift, or by weighted summation of phase shift and Doppler frequency shift. No specific limitation is made here.

[0094] In the above method, the current scene of the video sending device is determined based on the degree of information similarity, thereby flexibly adjusting the amount of video data transmitted in different scenes, taking into account both the clarity and smoothness of the video image presented by the video receiving device, and realizing high-definition video transmission in weak wireless network environments.

[0095] In an exemplary embodiment, the determination of the above-mentioned fluctuation index is further refined. Optionally, since each state parameter includes phase offset and amplitude attenuation, the determination method of the above-mentioned fluctuation index may include: determining a first rate of change of phase offset based on a preset sampling frequency and the parameter value of phase offset in different channel state information; and determining a second rate of change of amplitude attenuation based on a preset sampling frequency and the parameter value of amplitude attenuation in different channel state information; and determining the fluctuation index corresponding to the video transmission device based on the first rate of change and the second rate of change.

[0096] The first rate of change reflects the speed of fluctuation in the video image, while the second rate of change reflects the amplitude of fluctuation in the video image. The fluctuation index can be determined by weighted summation of the first and second rates of change, or by the product of the first and second rates of change, or by the average of the first and second rates of change. No specific limitation is made here.

[0097] In one alternative embodiment, the volatility index is determined according to the following formula:

[0098] Volatility index = a×(dΦ / dt) + b×(dA / dt)

[0099] Where dΦ / dt is the first rate of change, dA / dt is the second rate of change, Φ represents the phase shift, A represents the amplitude attenuation, and t represents the time interval corresponding to the preset sampling frequency. It should be noted that the weight 'a' of the first rate of change and the weight 'b' of the second rate of change can be set according to requirements, and no specific limitation is made here, such as a=0.7 and b=0.3, etc.

[0100] By determining the fluctuation index corresponding to the video transmitting device through phase offset and amplitude attenuation, the fluctuation of the current channel state information of the video transmitting device can be accurately reflected. Then, the current scene of the video transmitting device can be accurately determined based on the fluctuation index. This allows for flexible adjustment of the amount of video data transmitted in different scenarios, taking into account both the clarity and smoothness of the video image presented by the video receiving device, and realizing high-definition video transmission in weak wireless network environments.

[0101] After determining the fluctuation index of the video transmission device, in an exemplary embodiment, determining the current scene based on the magnitude of the fluctuation index may include the following:

[0102] If the volatility index is less than the first volatility threshold, the current scenario is determined to be the first dynamic scenario.

[0103] If the volatility index is not less than the first volatility threshold and is less than the second volatility threshold, the current scenario is determined to be the second dynamic scenario.

[0104] If the volatility index is not less than the second volatility threshold and is less than the third volatility threshold, the current scenario is determined to be the third dynamic scenario.

[0105] If the volatility index is not less than the third volatility threshold, the current scenario is determined to be the fourth dynamic scenario.

[0106] The first, second, and third fluctuation thresholds can be set according to actual needs, as long as the first fluctuation threshold < the second fluctuation threshold < the third fluctuation threshold. It can be seen that the fluctuation index increases sequentially from the first to the fourth dynamic scene. In other words, the degree of change in the video image gradually increases from the first to the fourth dynamic scene. Therefore, the first dynamic scene can also be called a low dynamic scene, the second a medium dynamic scene, the third a high dynamic scene, and the fourth a very high dynamic scene.

[0107] By dividing the scene into static, low dynamic, medium dynamic, high dynamic, and extremely high dynamic scenarios, the amount of data transmitted can be flexibly adjusted according to different scenarios. This ensures that the clarity and smoothness of the video image presented by the video receiving device are maintained in different scenarios, thus achieving high-definition video transmission in weak wireless network environments.

[0108] In an exemplary embodiment, when the current scene is a first dynamic scene, video frames are sequentially sent to the video receiving device based on the order of various types of video frames in the first target GOP. The first target GOP is obtained by adding B-frames to a base GOP.

[0109] In the first dynamic scenario, by adding B-frames to the base GOP, the number of video frames contained in each base GOP is increased. Since the frame rate at which the video transmitting device transmits video frames is constant, increasing the number of video frames contained in each base GOP reduces the number of GOPs transmitted per unit time. Furthermore, since each base GOP contains one I-frame, adding B-frames reduces the number of I-frames transmitted per unit time, which can reduce the complexity of inter-frame prediction search and bit rate, thereby avoiding stuttering.

[0110] Considering that traditional encoders need to refresh frames frequently (increasing I-frames) and have extremely high bitrates, which can easily cause preview stuttering in weak network environments with low bandwidth, this embodiment reduces the I-frame refresh rate (sending frequency) by avoiding unnecessary complex calculations and redundant frames, and intersperses smaller CSI replacement frames sent in static scenes to save bitrate (especially in dynamic scenes), thus achieving the effect of transmitting high-definition video streams in weak networks (interference / poor signal).

[0111] In an exemplary embodiment, when the current scene is a second dynamic scene, video frames are sequentially sent to the video receiving device based on the order of various types of video frames in the second target GOP. The second target GOP is obtained by replacing B frames with P frames in the base GOP.

[0112] This involves converting B-frames in the base GOP to P-frames. For example, converting the base GOP [IBBPBBPBB] to the second target GOP [IPPPBBPBB].

[0113] In a Group of Pictures (GOP), the reference chain consists of the video frames between the current frame and the nearest reference frame. When the current frame is a P-frame, the reference chain consists of the video frames between the current frame and the previous reference frame. When the current frame is a B-frame, the reference chain consists of the video frames between the current frame, the previous reference frame, and the next reference frame. For example, converting a B-frame in a base GOP to a P-frame changes the reference chain from IBBP to IPPP. In dynamic scenes, if the reference frame is too far from the current frame (in the case of a long reference chain), the content in the reference frame may not accurately predict the current frame due to the rapid movement of objects, resulting in a large prediction residual and a decrease in coding efficiency. Therefore, appropriately shortening the reference chain and using closer reference frames avoids the accumulated errors caused by a long reference chain, improving prediction accuracy and resulting in a better preview experience.

[0114] In an exemplary embodiment, when the current scene is a third dynamic scene, based on the order of various video frames in the base GOP, B frames in the base GOP are skipped, and I frames and P frames are sent to the video receiving device in sequence.

[0115] When encoding the video, it still follows the basic GOP [IBBPBBPBB] encoding, but when sending video frames to the video receiving device, it forces the skipping of B frames and continues to send subsequent P frames to avoid mosaic caused by broken B frame reference chains, shorten the decoding dependency path, reduce inter-frame redundant data by more than 50%, and reduce the amount of data transmitted.

[0116] In an exemplary embodiment, when the current scene is the fourth dynamic scene, dynamically encoded frames are sequentially sent to the video receiving device. These dynamically encoded frames are video frames obtained by encoding changing image regions. The changing image regions are determined based on video frame change data between the current video frame sent by the video transmitting device and a reference frame previously sent to the video receiving device. The reference frame is either an I-frame or a P-frame.

[0117] In the fourth dynamic scene, namely the extremely high dynamic scene, the idea of ​​region refresh can be adopted, that is, only the moving region (the changing image region) is encoded to obtain a dynamically coded frame, for example, denoted as The background data of a video frame is not encoded (the video receiving device simply copies the background data of the previous reference frame during decoding), thus saving a significant amount of bitrate. Therefore, although The frequency of frame occurrence has increased (because the reference chain is shorter, so...). (Frames are inserted frequently), but each The frame size is smaller, and the compression efficiency of P-frames is based on the latest... The frame rate is increased, so the overall bit rate can be controlled.

[0118] By utilizing CSI as an "environmental dynamic indicator" to replace the traditional encoding decision mechanism that relies on pixel analysis, it guides the video encoder to adjust its inter-frame prediction (P / B frame) strategy in real time. This reduces encoding redundancy and bitrate waste caused by weak network environments (high interference, long distance) that lead to video preview stuttering, improves the smoothness of video transmission in weak networks, and can adapt to different levels of dynamic scenes. It can dynamically adjust the encoding data according to the fluctuation index, balance video quality and transmission efficiency in dynamic scenes, intelligently optimize video encoding, and protect critical dynamic video information.

[0119] Taking the first fluctuation threshold as 0.3, the second fluctuation threshold as 0.7, and the third fluctuation threshold as 1.2 as examples, the coding strategies and bandwidth savings rates for low dynamic, medium dynamic, high dynamic, and extremely high dynamic scenarios are shown in Table 2 below.

[0120] Table 2

[0121]

[0122] In the above embodiments, a continuously running low-overhead perception layer is used for environmental anomaly detection or key event capture. Perception technology is deeply coupled with video encoding / decoding and transmission control: in dynamic scenes, once a preset event is detected, the video encoding module is immediately activated; in static mode, perception information is used to replace some traditional high-overhead intelligent encoding strategies, achieving "low-overhead" bitrate optimization. In weak wireless network environments, this effectively reduces the amount of transmitted data, allowing more wireless cameras to be connected to the wireless network without affecting preview quality.

[0123] like Figure 4A The diagram illustrates the scene determination process of the video transmitting device in this embodiment. The current scene of the video transmitting device is categorized into static, low-dynamic, medium-dynamic, high-dynamic, and extremely high-dynamic scenes. When the information similarity is greater than 0.98, the current scene of the video transmitting device is determined to be a static scene, and a frame data packet is sent to the video receiving device at a preset transmission frequency. I-frames are sent to the video receiving device every specified time interval. When the information similarity is not less than 0.98, the fluctuation index corresponding to the video transmitting device is determined. When the fluctuation index is less than 0.3, the current scene of the video transmitting device is determined to be a low-dynamic scene. In this scenario, B-frames are added to the base GOP. When the fluctuation index is not less than 0.3 and less than 0.7, the current scene of the video transmitting device is determined to be a medium dynamic scene. The B-frames in the base GOP are replaced with P-frames, and the reference chain is shortened. When the fluctuation index is not less than 0.7 and less than 1.2, the current scene of the video transmitting device is determined to be a high dynamic scene. The B-frames in the base GOP are skipped, and I-frames and P-frames are sent to the video receiving device in sequence, and the reference chain is shortened. When the fluctuation index is not less than 1.2, the current scene of the video transmitting device is determined to be an extremely high dynamic scene. Dynamically encoded frames obtained by encoding the changing image region are sent to the video receiving device.

[0124] like Figure 4B The diagram shown is a schematic of the frame sequence in an embodiment of this application. When sending video data to a video receiving device using the video transmission method in this embodiment, different frames are sent depending on the current scene of the video sending device. For example, at time 0, the current scene is a static scene, and a frame data packet is sent (…). Figure 4B (Represented by C in the original text). At time 1, the current scene is a high dynamic range scene, and a P frame is sent. At time 2, the current scene is an extremely high dynamic range scene, and an I1 frame is sent. At time 6, the current scene is a low dynamic range scene, and a B frame is sent. Specifically... Figure 4B They will not be listed one by one here.

[0125] By dividing scenes into static, low-dynamic, medium-dynamic, high-dynamic, and extremely high-dynamic scenarios, an adaptive encoding strategy is designed to minimize the refresh rate of I-frames or P-frames / I-frames, extend the Group of Pictures (GOP), reduce the transmission bitrate, and ensure reliable transmission of critical streaming media data frames under weak network and limited bandwidth conditions, thereby solving the video stuttering problem.

[0126] In an exemplary embodiment, the determination of the above-mentioned fluctuation index is further refined. Optionally, the determination of the fluctuation index may include: determining the cause of the difference between different channel state information based on the difference information between different channel state information; if the cause of the difference is the cause of target intrusion, determining the fluctuation index corresponding to the video transmission device based on the preset sampling frequency and the parameter values ​​of each state parameter in different channel state information; if the cause of the difference is environmental interference or unknown cause, determining the current scene as a static scene.

[0127] In this embodiment, the video transmitting device continuously monitors CSI fluctuations. Since environmental interference around the device can cause abnormal CSI fluctuations, while the actual video image changes little, a dynamic scene-specific video data transmission scheme is unnecessary. Therefore, when CSI fluctuations are detected, i.e., the information similarity is less than or equal to the similarity threshold, the cause of the channel state information difference needs to be determined. If the cause is target intrusion (moving object activity), the dynamic scene corresponding to the current scene is determined based on the fluctuation index. If the cause is environmental interference or an unknown reason, the current scene is determined to be a static scene. In this case, even though the information similarity is less than or equal to the similarity threshold, the static scene-specific video data transmission scheme is still used.

[0128] In the above method, by determining the reasons for the differences in channel state information, the current scenario of the video transmission device can be determined, and a corresponding video data transmission scheme can be adopted, which can enhance the accuracy of video data transmission.

[0129] Since CSI changes caused by moving objects (such as people / vehicles) have unique characteristics such as Doppler frequency shift and spatial directionality, while environmental interference (such as electrical switches) often manifests as transient spikes, machine learning algorithms can be used to train a model to determine the cause of the difference based on the above characteristics. The trained model can distinguish whether the difference between multiple channel state information is caused by moving objects, environmental interference, or unknown causes. The CSI characteristics and identification criteria caused by different differences are shown in Table 3.

[0130] Table 3

[0131]

[0132] In one exemplary embodiment, such as Figure 4C As shown, when the cause of the discrepancy between multiple channel state information is unknown, the video transmitting device can also perform the following steps:

[0133] S401, send P frames to the video receiving device;

[0134] S402, after sending a first number of P frames to the video receiving device, send an I frame to the video receiving device.

[0135] Specifically, when the cause of the difference is unknown, in order to balance the integrity of the video image and the amount of data transmitted, P frames are sent to the video receiving device. Furthermore, after sending the first number of P frames, in order to ensure the accuracy of the video image, I frames are sent to the video receiving device to refresh the current background image.

[0136] For example, if P frames are sent 5 times in a row, I frames are sent to refresh the current background screen. If there are no fluctuations in CSI or fluctuations due to environmental interference, replacement frame data packets are sent continuously (continuously sending very low amounts of CSI data).

[0137] Optionally, a P-frame may be sent to the video receiving device only when a second number of CSI fluctuations caused by unknown reasons occur consecutively; and then, an I-frame may be sent to the video receiving device every time a first number of P-frames are sent to the video receiving device.

[0138] By following the steps above, sending one I-frame after sending the first number of P-frames refreshes the current background image, ensuring the accuracy of the video image.

[0139] like Figure 5A The diagram illustrates a method for determining the cause of difference in an embodiment of this application. First, the channel state information is preprocessed (denoising / filtering) to obtain channel state features. Then, the channel state features are extracted to obtain time-domain and frequency-domain features. Next, the time-domain and frequency-domain features are fused to obtain channel fusion features. Finally, the channel fusion features are input into a trained difference cause classification model. When the difference cause output by the difference cause classification model is a moving object activity, the fluctuation index corresponding to the video transmitting device is determined to determine the dynamic scene corresponding to the video transmitting device. When the difference cause is no fluctuation or environmental interference, the current scene is determined to be a static scene, and a generation frame data packet is sent. When the difference cause is an unknown cause, a P-frame is sent to the video receiving device, and an I-frame is sent to the video receiving device after continuously sending a first number of P-frames.

[0140] Based on the above embodiments, in an exemplary embodiment, such as Figure 5BAs shown, taking a camera as the video transmitting device as an example, the method may include the following steps:

[0141] S501, when the network quality of the wireless network accessed by the video transmitting device is less than the quality threshold, acquire multiple channel state information of the wireless signal transmitted between the video transmitting device and the video receiving device according to the preset sampling frequency.

[0142] S502, when multiple channel state information is obtained, determine the information similarity between the multiple channel state information;

[0143] S503, if the information similarity is greater than the similarity threshold, determine that the current scene of the video sending device is a static scene;

[0144] S504, when the current scene is a static scene, sends frame data packets to the video receiving device at a preset sending frequency, and sends I frames to the video receiving device at specified intervals.

[0145] S505, when the information similarity is less than or equal to the similarity threshold, determine the cause of the difference between the multiple channel state information based on the difference information between the multiple channel state information.

[0146] S506, if the difference is due to environmental interference or unknown reasons, determine that the current scene is a static scene, send P frames to the video receiving device, and send I frames to the video receiving device after sending a first number of P frames.

[0147] S507, if the cause of the difference is the target intrusion, determine the first rate of change of phase offset based on the preset sampling frequency and the parameter value of phase offset; and determine the second rate of change of amplitude attenuation based on the preset sampling frequency and the parameter value of amplitude attenuation in different channel state information; and determine the fluctuation index corresponding to the video transmission device based on the first rate of change and the second rate of change.

[0148] S508, if the volatility index is less than the first volatility threshold, determine the current scenario as the first dynamic scenario; if the volatility index is not less than the first volatility threshold and less than the second volatility threshold, determine the current scenario as the second dynamic scenario; if the volatility index is not less than the second volatility threshold and less than the third volatility threshold, determine the current scenario as the third dynamic scenario; if the volatility index is not less than the third volatility threshold, determine the current scenario as the fourth dynamic scenario.

[0149] S509, when the current scenario is the first dynamic scenario, each video frame is sent to the video receiving device sequentially based on the order of various video frames in the first target GOP; when the current scenario is the second dynamic scenario, each video frame is sent to the video receiving device sequentially based on the order of various video frames in the second target GOP; when the current scenario is the third dynamic scenario, based on the order of various video frames in the base GOP, skipping the B frames in the base GOP, I frames and P frames are sent to the video receiving device sequentially; when the current scenario is the fourth dynamic scenario, dynamically encoded frames are sent to the video receiving device sequentially.

[0150] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0151] Based on the same inventive concept, this application also provides a video transmission apparatus for implementing the video transmission method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more video transmission apparatus embodiments provided below can be found in the limitations of the video transmission method described above, and will not be repeated here.

[0152] In one exemplary embodiment, such as Figure 6 As shown, a video transmission device is provided, including: an acquisition module 601, a determination module 602, and a transmission module 603, wherein:

[0153] The acquisition module 601 is used to acquire multiple channel state information of the wireless signal transmitted between the video transmitting device and the video receiving device according to a preset sampling frequency when the network quality of the wireless network accessed by the video transmitting device is less than a quality threshold.

[0154] The determination module 602 is used to determine the current scene of the video transmission device based on the information similarity between multiple channel state information.

[0155] The transmitting module 603 is used to transmit video data to the video receiving device using a video data transmission scheme that matches the current scene. The video data transmission scheme includes a scheme that transmits video data based on changes in video frames, or a scheme that transmits video data by adjusting the number of B frames transmitted in the basic GOP. The basic GOP is the GOP used when the network quality of the wireless network accessed by the video transmitting device is not less than the quality threshold.

[0156] In an exemplary embodiment, the determining module 602 is specifically used to: determine the information similarity between multiple channel state information based on the parameter values ​​of each state parameter in different channel state information; determine that the current scene of the video transmitting device is a static scene when the information similarity is greater than the similarity threshold; and determine the fluctuation index corresponding to the video transmitting device based on the preset sampling frequency and the parameter values ​​of each state parameter in the channel state information when the information similarity is less than or equal to the similarity threshold, and determine the current scene based on the fluctuation index.

[0157] In an exemplary embodiment, the determining module 602 is specifically configured to: determine a first rate of change of phase offset based on a preset sampling frequency and parameter values ​​of phase offset in different channel state information; and determine a second rate of change of amplitude attenuation based on a preset sampling frequency and parameter values ​​of amplitude attenuation in different channel state information; and determine the fluctuation index corresponding to the video transmission device based on the first rate of change and the second rate of change.

[0158] In an exemplary embodiment, the determining module 602 is specifically configured to: determine the current scenario as a first dynamic scenario when the volatility index is less than a first volatility threshold; determine the current scenario as a second dynamic scenario when the volatility index is not less than the first volatility threshold and is less than a second volatility threshold; determine the current scenario as a third dynamic scenario when the volatility index is not less than the second volatility threshold and is less than a third volatility threshold; and determine the current scenario as a fourth dynamic scenario when the volatility index is not less than the third volatility threshold.

[0159] In an exemplary embodiment, the determining module 602 is specifically used to: determine the cause of the difference between different channel state information based on the difference information between different channel state information; if the cause of the difference is the cause of target intrusion, determine the fluctuation index corresponding to the video sending device based on the preset sampling frequency and the parameter values ​​of each state parameter in the different channel state information; if the cause of the difference is the cause of environmental interference or unknown cause, determine that the current scene is a static scene.

[0160] In an exemplary embodiment, when the current scene is a static scene, the sending module 603 is specifically used to: send a frame data packet to the video receiving device according to a preset sending frequency; wherein, the frame data packet includes visual attribute adjustment parameters, which are determined based on the video image change data between the current video frame of the video sending device and the reference frame last sent to the video receiving device, and the reference frame is an I-frame or a P-frame; and send an I-frame to the video receiving device every specified time interval.

[0161] In an exemplary embodiment, when the cause of the difference between multiple channel state information is unknown, the sending module 603 is further configured to: send P frames to the video receiving device; and send I frames to the video receiving device for each first number of P frames sent to the video receiving device.

[0162] In an exemplary embodiment, the sending module 603 sends video data to the video receiving device in any of the following ways: When the current scene is a first dynamic scene, video frames are sent sequentially to the video receiving device based on the order of various types of video frames in a first target GOP; wherein the first target GOP is obtained by adding B-frames to a base GOP; when the current scene is a second dynamic scene, video frames are sent sequentially to the video receiving device based on the order of various types of video frames in a second target GOP; wherein the second target GOP is obtained by replacing B-frames in the base GOP with P-frames; when the current scene is a third dynamic scene, I-frames and P-frames are sent sequentially to the video receiving device based on the order of various types of video frames in the base GOP, skipping B-frames in the base GOP; when the current scene is a fourth dynamic scene, dynamically encoded frames are sent sequentially to the video receiving device; wherein the dynamically encoded frame is a video frame obtained by encoding a changing image region, and the changing image region is determined based on the video frame change data between the current video frame of the video sending device and the reference frame previously sent to the video receiving device, where the reference frame is an I-frame or a P-frame.

[0163] Each module in the aforementioned video transmission device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0164] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides deterministic and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores video data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a video transmission method.

[0165] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a video transmission method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0166] Those skilled in the art will understand that Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0167] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the various method embodiments of the video transmission method described above.

[0168] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the various method embodiments of the video transmission method described above.

[0169] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the various method embodiments of the video transmission method described above.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum deterministic data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0172] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of video transmission, characterized by, The method is applied to a video sending device, and comprises the following steps: In a case where network quality of a wireless network accessed by the video sending device is less than a quality threshold, acquiring a plurality of channel state information of wireless signals transmitted between the video sending device and a video receiving device according to a preset sampling frequency; Determining a current scene of the video sending device according to information similarity between the plurality of channel state information; Sending video data to the video receiving device by using a video data sending scheme matched with the current scene, wherein the video data sending scheme comprises a scheme of sending video data based on video picture change data or a scheme of sending video data by adjusting a sending quantity of B frames in a basic group of pictures (GOP), wherein the basic GOP is a GOP used in a case where network quality of the wireless network accessed by the video sending device is not less than the quality threshold.

2. The method of claim 1, wherein, The determining of the current scene of the video sending device according to the information similarity between the plurality of channel state information comprises: Determining the information similarity between the plurality of channel state information according to parameter values of state parameters in different channel state information; In a case where the information similarity is greater than a similarity threshold, determining that the current scene of the video sending device is a static scene; In a case where the information similarity is less than or equal to the similarity threshold, determining a fluctuation index corresponding to the video sending device according to the preset sampling frequency and the parameter values of the state parameters in different channel state information, and determining the current scene according to the fluctuation index.

3. The method of claim 2, wherein, The state parameters comprise phase offset and amplitude attenuation; and the determining of the fluctuation index corresponding to the video sending device according to the preset sampling frequency and the parameter values of the state parameters in different channel state information comprises: Determining a first change rate of the phase offset according to the preset sampling frequency and the parameter values of the phase offset in different channel state information, and determining a second change rate of the amplitude attenuation according to the preset sampling frequency and the parameter values of the amplitude attenuation in different channel state information; and Determining the fluctuation index corresponding to the video sending device according to the first change rate and the second change rate.

4. The method of claim 2, wherein, The determining of the current scene according to the fluctuation index comprises: In a case where the fluctuation index is less than a first fluctuation threshold, determining that the current scene is a first dynamic scene; In a case where the fluctuation index is not less than the first fluctuation threshold and less than a second fluctuation threshold, determining that the current scene is a second dynamic scene; In a case where the fluctuation index is not less than the second fluctuation threshold and less than a third fluctuation threshold, determining that the current scene is a third dynamic scene; In a case where the fluctuation index is not less than the third fluctuation threshold, determining that the current scene is a fourth dynamic scene.

5. The method of claim 2, wherein, The determining of the fluctuation index corresponding to the video sending device according to the preset sampling frequency and the parameter values of the state parameters in different channel state information comprises: Determining a difference reason between different channel state information according to difference information between the different channel state information; and In a case where the difference is caused by a target invasion, a fluctuation index corresponding to the video sending device is determined according to the preset sampling frequency and parameter values of state parameters in different channel state information; In a case where the difference is caused by environmental interference or unknown reasons, the current scene is determined to be a static scene.

6. The method according to any one of claims 2-5, characterized in that, In a case where the current scene is a static scene, the video data sending scheme matched with the current scene is adopted to send video data to the video receiving device, including: frame data packets are sent to the video receiving device at a preset sending frequency; wherein the frame data packets include visual attribute adjustment parameters, and the visual attribute adjustment parameters are determined based on video picture change data between a current video frame of the video sending device and a reference frame last sent to the video receiving device, and the reference frame is an I frame or a P frame; I frames are sent to the video receiving device every specified time length.

7. The method of claim 6, wherein, In a case where the difference is caused by unknown reasons, the method further includes: P frames are sent to the video receiving device; In a case where a first number of P frames are sent to the video receiving device, I frames are sent to the video receiving device.

8. The method of claim 4, wherein, The video data sending scheme matched with the current scene is adopted to send video data to the video receiving device, including any one of the following: In a case where the current scene is the first dynamic scene, video frames are sequentially sent to the video receiving device based on an order of video frames in a first target GOP; wherein the first target GOP is obtained by adding B frames to the basic GOP; In a case where the current scene is the second dynamic scene, video frames are sequentially sent to the video receiving device based on an order of video frames in a second target GOP; wherein the second target GOP is obtained by replacing B frames in the basic GOP with P frames; In a case where the current scene is the third dynamic scene, I frames and P frames are sequentially sent to the video receiving device based on an order of video frames in the basic GOP, and B frames in the basic GOP are skipped; In a case where the current scene is the fourth dynamic scene, dynamic coding frames are sequentially sent to the video receiving device; wherein the dynamic coding frames are video frames obtained by coding changed image regions, and the changed image regions are determined based on video picture change data between a current video frame of the video sending device and a reference frame last sent to the video receiving device, and the reference frame is an I frame or a P frame.

9. A video transmission apparatus, characterized by comprising: The device is applied to a video sending device, and includes: The acquisition module is configured to, in a case where network quality of a wireless network accessed by the video sending device is less than a quality threshold, acquire, according to a preset sampling frequency, a plurality of channel state information of wireless signals transmitted between the video sending device and a video receiving device; The determination module is configured to determine a current scene of the video sending device according to information similarity between the plurality of channel state information; The sending module is configured to send video data to the video receiving device by using a video data sending scheme matched with the current scene, wherein the video data sending scheme comprises a scheme of sending video data based on video picture change data or a scheme of sending video data by adjusting the sending quantity of B frames in a basic GOP. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-9. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

Images