Video transmission method, apparatus, computing device, and computer-readable storage medium

Abstract

Embodiments of the present application disclose a video transmission method and device, a computing device and a computer readable storage medium, comprising: obtaining a video stream to be transmitted; detecting target object information corresponding to each image frame based on a preset neural network and motion vectors corresponding to each image frame in the video stream; estimating a current available bandwidth according to a local sending time and transmission feedback information returned by a receiving end; determining a quantization parameter and a code rate weight corresponding to each coding unit based on the current available bandwidth and the target object information corresponding to each image frame; and transmitting the video stream according to the quantization parameter and the code rate weight. The video transmission scheme of the present application can simultaneously consider both the picture imaging quality and the data transmission efficiency, and is suitable for various complex and variable live transmission environments.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a video transmission method, apparatus, computing device, and computer-readable storage medium. Background Technology

[0002] Video transmission technology has now been widely adopted in numerous industries, including security monitoring, vehicle imaging, remote inspection, and home security, playing a crucial role in daily security management, on-site condition monitoring, and post-event video tracing. As application scenarios continue to expand, users are placing increasingly higher demands on the smoothness of video transmission, the clarity of the image, and its adaptability to complex network environments.

[0003] Currently, the common video transmission and processing methods in the industry mostly involve uniformly compressing the captured complete image using fixed encoding parameters, relying on simple detection methods to determine the basic network status, and then encapsulating the data according to preset rules and transmitting it outwards, thereby realizing the cross-terminal transmission of image data.

[0004] However, current video transmission solutions employ rigid encoding control modes. When network resources are abundant, even irrelevant background areas are encoded with fine detail, resulting in unnecessary bandwidth consumption. When network bandwidth is limited, the entire image data can only be compressed, causing key details to become blurry and reducing the actual usability of the image. It is evident that current video transmission solutions lack dynamic adaptability, making it difficult to simultaneously balance image quality and data transmission efficiency, and thus failing to adapt well to various complex and ever-changing on-site transmission environments. Summary of the Invention

[0005] This application provides a video transmission method, apparatus, computing device, and computer-readable storage medium that can simultaneously balance image quality and data transmission efficiency, adapting to various complex and ever-changing on-site transmission environments.

[0006] In a first aspect, embodiments of this application provide a video transmission method, including: Obtain the video stream to be transmitted; Based on a preset neural network and the motion vectors corresponding to each image frame in the video stream, the target object information corresponding to each image frame is detected. Based on the local transmission time and the transmission feedback information returned by the receiver, estimate the currently available bandwidth; Based on the currently available bandwidth and the target object information corresponding to each image frame, determine the quantization parameters and bitrate weights corresponding to each coding unit; The video stream is transmitted according to the quantization parameters and bitrate weights.

[0007] Secondly, embodiments of this application provide a video transmission device, including: The acquisition module is used to acquire the video stream to be transmitted; The detection module is used to detect the target object information corresponding to each image frame based on a preset neural network and the motion vectors corresponding to each image frame in the video stream. The estimation module is used to estimate the currently available bandwidth based on the local transmission time and the transmission feedback information returned by the receiver. The determination module is used to determine the quantization parameters and bitrate weights corresponding to each coding unit based on the currently available bandwidth and the target object information corresponding to each image frame; The transmission module is used to transmit the video stream according to the quantization parameters and bitrate weights.

[0008] Thirdly, embodiments of this application also provide a computing device, including: At least one processor; and A memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of any of the video transmission methods described above.

[0009] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the video transmission method as described in any of the preceding claims.

[0010] This application provides a video transmission method, apparatus, computing device, and computer-readable storage medium. After acquiring the video stream to be transmitted, based on a preset neural network and the motion vectors corresponding to each image frame in the video stream, target object information corresponding to each image frame is detected. Next, based on the local transmission time and transmission feedback information returned by the receiving end, the current available bandwidth is estimated. Then, based on the current available bandwidth and the target object information corresponding to each image frame, the quantization parameters and bitrate weights corresponding to each encoding unit are determined. Finally, the video stream is transmitted according to the quantization parameters and bitrate weights. In the video transmission scheme provided in this application, the network bandwidth status and target object information are linked to differentiate the quantization parameters and bitrate weights for different encoding units, eliminating the need for a uniform encoding method to process the entire frame. Finally, the video stream is transmitted according to the allocated encoding parameters. This reduces bitrate loss in invalid areas and improves bandwidth utilization efficiency when network conditions are good, while prioritizing the clarity of key image details in bandwidth-constrained scenarios, effectively balancing image quality and data transmission efficiency, and better adapting to complex and ever-changing network transmission scenarios. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is an application environment diagram of the video transmission method provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the video transmission method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the video transmission device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the computing device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," "third," and "fourth" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0015] This application provides a video transmission method, apparatus, computing device, and computer-readable storage medium, which will be described in detail below.

[0016] In the embodiments of the video transmission method of this application, a video transmission device is used as the execution subject. For simplicity and ease of description, this execution subject will be omitted in subsequent method embodiments. The video transmission device is applied to computing devices equipped with cameras, such as mobile phones, tablets, PCs, smart TVs, vehicle terminals, network cameras, high-definition cameras, and other terminal devices. Optionally, in some embodiments of this application, please refer to... Figure 1 , Figure 1 This diagram illustrates the application environment of the video transmission method provided in this embodiment. The video transmission method can be applied to a video transmission system. This video transmission system may include a terminal 110 and a server 120. The terminal 110 and server 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal; the mobile terminal can be at least one of a mobile phone, tablet computer, or laptop computer. The server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers. The server 120 can specifically be used for: The video stream to be transmitted is acquired. Based on a preset neural network and the motion vectors corresponding to each image frame in the video stream, the target object information corresponding to each image frame is detected. Based on the local transmission time and the transmission feedback information returned by the receiver, the current available bandwidth is estimated. Based on the current available bandwidth and the target object information corresponding to each image frame, the quantization parameters and bit rate weights corresponding to each coding unit are determined. The video stream is transmitted according to the quantization parameters and bit rate weights.

[0017] The video transmission scheme provided in this application integrates network bandwidth status and target object information to differentiate quantization parameters and bitrate weights for different encoding units, eliminating the need for a uniform encoding method to process entire frames. Finally, the video stream is transmitted according to the allocated encoding parameters. This approach reduces bitrate loss in invalid areas and improves bandwidth utilization efficiency when network conditions are good, while prioritizing the clarity of key image details in bandwidth-constrained scenarios. It effectively balances image quality and data transmission efficiency, better adapting to complex and ever-changing network transmission scenarios.

[0018] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0019] A video transmission method includes: acquiring a video stream to be transmitted; detecting target object information corresponding to each image frame based on a preset neural network and motion vectors corresponding to each image frame in the video stream; estimating the current available bandwidth based on the local transmission time and transmission feedback information returned by the receiving end; determining the quantization parameters and bitrate weights corresponding to each encoding unit based on the current available bandwidth and the target object information corresponding to each image frame; and transmitting the video stream according to the quantization parameters and bitrate weights.

[0020] Please see Figure 2 , Figure 2 This is a flowchart of a video transmission method provided in an embodiment of this application. The video transmission method may specifically include the following steps: 101. Obtain the video stream to be transmitted.

[0021] The video stream to be transmitted is a continuous image data stream composed of multiple frames of images captured by the acquisition device and arranged in chronological order, carrying original image information such as scene, object outline, and target shape.

[0022] For example, the device's built-in image acquisition component is activated to capture images of the scene in real time, continuously acquiring raw image data. Then, according to the device's preset resolution and frame rate parameters, independent image data is acquired. Next, all the acquired single-frame images are sorted and organized in chronological order according to the order in which the images were generated. Finally, all sorted image frames are aggregated and integrated to form a coherent and complete image data stream, resulting in the video stream to be transmitted.

[0023] Taking a security camera in a park as an example, after the camera is powered on, it captures scenes such as roads and walls in the park, continuously capturing static images at a frequency of 25 frames per second. Then, all the images are arranged in the order of capture, and the individual scattered images are combined to finally form a video stream of dynamic images of the park.

[0024] 102. Based on a preset neural network and the motion vectors corresponding to each image frame in the video stream, detect the target object information corresponding to each image frame.

[0025] The pre-set neural network refers to a pre-trained target detection neural network model deployed on the local device. It has the ability to identify targets in a specified scene and can determine the category and location of objects within the scene. Motion vectors refer to the displacement data formed by the positional shift of pixels and objects between two adjacent frames in the video stream, reflecting the direction and magnitude of object movement. Target object information is the data related to the detected target, including the coordinates of the target's location, its category, and its outline.

[0026] For example, specifically, image frame data is extracted frame by frame from the video stream to be transmitted as the object of detection processing. Then, a preset neural network is invoked to perform preliminary target detection on a single frame image, roughly locate the target present in the image and determine the target type. Then, the pixel position difference between adjacent image frames is calculated to obtain the motion vector corresponding to each image frame. Next, the preliminary detection results are verified and corrected by combining the object movement trajectory reflected by the motion vector, such as removing falsely detected targets and filling in missing targets. Finally, the corrected target position and category data are summarized to obtain the target object information corresponding to a single frame image.

[0027] Taking road security monitoring as an example, after the camera captures and uploads continuous road images, the road security monitoring system uses neural networks to identify vehicles and pedestrians in the images, then uses inter-frame motion vectors to determine the trajectory of objects, corrects target positioning deviations caused by shaking and occlusion, and finally outputs target object information such as the location and category of vehicles and pedestrians.

[0028] Optionally, in some embodiments of this application, the step "detecting target object information corresponding to each image frame based on a preset neural network and the motion vectors corresponding to each image frame in the video stream" may specifically include: Based on a pre-defined neural network, an object bounding box corresponding to the target object is generated in the first image frame of the video stream. Based on the object bounding box and the motion vectors corresponding to each image frame except the first image frame, output the target object information corresponding to each image frame.

[0029] The object bounding box is a bounding box drawn with the detected target in the image as the center. It is used to accurately mark the target's location and coverage area within the image.

[0030] For example, specifically, the first image frame is extracted from the video stream to be transmitted and used as the initial detection frame. Then, a locally deployed pre-defined neural network is invoked to perform target recognition operations on the first frame, identifying various targets within the frame. Based on the recognition results, a bounding box is drawn around each valid target, generating the object bounding box of the target in the first frame. Next, each subsequent image frame is read sequentially, and the pixel displacement changes between adjacent frames are calculated to obtain the motion vector for each image frame. Using the object bounding box generated in the first frame as a reference, and combining the object's movement amplitude and direction as reflected in the motion vector, the latest position of the target in the current frame is calculated. Finally, based on the calculation results, the target range is locked, and relevant information such as the target's location and category is compiled, ultimately outputting the target object information for each image frame.

[0031] Taking intersection surveillance video as an example, the first frame of the video shows moving vehicles and pedestrians. The system identifies the two types of targets through a preset neural network and draws corresponding bounding boxes around the vehicles and pedestrians respectively, locking the initial target area. Subsequently, it continuously captures subsequent images, calculates the motion vectors obtained from the changes between frames, and continuously calculates the new positions of the targets by referring to the bounding boxes of the first frame and combining the movement trajectories of the vehicles and pedestrians. It continuously outputs the target object information such as the position and category of vehicles and pedestrians in each frame.

[0032] Optionally, in some embodiments of this application, the step "outputting the target object information corresponding to each image frame based on the object frame and the motion vectors corresponding to each image frame excluding the first image frame" may specifically include: Obtain the motion vectors corresponding to each image frame except the first image frame; The image frame currently being processed is designated as the current frame. Using the object box as a reference, the current frame is projected forward based on the motion vector to obtain the candidate object box corresponding to the current frame; Detect whether the candidate object box meets the preset conditions; Based on the detection results, output the target object information corresponding to each image frame. Forward projection is a computational process that uses the standard object box determined in the first frame as a reference, combines the displacement parameters corresponding to the motion vectors between frames, and maps the position and range of the reference object box to the current frame according to the target's movement pattern. Essentially, it is a trajectory prediction of the target's position. The candidate object box is a temporary bounding box obtained through forward projection, which is a predicted bounding box of the target's position and coverage area in the current frame.

[0033] The preset conditions are pre-configured validity verification rules used to determine whether the candidate object box truly corresponds to the valid target. These include: the candidate box size change is within a reasonable range, the pixel features within the box match the original target features, the box position does not deviate significantly from the normal motion trajectory, and there are no abnormal situations such as crossing the screen boundary.

[0034] For example, specifically, for all image frames in the video stream except the first frame, the pixel displacement relationship between adjacent frames is calculated frame by frame, and the motion vectors corresponding to each frame are obtained sequentially. Then, the image frames to be processed are selected sequentially and defined as the current frame to start single-frame target tracking processing. Next, using the standard object box generated in the first frame as a reference, a forward projection operation is performed in combination with the motion vector corresponding to the current frame to calculate the estimated range of the target in the current frame, obtaining candidate object boxes. According to preset verification rules, each candidate object box is checked to see if it meets preset conditions. If the candidate object box meets the preset conditions, it is determined as the official target box of the current frame, and target object information is generated and output in combination with the target category, position, etc.; if it does not meet the preset conditions, it is determined that the target is lost or is an invalid region, and corresponding target object information with missing or invalid targets is output. Taking a road security monitoring scenario as an example, the first frame of the surveillance camera's video stream has already generated an object bounding box surrounding the moving vehicle through a neural network. For each subsequent frame, the system first calculates the motion vectors generated by the vehicle's movement between frames; then, each frame is sequentially set as the current frame, and using the vehicle object bounding box of the first frame as a reference, it performs forward projection based on the motion vectors to calculate the vehicle's position in the current frame, generating candidate object bounding boxes. The system performs verification according to preset conditions, such as requiring the aspect ratio of the candidate bounding box to match the vehicle's features, the texture features within the bounding box to be consistent with the original vehicle, and the position to be normally offset along the lane direction. If all candidate bounding boxes meet the criteria, the box is confirmed as the vehicle target bounding box, and the target object information such as the vehicle's position and category is output; if the vehicle is completely obscured by roadside trees, the candidate bounding box feature matching fails and the preset conditions are not met, then the target is determined to have temporarily disappeared, and the corresponding target object information is output simultaneously.

[0035] Optionally, in some embodiments of this application, the step "detecting whether the candidate object box meets the preset conditions" may specifically include: Calculate the consistency of motion vectors within the candidate object bounding box; Get the frame interval between the current frame and the last time the neural network was run; When the consistency level is greater than or equal to the preset threshold and the frame interval is less than the preset time value, the object frame is determined to meet the preset conditions.

[0036] The consistency of motion vectors refers to the degree of uniformity and similarity in the direction of movement and displacement magnitude of motion vectors at various local locations within the area covered by the candidate object bounding box. For example, the overall motion state of a real single target (such as a vehicle or pedestrian) tends to be consistent, and the differences in motion vectors at different locations within the area are small, indicating a high degree of consistency. If the candidate bounding box corresponds to the background, interfering pixels, or scattered motion areas, the vector direction and displacement magnitude are disordered, indicating a low degree of consistency.

[0037] The system locks onto the candidate bounding box to be verified in the current frame, extracts the motion vector data corresponding to all local regions within the bounding box, performs statistical calculations on all motion vectors within the box, analyzes the differences in vector direction and displacement, and calculates the consistency degree of the motion vectors. Simultaneously, it obtains the frame interval between the current frame and the last time the preset neural network was used for target detection. The calculated consistency degree is compared with a pre-set threshold, and the frame interval is compared with a preset time value. If the consistency degree of the motion vectors is greater than or equal to the preset threshold, and the frame interval is less than the preset time value, then the candidate bounding box is determined to meet the preset conditions; if either condition is not met, then the candidate bounding box is determined to not meet the preset conditions.

[0038] Taking a road security monitoring scenario as an example, the first frame of the video generates object boxes of vehicles on the road through a neural network, and subsequent frames obtain candidate object boxes corresponding to the vehicles through forward projection. The system first extracts the motion vectors of all pixel regions within the candidate box, calculates the overall uniformity of the vectors, and obtains a consistency value; then it counts the time interval between the current frame and the last time the neural network detection was started. Assume that the system presets a vector consistency threshold of 0.7 and a preset time value of 1 second. If the calculated consistency value is 0.8 (greater than or equal to the threshold) and the frame interval is 0.4 seconds (less than the preset time value), then the candidate object box is determined to meet the preset conditions and is confirmed as a valid target box; if the vehicle is obscured by roadside greenery, a large number of background pixels are mixed in the candidate box, the motion vectors are messy, and the consistency value is only 0.3, even if the frame interval meets the requirements, the candidate object box will be determined not to meet the preset conditions.

[0039] 103. Estimate the available bandwidth based on the local transmission time and the transmission feedback information returned by the receiver.

[0040] Local transmission time refers to the timestamp recorded in real time by the local system when the video sending device pushes the video stream to the transmission link. Transmission feedback information is the status data sent back to the sending end by the receiving end after successfully receiving the video stream. It includes the stream reception time, number of lost packets, frame arrival integrity, and link anomaly alerts, reflecting the actual operating status of the transmission link. Current available bandwidth is the maximum effective rate at which the current network transmission link can stably carry video data transmission, representing the network's current actual transmission capacity.

[0041] For example, specifically, each time the sending end pushes a video stream segment, it records the local transmission time corresponding to that segment and stores it in association with the stream data. Simultaneously, it monitors the communication link, receiving real-time transmission feedback information from the receiving end. Then, it extracts the stream reception time from the feedback information and combines it with the corresponding local transmission time to calculate transmission metrics such as one-way data transmission latency and latency fluctuation. Specifically, it can read packet loss statistics and data integrity data from the feedback information to calculate the real-time transmission capacity of the link, thereby obtaining the currently available bandwidth.

[0042] Taking the transmission of video from a security camera to a monitoring platform as an example, the camera, as the sender, records the local transmission time for each frame of video stream it sends. Upon receiving the stream, the remote monitoring platform (receiver) immediately sends its own reception time and information such as whether packet loss occurred back to the camera as transmission feedback. The camera compares the transmission and reception times to calculate the data transmission latency, and then, combined with the packet loss situation, comprehensively evaluates the link quality. Based on a preset algorithm, it calculates the maximum transmission rate that the network can currently support, which is the current available bandwidth. If the link is smooth and latency and packet loss are low, the calculated available bandwidth value is higher; if the network is congested and packet loss increases, the calculated available bandwidth will decrease accordingly.

[0043] Optionally, in some embodiments of this application, the step of "estimating the currently available bandwidth based on the local transmission time and the transmission feedback information returned by the receiver" may specifically include: Obtain the receiving timestamp, number of lost packets, and latency jitter data from the transmission feedback information returned by the receiving end; Calculate the one-way delay value of the current transmission based on the local sending time and the receiving timestamp; Based on the one-way latency and packet loss count, estimate the current available bandwidth.

[0044] The receive timestamp is a standard timestamp generated and recorded by the receiver's local system when the receiver completely receives the corresponding video data packet or image frame stream. It is bound to the sender's local transmission time and is the basic data for calculating transmission duration. Packet loss count is the total number of video data packets or frames that the sender normally transmits but the receiver fails to receive within the statistical period. Latency jitter data is the change in the difference between multiple consecutive sets of one-way delays, used to characterize the fluctuation range of network transmission delay; the larger the jitter value, the more unstable the network transmission state. One-way delay value is the one-way transmission time consumed for a single video data stream to travel from the sender to the receiver via the transmission link, excluding additional time spent on data return and processing.

[0045] For example, specifically, the system continuously receives transmission feedback information from the receiving end, parses the feedback data, and extracts the receiving timestamp, packet loss count, and latency jitter data. Then, it retrieves the local transmission time matching the current transmission data and, combined with the extracted receiving timestamp, calculates the one-way latency value corresponding to this data transmission. Using the calculated one-way latency value and the statistically obtained packet loss count as the core basis, and combining this with the latency jitter data, the link status is comprehensively evaluated, and the currently available bandwidth is estimated according to a preset bandwidth estimation rule.

[0046] Taking the transmission of video from a park security camera to a backend monitoring platform as an example, the camera, as the sender, records the local transmission time for each frame of video data it sends. After receiving the data, the remote monitoring platform generates a reception timestamp and calculates the number of packet losses and latency fluctuations during this transmission segment, sending this information back to the camera. The camera parses the feedback information to obtain the corresponding data and calculates the one-way latency using the local transmission time and reception timestamp. It then combines this latency value, the number of packet losses, and the magnitude of latency jitter to comprehensively assess the current network capacity. For example, if the latency is low, the number of packet losses is zero, and the latency jitter is minimal, the network condition is considered good, and a higher available bandwidth is estimated. If the latency increases significantly and the number of packet losses is high, network congestion is determined, and a lower available bandwidth is estimated accordingly.

[0047] 104. Based on the available bandwidth and the target object information corresponding to each image frame, determine the quantization parameters and bit rate weights corresponding to each coding unit.

[0048] In this context, a coding unit is the smallest processing block obtained by dividing a single frame of image according to the coding standard. It is the basic granularity for video coding, parameter configuration, and bitrate allocation. A frame of image is divided into multiple independent coding units, and all coding control operations are performed on a unit-by-unit basis. The quantization parameter is the core parameter controlling the coding compression intensity. A larger parameter value results in a higher degree of image compression and less transmission bitrate, but also a more significant loss of image detail; a smaller value results in a lower degree of compression, more complete preservation of image detail, but also consumes more bitrate resources. The bitrate weight is a proportional coefficient used to allocate overall bitrate resources, representing the share of the total bitrate that a single coding unit can occupy. A higher weight allocates more bitrate resources, ensuring better image quality in that area; a lower weight allocates less bitrate, allowing for a more appropriate increase in compression intensity. For example, specifically, based on the estimated current available bandwidth, the current network operating status is divided according to preset rules, distinguishing different operating conditions such as smooth network, moderate network, and congested network. The target object information corresponding to each image frame is retrieved, and the image frame is divided into object regions containing the target object, as well as background regions. Combining information such as the type and importance level of the target object, the priority of each object region and background region is determined sequentially. Simultaneously, based on the determined network operating status and region priority, a corresponding bitrate weight is assigned to each coding unit within the image. Finally, considering the current network operating status and the bitrate weight of each coding unit, the quantization parameters of each coding unit are adjusted accordingly, completing the configuration of all coding parameters.

[0049] Taking a road monitoring scenario as an example, the system first determines that the current network is congested based on the measured available bandwidth. Then, combined with target object information, it classifies the areas where vehicles and pedestrians are located in the image as high-priority object areas, and roads, green belts, etc., as low-priority background areas. For encoding units in high-priority areas, a higher bitrate weight is assigned, while the quantization parameter is lowered to reduce the compression level and ensure the clarity of the target image. For encoding units in background areas, a lower bitrate weight is assigned, and the quantization parameter is appropriately increased for compression to save bandwidth resources. If the bandwidth recovers and the network returns to a smooth state, the compression intensity of each encoding unit is reduced overall to further improve the visual effect of the entire image.

[0050] Optionally, in some embodiments of this application, the step "determining the quantization parameters and bitrate weights corresponding to each coding unit based on the currently available bandwidth and the target object information corresponding to each image frame" may specifically include: Network operating status is determined based on currently available bandwidth; In each image frame, identify the object region containing the target object and the background region excluding the object region; Determine the priority of each object region based on the target object information; Based on priority and network operating status, assign bitrate weights to each coding unit in the image frame; Adjust the quantization parameters corresponding to each coding unit based on the network operating status and bitrate weight.

[0051] Network operating status is a network condition level determined based on real-time calculated available bandwidth. It is used to intuitively distinguish the strength of the current network transmission capacity. In some embodiments of this application, it can be divided into levels such as smooth network, normal network, and congested network, serving as the overall basis for coordinating coding strategies and resource allocation. The object region is the area in an image frame that is locked based on target object information and contains effective monitoring targets such as pedestrians and vehicles. The background region is the remaining image area after removing all object regions within a single frame, mostly consisting of environmental scenes without key monitoring targets, such as roads, greenery, and buildings.

[0052] Taking a road intersection security monitoring scenario as an example, the system first determines the current network congestion state based on the calculated available bandwidth. Then, based on the previously detected vehicle and pedestrian target information, it defines the area where vehicles and pedestrians are located as the target area, and areas such as the road surface, green belt, and streetlights as the background area. According to business rules, the target areas corresponding to motor vehicles and pedestrians are set as high priority. Combining the high priority requirement with the congested network state, higher bitrate weights are assigned to the encoding units in the target area, and lower bitrate weights are assigned to the encoding units in the background area. Subsequently, the quantization parameters are adjusted according to the network state and bitrate weights, reducing the compression intensity of high-weight target areas to ensure clear target details, and appropriately increasing the compression intensity of low-weight background areas to save bandwidth resources.

[0053] 105. Transmit the video stream according to the quantization parameters and bitrate weights.

[0054] For example, specifically, all coding units within a single frame are traversed. Based on pre-allocated quantization parameters and bitrate weights, encoding and compression processing is performed on each coding unit to generate corresponding encoded data. Bitrate resources are allocated according to the bitrate weights, and the compression level is controlled by the quantization parameters to achieve differentiated encoding. Then, the encoded data of all coding units within the same image frame are summarized, spliced, and format-encapsulated to obtain a complete single-frame bitstream. Next, according to the acquisition sequence of each image frame in the video stream, the single-frame bitstreams generated frame by frame are sequentially sorted and combined to generate a complete video bitstream to be transmitted. Finally, following the established network transmission protocol, the assembled video bitstream is continuously pushed to the receiving end to complete the transmission of the video stream.

[0055] Optionally, in some embodiments of this application, the step "transmitting the video stream according to the quantization parameters and bitrate weights" may specifically include: Based on the quantization parameters and bitrate weights, each coding unit within the image frame is subjected to coding compression processing to generate the coding data corresponding to each coding unit. By integrating the encoded data of all coding units in the same image frame, a single-frame bitstream is obtained; Based on the order of each image frame and the corresponding single-frame bitstream, a video bitstream to be transmitted is generated and transmitted to the receiving end.

[0056] The encoding and compression process involves removing redundant information and compressing the original image data of an encoding unit according to preset video encoding rules, combined with allocated quantization parameters and bitrate weights. The quantization parameters determine the degree of compression, while the bitrate weights constrain the bitrate resources that the unit can utilize, thus achieving differentiated compression for different regions. Encoded data is the binary data stream generated after a single encoding unit completes its encoding and compression, carrying the pixel, texture, and other image information corresponding to that unit; it is the basic data unit after image compression. A single-frame bitstream is a complete data stream formed by integrating the encoded data of all encoding units within the same image frame according to the encoding format and arrangement rules, corresponding to all the compressed data of an entire image frame.

[0057] Taking the transmission of images from a security camera at an intersection as an example, the image frame is divided into several encoding units. The encoding units containing vehicles and pedestrians are compressed using low quantization parameters and high bitrate weights to preserve image details to the greatest extent possible. Background encoding units such as roads and greenery are compressed using high quantization parameters and low bitrate weights to reduce data volume. Each encoding unit generates independent encoded data after compression. Then, the encoded data of all encoding units within a frame are integrated to obtain the single-frame bitstream of that frame. Then, according to the order in which the images were captured, the single-frame bitstreams of multiple consecutive frames are concatenated into a complete video bitstream, which is finally sent to the backend monitoring platform (receiving end) via a network link.

[0058] This application provides a video transmission method. After acquiring the video stream to be transmitted, the method detects the target object information corresponding to each image frame based on a preset neural network and the motion vectors corresponding to each image frame in the video stream. Then, based on the local transmission time and the transmission feedback information returned by the receiving end, the available bandwidth is estimated. Next, based on the available bandwidth and the target object information corresponding to each image frame, the quantization parameters and bitrate weights corresponding to each encoding unit are determined. Finally, the video stream is transmitted according to the quantization parameters and bitrate weights. In the video transmission scheme provided in this application, the network bandwidth status and target object information are linked to differentiate the quantization parameters and bitrate weights for different encoding units, eliminating the need for a uniform encoding method to process the entire frame. Finally, the video stream is transmitted according to the allocated encoding parameters. This reduces bitrate loss in invalid areas and improves bandwidth utilization efficiency when network conditions are good, while prioritizing the clarity of key image details in bandwidth-constrained scenarios, effectively balancing image quality and data transmission efficiency, and better adapting to complex and ever-changing network transmission scenarios.

[0059] To facilitate better implementation of the video transmission method of the embodiments of this application, the embodiments of this application also provide a video transmission device, wherein the meanings of the terms are the same as those in the video transmission method described above, and specific implementation details can be found in the description of the system embodiments.

[0060] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a video transmission device provided in an embodiment of this application. Specifically, the video transmission device may include an acquisition module 201, a detection module 202, an estimation module 203, a determination module 204, and a transmission module 205, as follows: Acquisition module 201 is used to acquire the video stream to be transmitted; The detection module 202 is used to detect the target object information corresponding to each image frame based on a preset neural network and the motion vectors corresponding to each image frame in the video stream. The estimation module 203 is used to estimate the currently available bandwidth based on the local transmission time and the transmission feedback information returned by the receiving end. The determination module 204 is used to determine the quantization parameters and bit rate weights of each coding unit based on the currently available bandwidth and the target object information corresponding to each image frame. The transmission module 205 is used to transmit the video stream according to the quantization parameters and bit rate weights.

[0061] Optionally, in some embodiments of this application, the detection module 202 may specifically include: The generation unit is used to generate the object box corresponding to the target object in the first image frame of the video stream based on a preset neural network. The output unit is used to output the target object information corresponding to each image frame based on the object frame and the motion vectors corresponding to each image frame except the first image frame.

[0062] Optionally, in some embodiments of this application, the output unit may specifically be used for: Obtain the motion vectors corresponding to each image frame except the first image frame; The image frame currently being processed is designated as the current frame. Using the object box as a reference, the current frame is projected forward based on the motion vector to obtain the candidate object box corresponding to the current frame; Check whether the candidate object box meets the preset conditions; Output the target object information corresponding to each image frame based on the detection results.

[0063] Optionally, in some embodiments of this application, the output unit may specifically be used for: Calculate the consistency of motion vectors within the candidate object bounding box; Obtain the frame interval between the current frame and the last time the neural network was run; When the consistency level is greater than or equal to the preset threshold and the frame interval is less than the preset time value, the object frame is determined to meet the preset conditions.

[0064] Optionally, in some embodiments of this application, the estimation module 203 may specifically be used for: Obtain the receiving timestamp, number of lost packets, and latency jitter data from the transmission feedback information returned by the receiving end; Calculate the one-way delay value of the current transmission based on the local sending time and receiving timestamp; Based on the one-way latency and packet loss count, estimate the current available bandwidth.

[0065] Optionally, in some embodiments of this application, the determining module 204 may specifically be used for: The network operating status is determined based on the currently available bandwidth. In each image frame, determine the object region containing the target object and the background region excluding the object region; Determine the priority of each object region based on the target object information; Based on the priority and network operating status, assign bitrate weights to each coding unit in the image frame; The quantization parameters corresponding to each coding unit are adjusted according to the network operating status and bitrate weight.

[0066] Optionally, in some embodiments of this application, the transmission module 205 may specifically be used for: Based on the quantization parameters and bitrate weights, each coding unit within the image frame is subjected to coding compression processing to generate the coding data corresponding to each coding unit. By integrating the encoded data of all coding units in the same image frame, a single-frame bitstream is obtained; Based on the order of each image frame and the corresponding single-frame bitstream, a video bitstream to be transmitted is generated and transmitted to the receiving end.

[0067] This application provides a video transmission device. After the acquisition module 201 acquires the video stream to be transmitted, the detection module 202 detects the target object information corresponding to each image frame based on a preset neural network and the motion vectors corresponding to each image frame in the video stream. Then, the estimation module 203 estimates the current available bandwidth based on the local transmission time and the transmission feedback information returned by the receiving end. Next, the determination module 204 determines the quantization parameters and bitrate weights corresponding to each encoding unit based on the current available bandwidth and the target object information corresponding to each image frame. Finally, the transmission module 205 transmits the video stream according to the quantization parameters and bitrate weights. In the video transmission scheme provided in this application, the network bandwidth status and target object information are linked to differentiate the quantization parameters and bitrate weights for different encoding units, and the uniform encoding method is no longer used to process the entire frame. Finally, the video stream is transmitted according to the allocated encoding parameters. This can reduce the bitrate loss in invalid areas and improve bandwidth utilization efficiency when network conditions are good, and can also prioritize the clarity of key image details in bandwidth-limited scenarios, effectively balancing image quality and data transmission efficiency, and better adapting to complex and ever-changing network transmission scenarios.

[0068] Furthermore, embodiments of this application also provide a computing device, such as... Figure 4 As shown, it illustrates a schematic diagram of the computing device involved in the embodiments of this application, specifically: The computing device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0069] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other network elements such as clients or other servers. The processor 402 executes program 410, specifically performing the relevant steps in the above-described embodiment of the bandwidth acquisition method for computing devices.

[0070] Specifically, program 410 may include program code that includes computer operation instructions.

[0071] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0072] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0073] Specifically, program 410 can be used to cause processor 402 to execute the bandwidth acquisition method in any of the above method embodiments. The specific implementation of each step in program 410 can be found in the corresponding descriptions of the steps and units in the above bandwidth acquisition embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0074] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this application are not directed to any particular programming language. It should be understood that the contents of the embodiments of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best implementation of the embodiments of this application.

[0075] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0076] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are meant to be within the scope of the embodiments of this application and form different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0077] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of this application. The embodiments of this application can also be implemented as device or apparatus programs (e.g., computer programs and computer program products) for performing part or all of the methods described herein. Such programs implementing the embodiments of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form: Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the video transmission methods provided in embodiments of this application. For example, the instructions can execute the following steps: The video stream to be transmitted is acquired. Based on a preset neural network and the motion vectors corresponding to each image frame in the video stream, the target object information corresponding to each image frame is detected. Based on the local transmission time and the transmission feedback information returned by the receiver, the current available bandwidth is estimated. Based on the current available bandwidth and the target object information corresponding to each image frame, the quantization parameters and bit rate weights corresponding to each coding unit are determined. The video stream is transmitted according to the quantization parameters and bit rate weights.

[0078] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0079] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0080] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the video transmission methods provided in the embodiments of this application, the beneficial effects that any of the video transmission methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0081] The foregoing has provided a detailed description of a video transmission method, apparatus, computing device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A video transmission method, characterized in that, include: Obtain the video stream to be transmitted; Based on a preset neural network and the motion vectors corresponding to each image frame in the video stream, the target object information corresponding to each image frame is detected. Based on the local transmission time and the transmission feedback information returned by the receiver, estimate the currently available bandwidth; Based on the currently available bandwidth and the target object information corresponding to each image frame, determine the quantization parameters and bitrate weights corresponding to each coding unit; The video stream is transmitted according to the quantization parameters and bitrate weights.

2. The video transmission method according to claim 1, characterized in that, The method of detecting target object information corresponding to each image frame based on a preset neural network and motion vectors corresponding to each image frame in the video stream includes: Based on a preset neural network, an object box corresponding to the target object is generated in the first image frame of the video stream. Based on the object frame and the motion vectors corresponding to each image frame except the first image frame, output the target object information corresponding to each image frame.

3. The video transmission method according to claim 2, characterized in that, The step of outputting the target object information corresponding to each image frame based on the object frame and the motion vectors corresponding to each image frame except the first image frame includes: Obtain the motion vectors corresponding to each image frame except the first image frame; The image frame currently being processed is designated as the current frame. Using the object box as a reference, the current frame is projected forward based on the motion vector to obtain the candidate object box corresponding to the current frame; Detect whether the candidate object box meets the preset conditions; Based on the detection results, the target object information corresponding to each image frame is output.

4. The video transmission method according to claim 3, characterized in that, The step of detecting whether the candidate object box meets the preset conditions includes: Calculate the degree of consistency of motion vectors within the candidate object box; Obtain the frame interval between the current frame and the last time the neural network was run; When the consistency level is greater than or equal to a preset threshold and the frame interval is less than a preset time value, the object frame is determined to meet the preset conditions.

5. The video transmission method according to any one of claims 1 to 4, characterized in that, The estimation of currently available bandwidth based on the local transmission time and the transmission feedback information returned by the receiver includes: Obtain the receiving timestamp, number of lost packets, and latency jitter data from the transmission feedback information returned by the receiving end; Calculate the one-way delay value of the current transmission based on the local sending time and the receiving timestamp; Based on the one-way latency value and the number of packet losses, the current available bandwidth is estimated.

6. The video transmission method according to any one of claims 1 to 4, characterized in that, The step of determining the quantization parameters and bitrate weights for each coding unit based on the currently available bandwidth and the target object information corresponding to each image frame includes: The network operating status is determined based on the currently available bandwidth. In each image frame, determine the object region containing the target object and the background region excluding the object region; Determine the priority of each object region based on the target object information; Based on the priority and network operating status, assign bitrate weights to each coding unit in the image frame; The quantization parameters corresponding to each coding unit are adjusted according to the network operating status and bitrate weight.

7. The video transmission method according to any one of claims 1 to 4, characterized in that, The step of transmitting the video stream according to the quantization parameters and bitrate weights includes: Based on the quantization parameters and bitrate weights, each coding unit within the image frame is subjected to coding compression processing to generate the coding data corresponding to each coding unit. By integrating the encoded data of all coding units in the same image frame, a single-frame bitstream is obtained; Based on the order of the image frames and the corresponding single-frame bitstream, a video bitstream to be transmitted is generated and transmitted to the receiving end.

8. A video transmission device, characterized in that, include: The acquisition module is used to acquire the video stream to be transmitted; The detection module is used to detect the target object information corresponding to each image frame based on a preset neural network and the motion vectors corresponding to each image frame in the video stream. The estimation module is used to estimate the currently available bandwidth based on the local transmission time and the transmission feedback information returned by the receiver. The determination module is used to determine the quantization parameters and bitrate weights corresponding to each coding unit based on the currently available bandwidth and the target object information corresponding to each image frame; The transmission module is used to transmit the video stream according to the quantization parameters and bitrate weights.

9. A computing device, characterized in that, include: At least one processor; and A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of the video transmission method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the steps of the video transmission method as described in any one of claims 1 to 7.

Images