Video data processing method and system, electronic equipment and storage medium

Abstract

The invention relates to a video data processing method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring an original image frame; identifying a target object in the original image frame, and determining a target area where the target object is located; determining a background region based on the target region; acquiring a preset compression parameter; the preset compression parameters comprise a first compression parameter and a second compression parameter; according to a preset compression parameter, coding the target region and the background region to obtain a coded data packet; and generating target video data based on the coded data packet. According to the technical scheme provided by the invention, the target region and the background region are distinguished in the same original image frame, and different compression parameters are adopted for coding, so that the image presentation quality of the target region can be improved while the overall coding resource occupation and the data volume are controlled; and the comprehensive display effect of the video data can be improved in a scene with limited bandwidth or processing capability.

Description

Technical Field

[0001] This disclosure relates to the field of video data processing technology, and in particular to a video data processing method, system, electronic device and storage medium. Background Technology

[0002] With the development of network communication technology and multimedia coding standards, real-time video calling services are widely used in smartphones, tablets, personal computers, and various smart home terminals. Existing video calling systems typically employ standardized video codec protocols such as H.264 (Advanced Video Coding) or H.265 (High Efficiency Video Coding), with each party's terminal performing real-time encoding of their local video and real-time decoding of the received video. To provide clear video images at higher resolutions, terminals often rely on a central processing unit (CPU) or dedicated hardware decoding unit to provide high computing power and storage bandwidth.

[0003] In another approach, video data is transmitted using formats such as MJPEG (Motion JPEG), treating the video stream as a series of consecutive JPEG (Joint Photographic Experts Group) still images. The receiving end decodes each frame of the JPEG image and refreshes it on the display frame by frame. This approach is relatively simple to implement and does not rely on complex video prediction structures. However, it still requires high decoding frequency and data bandwidth when transmitting high-resolution video. The decoding side needs to decode the entire image frame by frame, which can easily lead to a heavy computational load on the terminal side.

[0004] In recent years, a large number of networked display devices based on microcontroller units (MCUs) have emerged, such as smart photo frames and information boards with high-resolution displays but only equipped with low-frequency processors and small-capacity memory. These devices typically lack a complete operating system and high-performance multimedia processing units. When attempting to directly adopt the aforementioned end-to-end video encoding and decoding schemes or simple MJPEG-based schemes, they struggle to complete real-time decoding of standard video encoding formats at high resolutions, easily resulting in problems such as excessively low video frame rates and severe stuttering. For example, when such devices attempt to directly decode video streams with a resolution close to 1280×800 pixels locally, they often only achieve a playback frame rate of about 1 to 2 frames per second. The user's experience is close to a static image scrolling through pages, resulting in low usability for video calls.

[0005] To alleviate the aforementioned problems, existing technologies can adjust video call configurations by uniformly reducing the encoding bitrate, video resolution, or video frame rate to minimize the decoding load on the terminal and network bandwidth pressure. However, since the entire image is usually processed using uniform encoding parameters, existing solutions often cannot differentiate the encoding quality based on the areas of the image that the user cares about more and the relatively unimportant areas. Under conditions of limited terminal computing power and network resources, it is still difficult to simultaneously ensure the clarity of key images and the smoothness of overall playback. Summary of the Invention

[0006] This disclosure provides a video data processing method and system to at least solve the problem in related technologies where it is difficult to balance the clarity of key images with video data transmission efficiency under limited encoding resources and transmission bandwidth. The technical solution of this disclosure is as follows: According to a first aspect of the embodiments of this specification, a video data processing method is provided, applied to a server; comprising: Acquire raw image frames; the raw image frames represent single frame image data in the video data stream; Identify the target object in the original image frame and determine the target region where the target object is located; Based on the target region, a background region is determined; the background region represents the area in the original image frame other than the target region. Obtain preset compression parameters; the preset compression parameters include a first compression parameter and a second compression parameter; the image encoding quality corresponding to the first compression parameter is higher than the image encoding quality corresponding to the second compression parameter. According to the preset compression parameters, the target region and the background region are encoded respectively to obtain an encoded data packet; Based on the encoded data packet, target video data is generated.

[0007] According to a second aspect of the embodiments of this specification, a video data processing system is provided, comprising: The image acquisition module is used to acquire raw image frames; the raw image frames represent single frame image data in the video data stream. An object recognition module is used to identify target objects in the original image frame and determine the target region where the target object is located. The background determination module is used to determine a background region based on the target region; the background region represents the area in the original image frame other than the target region. A parameter acquisition module is used to acquire preset compression parameters; the preset compression parameters include a first compression parameter and a second compression parameter; the image encoding quality corresponding to the first compression parameter is higher than the image encoding quality corresponding to the second compression parameter. The classification encoding module is used to encode the target region and the background region respectively according to the preset compression parameters to obtain encoded data packets; The video generation module is used to generate target video data based on the encoded data packet.

[0008] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method as described in any one of the first aspects above.

[0009] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided such that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the methods described in the first aspect of the present disclosure.

[0010] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: By acquiring the original image frame representing a single frame of image data in a video data stream from the server, identifying the target object in the original image frame, determining the target region where the target object is located, and determining the background region based on the target region, the video data processing method can distinguish image regions of different importance within the original image frame. This improves the flexibility of configuring image encoding quality and other compression-related parameters separately for the target region and the background region.

[0011] By acquiring preset compression parameters including a first compression parameter and a second compression parameter, and making the image encoding quality corresponding to the first compression parameter higher than the image encoding quality corresponding to the second compression parameter, and then encoding the target area and the background area respectively according to the preset compression parameters to obtain encoded data packets, the video data processing method can allocate higher image encoding quality to the target area when encoding the original image frame, thereby improving the degree of image detail preservation and visual clarity of the target object in the target video data.

[0012] By encoding the background region with a second compression parameter whose image encoding quality is lower than that of the target region, and generating target video data based on encoded data packets containing both target region encoded data and background region encoded data, the video data processing method can reduce the amount of encoded data corresponding to the target video data compared to encoding the entire original image frame with a uniform image encoding quality. This reduces the bandwidth and storage requirements of the target video data during transmission and storage, and helps improve the transmission efficiency of the target video data under resource-constrained conditions.

[0013] In summary, by identifying the target object and distinguishing the target area from the background area based on the original image frame in the server, and encoding the target area and the background area separately using preset compression parameters to generate target video data, the image encoding quality of the target area in the target video data can be improved while controlling the amount of encoded target video data.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0016] Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment.

[0017] Figure 2 This is a flowchart illustrating a video data processing method according to an exemplary embodiment.

[0018] Figure 3 This is a flowchart illustrating a video frame differential encoding method based on the division of face region and background region according to an exemplary embodiment.

[0019] Figure 4 This is a schematic diagram illustrating video data uplink and differentiated downlink interaction between a resource-constrained electronic photo frame and a cloud server, according to an exemplary embodiment.

[0020] Figure 5 This is a block diagram of a video data processing system according to an exemplary embodiment.

[0021] Figure 6 This is a block diagram illustrating an electronic device for video data processing according to an exemplary embodiment. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0023] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of systems and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0024] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment. For example... Figure 1 As shown, the application environment may include server 01, first terminal 02, and second terminal 03. Server 01 is communicatively connected to both first terminal 02 and second terminal 03, and server 01 is used to provide video data processing and forwarding capabilities between first terminal 02 and second terminal 03. The communication connection can be a local area network (LAN) connection or a wide area network (WAN) connection via the Internet, and can include a wired Ethernet connection or a wireless network connection based on Wi-Fi (Wireless Fidelity) technology.

[0025] In an optional embodiment, the first terminal 02 and the second terminal 03 can be networked electronic photo frames of the same type. Each networked electronic photo frame can include a display screen assembly, a camera assembly, a microphone assembly, a speaker assembly, a processing unit, a storage unit, a first-type encoder, a second-type decoder, and a network interface assembly. The display screen assembly can be used to display static images or video images, the camera assembly can be used to collect continuous image data of the local user, the microphone assembly can be used to collect voice data of the local user, and the speaker assembly can be used to output voice of the remote user. The processing unit can be an MCU or an application processor, and the processing unit can communicate with the storage unit, the first-type encoder, the second-type decoder, and the network interface assembly. The first-type encoder can be deployed in the processing unit or configured as a hardware encoding module inside the networked electronic photo frame for video encoding of the collected continuous image data to generate a first-format video stream; the second-type decoder can be deployed in the processing unit or configured as a hardware decoding module inside the networked electronic photo frame for decoding the target video data sent by the server 01 to display decoded image frames on the display screen assembly. To distinguish between different video data transmission directions, the networked electronic photo frame on the video sending side can be referred to as the first terminal 02, and the networked electronic photo frame on the video receiving side can be referred to as the second terminal 03.

[0026] In one example, the first terminal 02 can be located in the living environment of the first user, such as in the living room, to collect real-time image data of the first user and generate a video stream in a first format using a first type encoder; the second terminal 03 can be located in the living environment of the second user, such as in a remote bedroom or living room, to display video images related to the first user on a high-resolution display screen, and to send the second user's video data to the server 01 via a local first type encoder when needed. Exemplarily, the display screen component of each networked electronic photo frame can be a display screen with a resolution of 1280×800 pixels, the camera component can collect image data at a frame rate of not less than 30 frames per second, the main frequency of the processing unit can be approximately 400MHz, the second type decoder can be a decoder supporting JPEG image decoding, and the first type encoder can be an encoder supporting video encoding standards such as H.264.

[0027] Server 01 can be a video data processing device configured with a network interface, a processing unit, and a storage unit. Server 01 may include one or more processors, system memory, and non-volatile storage media. Server 01 can store program code for executing video data processing methods in the storage unit and run the program code on the processing unit. Server 01 can communicate with a first terminal 02 via the network interface to receive a first-format video stream generated by a first-type encoder, and can also communicate with a second terminal 03 via the network interface to send target video data to the second terminal 03. For example, server 01 can be deployed in a cloud data center or on an edge computing node close to the user. Server 01 can support H.264 video decoding, target object recognition, and image encoding processing with image encoding quality configured based on the target area and background area respectively, to generate target video data adapted to the second-type decoder.

[0028] In one possible implementation, the first terminal 02 and the second terminal 03 can be set up in different home environments to enable video communication between remote family members. For example, the first terminal 02 can be set up in the home of the children to collect image data of the children, and the second terminal 03 can be set up in the home of the elderly to display the image of the children. The first terminal 02 can send a first-format video stream to the server 01 through a communication connection. The server 01 can acquire the original image frames based on the first-format video stream and execute a video data processing method to generate target video data. The second terminal 03 can receive the target video data from the server 01 through a communication connection and decode it based on a second-type decoder to display the video image related to the first terminal 02 on the display screen component.

[0029] In another scenario, the second terminal 03 can also serve as a video sending terminal. In this case, the second terminal 03 can generate a first format video stream through a local first type encoder and send it to the server 01. The server 01 can generate target video data for the first terminal 02 by referring to the same processing flow.

[0030] In addition, it should be noted that, Figure 1 The example shown is merely one application environment of the video data processing method provided in this disclosure.

[0031] It should be noted that the following diagram illustrates one possible sequence of steps, and it is not strictly required to follow this order. Some steps can be performed in parallel without interdependence. The user information (including but not limited to user device information, user personal information, user behavior information, etc.) and data (including but not limited to data used for display, training data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0032] The following describes a specific embodiment of a video data processing method disclosed herein. Figure 2 This is a flowchart illustrating a video data processing method according to an exemplary embodiment. Figure 2 As shown, it may include the following steps.

[0033] In step S201, the original image frame is acquired; the original image frame represents a single frame of image data in the video data stream.

[0034] In the embodiments of this specification, the original image frame can be an image data unit obtained based on the parsing of a video data stream. The original image frame is used to represent a single frame of image data in the video data stream, and the pixel array in the original image frame can represent the complete screen content corresponding to a certain moment in the video data stream in pixel coordinates. The video data stream can be a sequence of coded data used to represent continuous video frames, and the video data stream can include multiple image coded data segments used to reconstruct the original image frame. Each image coded data segment corresponds to an original image frame after decoding.

[0035] In one possible implementation, the video data stream can be parsed frame by frame. Specifically, during the reception of the video data stream, boundary detection can be performed to identify the start and end positions of each image-coded data segment. When a complete image-coded data segment is detected, a decoding operation can be triggered to decode the segment and obtain the corresponding original image frame. The parsed original image frame can be stored in an image frame buffer for subsequent steps of the video data processing method to perform processing based on the original image frame.

[0036] For example, the video data stream can be encoded in H.264 or H.265 format, and each image-encoded data segment in the video data stream can correspond to one frame of encoded data. A decoding algorithm matching the encoding format can be invoked to decode the image-encoded data segments in the video data stream to obtain original image frames with a resolution of 1280×800 pixels. In another example, the video data stream can be in MJPEG format, and each image-encoded data segment in the video data stream can correspond to one frame of image-encoded data. Each frame of JPEG image-encoded data can be decoded sequentially to obtain multiple original image frames.

[0037] In one optional implementation, while acquiring the original image frame, the timestamp information or frame sequence number corresponding to the original image frame can be extracted, and the timestamp information or frame sequence number can be associated with and stored with the original image frame. Optionally, in a scenario where multiple video data streams are received, a corresponding queue of original image frames can be maintained for each different video data stream, and the original image frames can be sorted according to the timestamp information or frame sequence number within each queue, so as to distinguish the source of the original image frames among different video data streams.

[0038] Optionally, the sampling frequency of the original image frames participating in the video data processing method can be controlled based on timestamp information. For example, if the frame rate of the video data stream is thirty frames per second, ten original image frames per second can be selected from the video data stream as input for subsequent processing. Furthermore, a discarding strategy can be implemented for the original image frames based on timestamp information or frame sequence numbers. For example, when buffer space is tight, original image frames exceeding a preset time threshold can be discarded.

[0039] In practical applications, by acquiring the original image frames frame by frame from the video data stream and making the original image frames represent the single-frame image data in the video data stream, the subsequent recognition and encoding processing of the original image frames can be based on the single-frame image data with a unified format. This helps to improve the reliability of video data processing methods in the context of continuous video data streams.

[0040] In one possible implementation, a first format video stream is received from a first terminal; the first format video stream is generated by the first terminal based on a first type encoder; the first format video stream is decoded to obtain the original image frame.

[0041] In the embodiments of this specification, the first terminal may be a video acquisition device configured with an image acquisition component and a first type encoder. The first terminal can locally acquire continuous image data to represent a user scene, and encode the continuous image data based on the first type encoder to generate a first format video stream to represent continuous video frames. The first type encoder may be a video encoding function module deployed in the first terminal, and the first type encoder can compress and encode the acquired image data according to a preset encoding standard and encoding parameters.

[0042] The first format video stream can be a sequence of encoded video data output by a first type encoder, and the first format video stream contains encoded data for reconstruction into multiple original image frames.

[0043] In one possible implementation, a communication connection can be established between the first terminal and the server. After the connection is established, the first terminal can continuously encode the acquired multi-frame image data based on a first-type encoder to generate a first-format video stream, and send the first-format video stream to the receiving side in the form of data segments. The received data segments can be sequentially reassembled to recover the continuous first-format video stream at the receiving side. The first-format video stream can be input into a decoding module corresponding to the first-type encoder to decode the first-format video stream, thereby obtaining multiple original image frames sequentially in time. The original image frames can be stored in an image frame buffer space for subsequent steps of the video data processing method to process based on the original image frames.

[0044] For example, the first terminal can be an electronic photo frame equipped with a camera component and an H.264 encoding module, the first type of encoder can be an H.264 encoder, and the first format video stream can be an H.264 encoded video stream. A decoding algorithm matching the H.264 encoder can be used to decode the first format video stream to obtain an original image frame sequence with a resolution of 1280×800 pixels.

[0045] In another example, the first terminal can be a smart terminal equipped with a camera component and an H.265 encoding module. The first type of encoder can be an H.265 encoder, and the first format video stream can be an H.265 encoded video stream. A decoding algorithm matching the H.265 encoder can be used to decode the first format video stream to obtain multiple original image frames. Furthermore, the first type of encoder can encode the image data at a frame rate of thirty frames per second, and the decoded original image frames can be used as input to the video data processing method at a frame rate of thirty frames per second or less.

[0046] In one optional implementation, the first format video stream can be encapsulated into multiple information units with sequence identifiers at the transport layer. The received information units can be sorted and reassembled based on the sequence identifiers to recover the frame order of the first format video stream generated on the encoding side. Optionally, timestamp information or frame sequence number information can be extracted simultaneously during the decoding process of the first format video stream, and the timestamp information or frame sequence number information can be associated with the decoded original image frames to maintain the temporal correspondence with the first format video stream during subsequent processing based on the original image frames.

[0047] In practical applications, by receiving the first format video stream generated by the first terminal based on the first type of encoder and decoding the first format video stream, the source of the original image frame is decoupled from the encoding method of the first terminal. Without changing the encoding implementation of the first terminal, the original image frame can be used as a unified input for the video data processing method, which facilitates the subsequent video data processing of the original image frame in different types of first terminal access scenarios.

[0048] In step S203, the target object in the original image frame is identified, and the target area where the target object is located is determined.

[0049] In the embodiments of this specification, the target object can be an object used to represent the image of a user participating in video communication in the original image frame. The target object can be described by the pixel distribution and texture features in the original image frame. The target region can be a continuous pixel region determined around the position of the target object in a pixel coordinate system. The target region is used to indicate the range of image data corresponding to the target object, and the target region can be distinguished from the background region in the original image frame.

[0050] In one possible implementation, feature analysis can be performed on the original image frame to identify candidate objects present in the frame. Specifically, feature extraction can be performed on the original image frame to obtain feature data representing local texture, edges, and shape information. Then, the feature data is matched based on a pre-defined object detection model to obtain a set of detection results representing multiple candidate objects. Detection results that meet pre-set confidence conditions can be identified as the detection results corresponding to candidate objects. The set of pixels associated with the detection results corresponding to each candidate object is identified as the position of each candidate object in the original image frame. Based on this, the target object is determined according to pre-defined target object selection conditions. Based on the position of the target object, boundary coordinate information representing the target region can be constructed. This boundary coordinate information can include the horizontal and vertical boundary positions of the target region, used to define the pixel range covered by the target region.

[0051] For example, the original image frame can be a color image with a resolution of 1280×800 pixels. The original image frame can be processed based on a face detection algorithm to obtain at least one detection result representing the user's face. In one example, the target object can be a face detection result with a confidence level not lower than a preset face detection threshold. The target region can correspond to a rectangular bounding box region surrounding the target object, which can be represented by the coordinates of its upper left and lower right corners in a pixel coordinate system. Furthermore, the boundaries of the rectangular bounding box region can be appropriately extended so that the target region extends beyond the outer contour of the target object by several pixels in both the horizontal and vertical directions, so that the target region can retain some image context information adjacent to the target object during subsequent display.

[0052] In one optional implementation, the original image frames can be preprocessed before target object recognition, such as performing color space conversion, brightness normalization, or noise suppression to reduce the impact of illumination changes and noise interference on target object recognition. Optionally, when multiple object detection models are available, a detection model for detecting face objects, human objects, or other objects of business interest can be selected according to the application scenario, and the detection results obtained based on the selected detection model can be used as the basis for target object recognition. After determining the target region, the boundary coordinate information used to characterize the target region can be associated and stored with the original image frames so that subsequent steps of the video data processing method can perform differentiated processing on the original image frames based on the target region. Optionally, in the processing scenario of continuous video data streams, the processing of identifying target objects and determining target regions can be performed separately for each original image frame to update the position and size of the target region as the original image frames change on the time axis.

[0053] In practical applications, by identifying the target object and determining the target area in the original image frame, the video data processing method can separately identify and manage the image data associated with the target object in subsequent processing. This helps to adopt different processing strategies for the target area and the background area during image encoding and transmission, thereby improving the adjustability of image presentation quality and encoding resource utilization efficiency for the target object.

[0054] In one possible implementation, the number of objects in the original image frame is detected; if there are multiple candidate objects in the original image frame, the area occupied by each candidate object in the original image frame is obtained; based on the area occupied information, the multiple candidate objects are sorted to obtain the sorting result; the candidate object that ranks first in the sorting result is determined as the target object, and the area where the target object is located is determined as the target area.

[0055] In the embodiments of this specification, the candidate object can be a member of the object set obtained based on the object detection results in the original image frame. The candidate object is used to characterize an object in the original image frame that meets the preset detection conditions and has the possibility of becoming a target object.

[0056] Occupied area information can be a parameter used to characterize the size of the image range occupied by the candidate object in the pixel coordinate system of the original image frame. The occupied area information can be calculated based on the boundary range of the candidate object in the original image frame.

[0057] The sorting result can be an ordered set of objects obtained by sorting the candidate objects based on the area occupied by each candidate object. The order of the candidate objects in the sorting result is used to characterize the relative size relationship of the candidate objects in terms of area occupied.

[0058] In one possible implementation, after object detection is performed on the original image frame and at least one candidate object is obtained, the number of objects in the original image frame can be detected to determine whether multiple candidate objects exist. If multiple candidate objects exist in the original image frame, the coverage area of ​​each candidate object in the original image frame can be determined based on its boundary information, and the area of ​​each coverage area is used as the corresponding occupied area information. Specifically, the pixel span in the horizontal direction and the pixel span in the vertical direction of each candidate object can be obtained, and the product of the horizontal and vertical pixel spans is used as the occupied area information of that candidate object. Based on the obtained occupied area information, the multiple candidate objects can be sorted in descending order of occupied area information to obtain a sorting result. The candidate object ranked first in the sorting result can be determined as the target object, and the target area is determined based on the coverage area corresponding to that candidate object, so that the target area covers the image range occupied by the candidate object ranked first in the sorting result in the pixel coordinate system.

[0059] For example, the original image frame can be a color image with a resolution of 1280×800 pixels. Multiple candidate objects representing different users' faces can be detected simultaneously in the original image frame, each corresponding to a bounding rectangle region of the face in the original image frame. The product of the pixel width and pixel height of each bounding rectangle region of the face can be used as the area occupied by the corresponding candidate object. Based on this area information, the multiple candidate objects are sorted to obtain a ranking result. The candidate object that ranks first in the ranking result has the largest area occupied by its bounding rectangle region in the original image frame among the multiple candidate objects. This candidate object can be identified as the target object, and its bounding rectangle region can be identified as the target region.

[0060] In one optional implementation, before sorting multiple candidate objects, the occupied area information can be filtered according to a preset area threshold, retaining only candidate objects whose occupied area information is not lower than the area threshold for sorting, thereby reducing the influence of distant or noisy candidate objects on the target object determination result. Optionally, when there are multiple candidate objects with similar occupied area information in the sorting result, the candidate object closer to the center of the original image frame can be selected as the target object based on the positional relationship of the candidate objects in the original image frame, so that the target area is closer to the user's area of ​​interest.

[0061] In step S205, the background region is determined based on the target region.

[0062] In the embodiments of this specification, the background region can be an image region composed of a set of pixels in the original image frame that do not belong to the target region. The background region is used to represent image content in the original image frame other than the image content corresponding to the target region. The background region can correspond to image content that is processed with different image encoding parameters than the target region in subsequent processing. The background region complements the target region in the pixel coordinate system, so that the pixel sets of the target region and the background region form a complete coverage relationship in the original image frame. That is, the background region represents the region in the original image frame other than the target region.

[0063] In one possible implementation, the boundary range of the original image frame and the boundary coordinate information used to characterize the target region can be obtained in a pixel coordinate system. The boundary coordinate information may include the start and end coordinates of the target region in the horizontal direction and in the vertical direction. Based on the boundary coordinate information, a target region mask can be constructed on the pixel plane corresponding to the original image frame to mark the pixel positions belonging to the target region. For pixel positions on the pixel plane not marked by the target region mask, the corresponding pixels can be considered as pixels belonging to the background region, and these pixels constitute the background region in the pixel coordinate system.

[0064] Exemplarily, the original image frame can be a color image with a resolution of 1280×800 pixels. The target area can be characterized by the upper-left coordinate (x1, y1) and the lower-right coordinate (x2, y2) in the pixel coordinate system, where x1 and x2 represent the pixel coordinates in the horizontal direction, and y1 and y2 represent the pixel coordinates in the vertical direction. In this example, pixels satisfying x < x1, pixels satisfying x > x2, pixels satisfying y < y1, and pixels satisfying y > y2 can be regarded as pixels belonging to the background area, so that the background area includes pixel sets located on the left, right, above, and below the target area. In this case, the background area can be jointly composed of several geometrically separated pixel areas, but as a whole, it represents the area in the original image frame except for the target area in the sense of pixel sets.

[0065] In an optional implementation manner, when determining the background area, boundary correction processing can be performed on the boundary coordinate information used to characterize the target area, so that the boundary of the target area is within the boundary range of the original image frame. Optionally, in a processing scenario with multiple original image frames, target area information and background area information corresponding to each original image frame can be maintained respectively to avoid confusion between the target area and the background area of different original image frames.

[0066] Optionally, when the target area covers all the pixels of the original image frame in the pixel coordinate system, the background area can be an empty set. At this time, all the pixels corresponding to the original image frame can be regarded as the image data range corresponding to the target area.

[0067] Furthermore, before subsequent processing of the background area, the pixel set corresponding to the background area can be sparsely represented or region-marked encoded as needed, so as to distinguish the pixel ranges of the target area and the background area during subsequent encoding processing.

[0068] In practical applications, by determining the background area based on the target area in the original image frame and making the background area represent the area in the original image frame except for the target area, the image data associated with the target object and the remaining image data can be regionally divided in the video data processing method, so as to separately configure image coding parameters for the target area and the background area during subsequent image encoding and transmission, and control the balance relationship between the image coding quality of the target area and the overall coding data volume.

[0069] In step S207, preset compression parameters are obtained; the preset compression parameters include a first compression parameter and a second compression parameter; the image coding quality corresponding to the first compression parameter is higher than the image coding quality corresponding to the second compression parameter.

[0070] In the embodiments of this specification, the preset compression parameters can be configuration parameters set during the image encoding process to control the balance between image data compression efficiency and image quality. Specifically, the preset compression parameters include two different sets of compression parameters: a first compression parameter and a second compression parameter. The first compression parameter may correspond to an image encoding quality parameter with a value greater than the value of the image encoding quality parameter corresponding to the second compression parameter. The first compression parameter can be used to encode the target region, and the second compression parameter can be used to encode the background region.

[0071] In one possible implementation, the first compression parameter and the second compression parameter can be associated with the image coding quality control parameter and the compression ratio parameter in the image coding algorithm, respectively. The first compression parameter can be configured to ensure that the image coding quality control parameter value for the target region falls within a preset first value range, and the second compression parameter can be configured to ensure that the image coding quality control parameter value for the background region falls within a preset second value range, where the value in the first value range is greater than the value in the second value range. The first compression parameter can correspond to a coding configuration with a lower degree of distortion, and the second compression parameter can correspond to a coding configuration with a higher compression ratio. In a specific implementation, the first compression parameter can be associated with a first value of the quantization step size parameter and a first value of the compression ratio parameter, and the second compression parameter can be associated with a second value of the quantization step size parameter and a second value of the compression ratio parameter, where the first value is less than the second value.

[0072] For example, preset compression parameters can be associated with JPEG encoding. The first compression parameter can correspond to a JPEG encoding quality factor of 98, and the second compression parameter can correspond to a JPEG encoding quality factor of 60. The image encoding quality factor value corresponding to the first compression parameter is greater than the image encoding quality factor value corresponding to the second compression parameter.

[0073] In another example, the preset compression parameters can be associated with H.264 or H.265 encoding methods. The first compression parameter can be associated with the first set of quantization parameter configurations in intra-frame prediction mode, and the second compression parameter can be associated with the second set of quantization parameter configurations in inter-frame prediction mode. The quantization parameter value corresponding to the first set of quantization parameter configurations is less than the quantization parameter value corresponding to the second set of quantization parameter configurations.

[0074] In one optional implementation, the preset compression parameters can be adjusted according to different application scenarios. For example, in a network environment with limited bandwidth, the image encoding quality parameter corresponding to the second compression parameter can be set to a value lower than the default configuration to reduce the amount of image encoding data in the background area, and the overall video data volume can be controlled while keeping the value corresponding to the first compression parameter unchanged or with a small adjustment. In a network environment with relatively sufficient bandwidth resources, the image encoding quality parameters corresponding to the first and second compression parameters can be set to values ​​higher than those in the bandwidth-limited scenario to improve the image encoding quality of the target area and the background area.

[0075] Furthermore, preset compression parameters can be set according to the server's processing capacity, network conditions, or user needs. The range and combination of preset compression parameters can be determined through automatic optimization, user configuration, or predefined strategies to coordinate image coding quality control parameters and coding data volume control parameters at various stages of the video data processing method.

[0076] In practical applications, by using different preset compression parameters for the target area and the background area, different image coding quality control strategies can be applied to the target area and the background area within the same original image frame. This allows the image coding quality parameter value corresponding to the target area to be higher than that corresponding to the background area. Furthermore, image coding resources can be allocated while controlling the overall amount of encoded data, which helps to transmit target video data that meets the application scenario requirements under limited network bandwidth conditions.

[0077] In step S209, the target area and the background area are encoded according to the preset compression parameters to obtain an encoded data packet.

[0078] In the embodiments of this specification, the encoded data packet can be an encapsulated data unit generated by encoding the target region and the background region separately. The encoded data packet can include encoded image data to represent the target region or background region in the original image frame, and the encoded data packet can establish a correspondence with the pixel range of the target region or background region in the original image frame. During the image encoding process, the encoding process corresponding to the target region can be configured using a first compression parameter, and the encoding process corresponding to the background region can be configured using a second compression parameter, so that the encoded data packet corresponding to the target region and the encoded data packet corresponding to the background region have different values ​​in the image encoding quality control parameters.

[0079] In one possible implementation, encoding the target region can employ a first compression parameter. This first compression parameter can be associated with a first set of image encoding quality control parameters and a first set of compression ratio parameters, ensuring that the encoding result for the target region meets preset distortion and data volume ranges. During encoding, image data corresponding to the target region can be extracted from the original image frame based on the pixel set of the target region in the pixel coordinate system. This image data is then encoded according to the image encoding quality control parameters and compression ratio parameters corresponding to the first compression parameter, resulting in an encoded data packet corresponding to the target region.

[0080] The background region can be encoded using a second compression parameter. This second compression parameter can be associated with a second set of image coding quality control parameters and a second set of compression ratio parameters. The value of the second set of image coding quality control parameters can be less than the value of the first set of image coding quality control parameters, and the value of the second set of compression ratio parameters can be greater than the value of the first set of compression ratio parameters. During the encoding of the background region, image data corresponding to the background region can be extracted from the original image frame based on the pixel set of the background region in the pixel coordinate system. This image data is then encoded according to the image coding quality control parameters and compression ratio parameters corresponding to the second compression parameter, resulting in an encoded data packet corresponding to the background region. The encoded data packets corresponding to the target region and the encoded data packets corresponding to the background region can be retained separately during subsequent encapsulation for later combination in the target video data.

[0081] For example, the target region can be the region in the original image frame used to represent the user's face, and the background region can be any region other than the target region. During encoding, JPEG encoding can be used. The first compression parameter can correspond to a JPEG encoding quality factor of 98, and the second compression parameter can correspond to a JPEG encoding quality factor of 60. The JPEG encoding quality factor value corresponding to the first compression parameter is greater than the JPEG encoding quality factor value corresponding to the second compression parameter. In this example, encoding the image data corresponding to the target region based on the first compression parameter yields an encoded data packet corresponding to the target region, and encoding the image data corresponding to the background region based on the second compression parameter yields an encoded data packet corresponding to the background region. Each encoded data packet can carry an information field representing its corresponding region identifier and an information field representing the configuration of image encoding quality control parameters.

[0082] In one optional implementation, the encoded data packet may further include other information, such as image size, frame sequence number, or identification information representing the original image frame index, for identification and management of the encoded data packet during subsequent encapsulation and decoding. Optionally, when decoding the encoded data packet, the decoding process can be configured according to the image encoding quality control parameters carried in the encoded data packet to restore the image data corresponding to the target area and background area at the decoding end.

[0083] In practical applications, by encoding the target region and the background region separately using different preset compression parameters, and by making the encoded data packets corresponding to the target region and the encoded data packets corresponding to the background region differ in the values ​​of image encoding quality control parameters, a relatively higher image encoding quality control parameter value can be assigned to the target region under the condition that the overall encoded data volume is limited. This reduces the encoded data volume of the background region compared to the case where a uniform image encoding quality control parameter configuration is used for the entire frame, thereby establishing a correspondence between the image encoding quality control of the target region and the control of the overall encoded data volume.

[0084] In one possible implementation, a target encoding protocol adapted to the second type of decoder is obtained; and the target region and the background region are encoded according to the target encoding protocol and preset compression parameters.

[0085] In the embodiments of this specification, the target encoding protocol may be a set of encoding rules used to define the encoding format, frame organization, and bitstream structure of the target video data. The target encoding protocol enables the second type of decoder to decode the target video data. The second type of decoder may be a decoding function module configured in the second terminal. The second type of decoder is used to decode the target video data conforming to the target encoding protocol to recover the image frames for presentation in the second terminal. The target video data may be a video data sequence generated based on encoded data packets, and the target video data is used for decoding and display processing in the second terminal.

[0086] In one possible implementation, a target encoding protocol matching the second type of decoder can be determined from a preset set of encoding protocols based on the decoding capability information of the second terminal. The decoding capability information may include parameters such as the encoding format identifiers supported by the second type of decoder, the supported image resolution range, and the supported frame organization forms. After determining the target encoding protocol, the encoding process can be constrained when encoding the target and background regions to ensure that the encoding format and encapsulation structure of the encoded data packets meet the requirements of the target encoding protocol. Specifically, when encoding the target region, a first compression parameter can be combined with the encoding format and image parameters specified in the target encoding protocol to maintain a correspondence between the image encoding quality configuration and the format constraints of the target encoding protocol in the encoded data packets obtained based on the target region. When encoding the background region, a second compression parameter can be combined with the encoding format and image parameters specified in the target encoding protocol to ensure that the encoded data packets obtained based on the background region correspond to the second compression parameter in image encoding quality configuration and to the target encoding protocol in data organization form. The encoded data packets obtained based on the above encoding process can be arranged according to the frame structure and bit stream organization method specified by the target encoding protocol during encapsulation to form target video data that meets the decoding requirements of the second type of decoder.

[0087] For example, the second type of decoder can be a decoding module supporting the MJPEG encoding protocol, and the target encoding protocol can be a video data encapsulation protocol based on JPEG image encoded data. In this example, JPEG encoding can be used to encode the target area and the background area, and the first compression parameter and the second compression parameter are used to configure the JPEG encoding parameters corresponding to the target area and the background area, respectively. The encoded image encoded data can be combined according to the frame order, frame boundary markers, and control information specified by the target encoding protocol to form target video data. The target video data can be decoded frame by frame based on the second type of decoder to obtain decoded image frames for display.

[0088] In one optional implementation, after generating the target video data, second terminal identification information and transmission path information can be associated with the target video data. If a communication connection is available, the target video data can be sent to the second terminal in frame order through a preset transmission channel. Optionally, frame sequence numbers or timestamp information can be assigned to each frame in the target video data, enabling the second terminal to sort and cache frames based on the frame sequence numbers or timestamp information when receiving the target video data. Further, after receiving the target video data, the second terminal can decode the target video data using a second type of decoder to obtain decoded image frames for display, which are then presented on the second terminal's display screen.

[0089] In practical applications, by introducing a target encoding protocol adapted to the second type of decoder when encoding the target area and background area, and sending the target video data to the second terminal after generating the target video data, the target video data can meet the image encoding quality configuration determined by the preset compression parameters while conforming to the decoding capability of the second type of decoder. This helps to maintain the decoding compatibility of the target video data between different types of second terminals and configure the transmission of the target video data in network environments with large differences in transmission conditions.

[0090] In one possible implementation, a target sub-image corresponding to the target region is obtained based on the target region; the target sub-image is encoded according to a first compression parameter to obtain a first encoded data packet; the frame boundary of the original image frame and the target boundary of the target region are obtained; based on the positional relationship between the target boundary and the frame boundary, the image corresponding to the background region is divided into multiple background sub-images; based on a second compression parameter, the multiple background sub-images are encoded respectively to obtain multiple second encoded data packets.

[0091] In the embodiments of this specification, the target sub-image can be a subset of image data extracted from the original image frame based on a target region. The target region is used to indicate the range of pixels in the original image frame that need to be encoded according to the first compression parameters, and the target sub-image is used as an independent image encoding input during the image encoding process.

[0092] The first encoded data packet may be an encoded data unit obtained by encoding the target sub-image according to the first compression parameters; the second encoded data packet may be an encoded data unit obtained by encoding each of the multiple background sub-images according to the second compression parameters. The encoded data packet may include the first encoded data packet and multiple second encoded data packets.

[0093] Frame boundaries can be the set of boundary positions of the original image frame in pixel coordinates, used to limit the effective pixel range of the original image frame in the horizontal and vertical directions; target boundaries can be the set of boundary positions of the target region in pixel coordinates, used to limit the geometric range of the target region within the frame boundaries.

[0094] Background sub-images can be multiple subsets of image data obtained by dividing the background region, with each background sub-image corresponding to a sub-region within the background region. The image coding block size can be a combination of parameters in pixel coordinates representing the width and height of the partitioning unit during image encoding. The image coding block size is used to limit the pixel granularity of image data partitioning during image encoding. It is possible to ensure that the pixel dimensions of the target sub-image and each background sub-image in the horizontal and vertical directions are integer multiples of the number of pixels corresponding to the image coding block size.

[0095] In one possible implementation, the original image frame can be cropped based on the boundary coordinates of the target region in the pixel coordinate system to obtain a target sub-image. The pixel dimensions of the target sub-image in the horizontal and vertical directions can be consistent with the pixel dimensions of the target region in the pixel coordinate system. If an image coding block size exists, the boundary coordinates of the target region can be aligned based on the image coding block size, ensuring that the pixel dimensions of the target sub-image in the horizontal and vertical directions are integer multiples of the number of pixels in the corresponding direction of the image coding block size, and that the starting coordinates of the target sub-image in the pixel coordinate system satisfy an integer multiple relationship with the number of pixels corresponding to the image coding block size in the corresponding direction. Image encoding can be performed on the target sub-image according to the first compression parameters to obtain a first encoded data packet. The boundary coordinates of the frame boundary and the target boundary can be read, and based on the relative positional relationship between the frame boundary and the target boundary in the horizontal and vertical directions, the image corresponding to the background region can be divided to obtain multiple background sub-images. The pixel set resulting from combining multiple background sub-images in the pixel coordinate system can cover the pixel range corresponding to the background region, and the pixel set of multiple background sub-images and the target sub-image, when combined in the pixel coordinate system, can correspond to the pixel set of the original image frame. When an image coding block size exists, the starting coordinates and pixel size of each background sub-image in the horizontal and vertical directions can be made an integer multiple of the number of pixels in the corresponding direction of the image coding block size during background region division. This ensures that the background sub-images match the granularity of the image coding block division during subsequent image coding. Based on the second compression parameters, image coding operations can be performed on each of the multiple background sub-images to obtain multiple second-coded data packets.

[0096] In one example, the original image frame can be a color image with a resolution of 1280×800 pixels, and the target region can be a face region determined by a face detection algorithm. The top-left corner coordinates of the target region in the pixel coordinate system can be (x1, y1), and the bottom-right corner coordinates can be (x2, y2). Target sub-images can be cropped from the original image frame based on coordinates (x1, y1) and (x2, y2), and image encoding of the target sub-images can be performed using the first compression parameters based on the JPEG (Joint Photographic Experts Group) encoding method to obtain a first encoded data packet. The top-left corner coordinates of the frame boundary in the pixel coordinate system can be (0, 0), and the bottom-right corner coordinates can be (1279, 799). Based on the horizontal and vertical positional relationship between the frame boundary and the target boundary, the background region can be divided into several rectangular background sub-images, each identified by its corresponding top-left and bottom-right corner coordinates in the pixel coordinate system. The second compression parameters based on the JPEG encoding method can be used to image encode each background sub-image to obtain multiple second encoded data packets. In this example, the image encoding block size can be set to 12 pixels in both the horizontal and vertical directions. When determining x1, x2, y1, and y2, the values ​​of x1, x2, y1, and y2 in the corresponding directions can be multiples of 12, so that the pixel width and pixel height of the target sub-image and each background sub-image are multiples of 12.

[0097] In practical applications, by obtaining target sub-images based on the target region and encoding them according to the first compression parameter, and simultaneously dividing the background region based on the frame boundary and the target boundary and encoding multiple background sub-images according to the second compression parameter, the encoded data packet can include a first encoded data packet for representing the target region image data and multiple second encoded data packets for representing the background region image data. This establishes a correspondence between image encoding quality allocation and encoding data volume allocation when encoding the original image frame.

[0098] In one possible implementation, the region between the line containing the first target edge and the edge of the first frame is defined as the first background sub-image; the region between the line containing the second target edge and the edge of the second frame is defined as the second background sub-image; the region between the line containing the first target edge and the line containing the second target edge is defined as the intermediate strip region; the intermediate strip region includes the target sub-image; based on the third target edge and the fourth target edge, the third background sub-image and the fourth background sub-image in the intermediate strip region are determined; wherein, the first direction is the height direction or width direction of the original image frame; the second direction is perpendicular to the first direction.

[0099] In the embodiments described in this specification, the first direction may be the height direction or the width direction of the original image frame, and the second direction may be the direction perpendicular to the first direction in the pixel coordinate system.

[0100] The first frame edge can be an edge in the frame boundary in the first direction, and the second frame edge can be another frame boundary in the first direction opposite to the first frame edge.

[0101] The first target edge can be an edge in the target boundary in the first direction, the second target edge can be another target boundary in the first direction opposite to the first target edge, the third target edge can be an edge in the target boundary in the second direction, and the fourth target edge can be another target boundary in the second direction opposite to the third target edge.

[0102] The intermediate strip region can be the area defined by the straight line containing the first target edge and the straight line containing the second target edge in the pixel coordinate system. The intermediate strip region includes the target sub-image and the background region located above and below the target sub-image.

[0103] The first background sub-image can be a subset of image data obtained based on the image region between the line where the first target edge is located and the edge of the first frame. The second background sub-image can be a subset of image data obtained based on the image region between the line where the second target edge is located and the edge of the second frame. The third background sub-image can be a subset of image data obtained based on the image region defined by the third target edge in the middle strip region. The fourth background sub-image can be a subset of image data obtained based on the image region defined by the fourth target edge in the middle strip region.

[0104] In one possible implementation, the frame boundaries of the original image frame and the target boundaries of the target region can be obtained in a pixel coordinate system, and a first direction and a second direction can be determined. When the first direction is the height direction of the original image frame, the first frame edge can correspond to the top boundary of the original image frame in the height direction, the second frame edge can correspond to the bottom boundary of the original image frame in the height direction, the first target edge and the second target edge can correspond to the upper and lower boundaries of the target region in the height direction, respectively, and the third and fourth target edges can correspond to the left and right boundaries of the target region in the width direction. When the first direction is the width direction of the original image frame, the first frame edge and the second frame edge can correspond to the left and right boundaries of the original image frame in the width direction, the first target edge and the second target edge can correspond to the left and right boundaries of the target region in the width direction, and the third and fourth target edges can correspond to the upper and lower boundaries of the target region in the height direction. Based on the selection of the first and second directions, the original image frame can be divided into several rectangular regions defined by straight lines in the pixel coordinate system. The region between the line containing the first target edge and the edge of the first frame can be defined as the region corresponding to the first background sub-image. The region between the line containing the second target edge and the edge of the second frame can be defined as the region corresponding to the second background sub-image. The region between the lines containing the first and second target edges can be defined as the intermediate strip region, which includes the region corresponding to the target sub-image. Based on the positions of the third and fourth target edges in the second direction, the intermediate strip region can be divided into three adjacent regions in the second direction. The middle region corresponds to the target sub-image, the upper region corresponds to the region corresponding to the third background sub-image, and the lower region corresponds to the region corresponding to the fourth background sub-image. Through the above division, the pixel sets corresponding to the first, second, third, and fourth background sub-images, combined with the pixel set corresponding to the target sub-image, can cover the pixel set of the original image frame.

[0105] In an alternative embodiment, when determining the coordinate positions of the first frame edge, the second frame edge, the first target edge, the second target edge, the third target edge, and the fourth target edge in the pixel coordinate system, the coordinate values of the above edges in the horizontal and vertical directions should satisfy an integer multiple relationship with the number of pixels corresponding to the preset image coding block size. The image coding block size can be consistent with the image coding block size used in the embodiment of claim 2 and is used as the pixel granularity for dividing image data during the image coding process. In this alternative embodiment, the pixel widths and pixel heights of the first background sub-image, the second background sub-image, the third background sub-image, and the fourth background sub-image in the horizontal and vertical directions can be integer multiples of the number of pixels of the image coding block size in the corresponding direction, and the starting coordinates of each background sub-image in the corresponding direction are integer multiples of the number of pixels of the image coding block size in that direction.

[0106] In one example, the original image frame can be an image with a resolution of 1280×800 pixels. In the pixel coordinate system, the horizontal direction can be defined as the second direction, and the vertical direction can be defined as the first direction. The first frame edge of the frame boundary in the first direction can be the position where the vertical pixel coordinate y is equal to 0, and the second frame edge can be the position where the vertical pixel coordinate y is equal to 799. The first target edge of the target area in the first direction can be the position where the vertical pixel coordinate y is equal to y1, and the second target edge can be the position where the vertical pixel coordinate y is equal to y2. The third target edge of the target area in the second direction can be the position where the horizontal pixel coordinate x is equal to x1, and the fourth target edge can be the position where the horizontal pixel coordinate x is equal to x2, where 0≤y1≤y2≤799 and 0≤x1≤x2≤1279. The image area corresponding to the pixels satisfying 0≤y<y1 can be determined as the area corresponding to the first background sub-image, the image area corresponding to the pixels satisfying y2<y≤799 can be determined as the area corresponding to the second background sub-image, and the image area corresponding to the pixels satisfying y1≤y≤y2 can be determined as the middle strip area. In the middle strip area, the area corresponding to the pixels satisfying x<x1 can be determined as the area corresponding to the third background sub-image, the area corresponding to the pixels satisfying x1≤x≤x2 and y1≤y≤y2 can be determined as the area corresponding to the target sub-image, and the area corresponding to the pixels satisfying x>x2 can be determined as the area corresponding to the fourth background sub-image. Exemplarily, the image coding block size can be set to 12 pixels in both the horizontal and vertical directions. When determining y1, y2, x1, and x2, the values of y1 and y2 in the first direction are integer multiples of 12, and the values of x1 and x2 in the second direction are integer multiples of 12, so that the pixel widths and pixel heights of the target sub-image and each background sub-image satisfy an integer multiple relationship with the number of pixels of the image coding block size in the corresponding direction.

[0107] In one alternative implementation, when the target region is adjacent to a frame edge in the original image frame, when dividing based on the frame boundary and the target boundary in the corresponding direction, the region corresponding to a background sub-image can degenerate into a region with zero width or zero height in the pixel coordinate system. At this time, the background sub-image can be regarded as not containing effective pixels.

[0108] In practical applications, by dividing the background region into a first background sub-image, a second background sub-image, a third background sub-image, and a fourth background sub-image based on the relative positional relationship between the frame boundary and the target boundary in the pixel coordinate system, the background region can be geometrically divided into several regular rectangular background sub-images. This helps to encode, transmit, and stitch the target sub-image and each background sub-image separately in subsequent encoding and image reconstruction processes.

[0109] In step S211, target video data is generated based on the encoded data packets.

[0110] In the embodiments of this specification, the target video data can be a collection of video data organized based on encoded data packets. The target video data is used to represent multiple encoded image frames in the time dimension. The target video data may include multiple video data units arranged in chronological order, each video data unit being associated with at least one encoded data packet. The video data units are used to reconstruct the image content corresponding to the original image frames during decoding.

[0111] In one possible implementation, coded data packets obtained from the same original image frame can be grouped into a single frame-level data unit, and frame identification information can be assigned to this unit. The frame identification information can be used to characterize the position of the frame-level data unit within the target video data. The frame-level data units can be arranged according to the order of the frame identification information to form target video data containing multiple frame-level data units. Each frame-level data unit can include a set of coded data packets associated with the corresponding original image frame to maintain the association between the coded data packets and the original image frame at the frame-level granularity. During the generation of the target video data, the association between the coded data packets within frames can be maintained, ensuring that coded data packets corresponding to the same original image frame belong to the same frame-level data unit in the target video data.

[0112] For example, the encoded data packets corresponding to each original image frame can be combined into a video data unit. The video data unit may include encoded data packets representing the target area and encoded data packets representing the background area, and may also include a timestamp field or a frame sequence number field representing the position of the video data unit on the timeline. Multiple video data units, after being sorted according to the timestamp field or frame sequence number field, can be arranged sequentially to form target video data, which can be transmitted via a communication connection in bitstream form.

[0113] In one optional implementation, when generating the target video data, overall stream identification information, resolution information, and frame rate information can be configured for the target video data to indicate the basic attributes of the target video data at the decoding end. Optionally, video data units for boundary marking can be inserted into the target video data to distinguish target video data sequences corresponding to different video segments or different sessions. Further, a buffer queue can be configured for the target video data, which can be written to the buffer queue sequentially after generation so that it can be read in the order of generation when sent to the second terminal.

[0114] In practical applications, by generating target video data based on encoded data packets and maintaining the intra-frame correlation between encoded data packets for the same original image frame in the target video data, the subsequent transmission, caching, and decoding of the target video data can be managed based on a frame-level organization, which helps to maintain the one-to-one correspondence between encoded data packets and original image frames in continuous video processing scenarios.

[0115] In one possible implementation, the geometric parameters corresponding to the target sub-image and each background sub-image are obtained to obtain a first position parameter and multiple second position parameters; based on the first position parameter and multiple second position parameters, layout description information is generated; the encoded data packet and the layout description information are encapsulated to obtain the target video data.

[0116] In the embodiments of this specification, geometric parameters can be a set of parameters used to characterize the position and size of the target sub-image and each background sub-image in the pixel coordinate system of the original image frame. Geometric parameters may include the horizontal and vertical coordinate values ​​representing the top-left corner pixel coordinates, and the width and height pixel counts representing the image size. When an image coding block size exists, the width and height pixel counts in the geometric parameters can satisfy an integer multiple relationship with the number of pixels in the corresponding direction of the image coding block size, and the horizontal and vertical coordinate values ​​in the geometric parameters can satisfy an integer multiple relationship with the number of pixels in the corresponding direction of the image coding block size, so as to maintain consistency with the partitioning granularity based on the image coding block size during image coding and image reconstruction.

[0117] The first positional parameter can be a geometric parameter corresponding to the target sub-image, used to characterize the coordinate position and image size of the target sub-image in the pixel coordinate system of the original image frame. Multiple second positional parameters can be geometric parameters corresponding to multiple background sub-images, used to characterize the coordinate position and image size of the multiple background sub-images in the pixel coordinate system of the original image frame. The layout description information can be descriptive data generated based on the first positional parameter and the multiple second positional parameters. The layout description information is used to indicate the relative positional relationship between the target sub-image and each background sub-image in the pixel coordinate system of the original image frame, and is used to carry position-related information in the target video data.

[0118] In one possible implementation, after obtaining the target sub-image and multiple background sub-images, the geometric parameters corresponding to the target sub-image and each background sub-image can be obtained based on their pixel coverage in the original image frame. The geometric parameters corresponding to the target sub-image can be defined as first positional parameters, and the geometric parameters corresponding to each background sub-image can be defined as multiple second positional parameters. In scenarios where image encoding is performed using image coding block size, when determining the geometric parameters of the target sub-image and each background sub-image, the boundary positions of the target sub-image and each background sub-image in the pixel coordinate system can be aligned based on the image coding block size, ensuring that the parameter values ​​representing position and size in the geometric parameters satisfy an integer multiple relationship with the number of pixels in the corresponding direction of the image coding block size. Layout description information can be generated based on the first positional parameters and multiple second positional parameters. The layout description information can set record entries for the target sub-image and each background sub-image, and each record entry can include an identifier field for identifying the sub-image's identity and parameter fields for representing the sub-image's coordinate position and image size in the pixel coordinate system. The layout description information can establish a correspondence between the first position parameter and the coded data packet corresponding to the target sub-image, and establish correspondences between multiple second position parameters and the coded data packets corresponding to multiple background sub-images respectively. When generating the target video data, the coded data packets and the layout description information can be encapsulated so that the target video data simultaneously contains image coded data and layout description information for image stitching at the frame-level granularity.

[0119] For example, the original image frame can be an image with a resolution of 1280×800 pixels, where the horizontal axis coordinates in the pixel coordinate system can range from 0 to 1279, and the vertical axis coordinates can range from 0 to 799. The target sub-image can correspond to a pixel region with horizontal axis coordinates between x1 and x2 and vertical axis coordinates between y1 and y2. The first position parameter can include the upper left corner coordinates (x1, y1), as well as the width (x2−x1+1) and height (y2−y1+1). The first background sub-image, the second background sub-image, the third background sub-image, and the fourth background sub-image can each correspond to several rectangular regions other than the target sub-image, and the multiple second position parameters can each include the upper left corner coordinates, as well as the width and height of the corresponding background sub-image. The layout description information can be organized in structured text or structured binary form. Each record entry in the layout description information can include a sub-image type field, a top-left corner coordinate field, and a size field. The sub-image type field is used to distinguish between the target sub-image and each background sub-image, and the top-left corner coordinate field and the size field are used to represent a first position parameter or multiple second position parameters. The encoded data packet can include a first encoded data packet and multiple second encoded data packets. The first position parameter can be associated with the first encoded data packet in the layout description information, and multiple second position parameters can be associated with each of the multiple second encoded data packets. In the implementation of image encoding based on a fixed image encoding block size, the image encoding block size can be set to 12 pixels in both the horizontal and vertical directions. When determining (x1, y1), (x2, y2), and the top-left corner coordinates, width, and height of each background sub-image, the coordinate values, width, and height are integer multiples of 12 in the corresponding directions, so that image reconstruction can be performed on the decoding side according to the granularity of the image encoding block based on the layout description information and the encoded data packet.

[0120] In an optional implementation, the layout description information may further include the overall resolution information of the original image frame, used to recover the pixel coordinate system of the original image frame at the decoding end. Optionally, the record entries in the layout description information can be arranged in a preset order. For example, the record entries corresponding to the target sub-image can be arranged at the beginning of the layout description information, and the record entries corresponding to multiple background sub-images can be arranged at the end of the layout description information, so as to prioritize the acquisition of the position parameters of the target sub-image when parsing the layout description information. When encapsulating the target video data, a layout description information segment and a set of encoded data packets corresponding to the layout description information of that frame can be configured for each frame of target video data, so that each frame of target video data structurally includes a position description part and an image encoding part.

[0121] In practical applications, by obtaining the geometric parameters corresponding to the target sub-image and each background sub-image during the generation of target video data based on encoded data packets, a first position parameter and multiple second position parameters are obtained. Based on the first position parameter and multiple second position parameters, layout description information is generated. Then, the encoded data packet and layout description information are encapsulated. This allows the target video data to carry information indicating the relative positions of the target sub-image and each background sub-image in the pixel coordinate system while carrying the image encoded data. This helps to reconstruct and stitch the target sub-image and each background sub-image based on the target video data on the decoding side, so as to restore the overall picture layout corresponding to the original image frame.

[0122] In one possible implementation, after the step of generating target video data based on encoded data packets, the target video data is sent to a second terminal.

[0123] In the embodiments described in this specification, the second terminal may be a video presentation device configured with a display component and a second type of decoder. It can receive target video data and decode the target video data based on the second type of decoder to obtain decoded image frames for display.

[0124] The target video data can be a collection of multiple video data units organized in a preset order. Each video data unit can correspond to one or more frames of image encoded data and control information associated with the image encoded data. The communication connection can be a data transmission link established between the second terminal and the server for transmitting the target video data. The communication connection can be a wired communication link or a wireless communication link.

[0125] In one possible implementation, a long-term communication connection can be established between the second terminal and the server. When the communication connection is available, the target video data is divided into multiple transmission units according to frame order. Each transmission unit may contain encoded data corresponding to at least one video data unit, as well as frame sequence number information or timestamp information representing the position of that video data unit in the target video data. Based on the frame sequence number information or timestamp information, each transmission unit can be sent in a preset order within the communication connection, enabling the second terminal to reassemble the transmission units according to the frame sequence number information or timestamp information when receiving the target video data, thereby recovering the frame sequence corresponding to the target video data. In an implementation where layout description information and encoded data packets are used together to represent the content of the original image frame, the transmission unit may also include layout description information associated with the corresponding video data unit, so that the second type of decoder can recover the screen layout corresponding to the original image frame based on the layout description information after decoding the image encoded data.

[0126] For example, the second terminal can be an electronic photo frame device equipped with a display screen and a second type of decoder supporting MJPEG decoding. The communication connection can be a long-lived connection based on the Transmission Control Protocol (TCP). The target video data can be encapsulated into multiple application layer messages, each carrying encoded data corresponding to a video data unit and a frame sequence number field. After receiving the application layer messages, the second terminal can sort the video data units based on the frame sequence number field and input the sorted video data units into the second type of decoder for decoding to obtain decoded image frames for display.

[0127] In one optional implementation, a verification information field can be attached to each transmission unit when sending the target video data, for use by the second terminal to perform integrity verification on the transmission unit. Optionally, retransmission control parameters can be configured in the communication connection to trigger retransmission control when a transmission unit is detected as lost or verification fails, thereby maintaining the continuity of the target video data at the communication layer. Furthermore, a receive buffer can be configured within the second terminal for the target video data, caching several video data units according to frame sequence number information or timestamp information in the receive buffer, thereby reducing the sensitivity of the target video data to reception delay fluctuations during decoding and display.

[0128] In practical applications, after generating target video data based on encoded data packets, the target video data is sent to the second terminal, enabling the second terminal to decode the target video data and present the image content based on the second type of decoder. This can realize a video data interaction path between the first terminal, the server, and the second terminal based on different encoding formats.

[0129] For example, with Figure 3 For example, Figure 3 This is a flowchart illustrating a video frame differential encoding method based on the division of face and background regions, according to an exemplary embodiment. In the process, encoded image frames generated based on the H.264 format can be received, and the process of "receiving H.264 frames and decoding them into the original image" is performed, so that the encoded image frames are restored to the original image frames used to represent the complete picture content after decoding.

[0130] Based on the decoded original image frame, a "face detection & bounding box alignment (Alignment to 16×16 block)" process can be performed. Face detection can be performed on the original image frame to obtain one or more candidate regions for representing face objects, and a bounding box is generated for each candidate region. Based on the constraint of alignment with the coded macroblock, the boundary positions of the bounding box can be adjusted so that the coordinates of the upper, lower, left, and right boundaries are aligned with the boundaries of the 16×16 pixel block. The image area covered by the aligned face bounding box is regarded as the "face region (Img_Face)" in this process and used as a key region in subsequent processing stages.

[0131] After the face region (Img_Face) is determined, dynamic image segmentation can be performed on the original image frame. Dynamic image segmentation divides the original image frame into two parts in the pixel coordinate system: a key region and a non-key region. The key region corresponds to the face region (Img_Face), and the non-key region corresponds to the remaining pixel set. To accommodate subsequent encoding processing and display stitching, the non-key region can be further divided into "background regions (Top / Bot / L / R)". This means that the background region is decomposed into the Top sub-region above the face region, the Bot sub-region below the face region, and the L and R sub-regions on the left and right sides of the face region, so that the original image frame is geometrically composed of one face region and four background sub-regions.

[0132] Based on the completed region segmentation, differentiated JPEG encoding configurations can be applied to different regions. The face region (Img_Face) can enter the "High-Quality JPEG Encoding (Q=98%)" branch, using JPEG encoding with the quality factor Q set to the example value of 98. This compresses the image data corresponding to the face region, preserving as much of the contour and texture information related to the face as possible. The background regions (Top / Bot / L / R) can enter the "Low-Quality JPEG Encoding (Q=50%)" branch, using the same JPEG encoding method with the quality factor Q set to the example value of 50. This compresses the Top, Bot, L, and R sub-regions separately, reducing the amount of encoded data for the background while maintaining basic recognizability. Through this differentiated configuration, one high-quality JPEG encoded image corresponding to the face region and four low-quality JPEG encoded images corresponding to each background sub-region can be obtained.

[0133] After JPEG encoding is complete, the "packaging: 5 JPEGs + coordinate metadata" process can be performed. The high-quality JPEG encoded image corresponding to the face region (Img_Face) can be combined with four low-quality JPEG encoded images corresponding to the Top, Bot, L, and R sub-regions into 5 JPEG image data units, and coordinate metadata is generated for each JPEG image data unit. The coordinate metadata characterizes the top-left corner coordinates, width, and height of the JPEG image data unit in the original image frame pixel coordinate system, and is used for layout restoration and image stitching on the terminal side. Through this packaging method, a frame-level data structure consisting of 5 JPEG image data units and corresponding coordinate metadata can be obtained. The JPEG image data corresponding to the face region (Img_Face) is associated with the first compression parameter, and the JPEG image data corresponding to the background regions (Top / Bot / L / R) is associated with the second compression parameter. The 5 JPEG image data units can correspond to a first encoded data packet and multiple second encoded data packets, and the coordinate metadata provides layout description information used to describe the positional relationship between the target sub-image and multiple background sub-images.

[0134] For example, with Figure 4 For example, Figure 4 This is a schematic diagram illustrating the uplink and differentiated downlink video data interaction between a resource-constrained electronic photo frame and a cloud server, according to an exemplary embodiment. Client A, as the acquisition end, is a resource-constrained device, corresponding to the first terminal, and is equipped with a camera module, an MCU main controller, and a hardware H.264 encoder. The camera module can acquire continuous image data of the local scene at a resolution of 1280×800 pixels. The MCU main controller can send the acquired image data to the hardware H.264 encoder for compression encoding, generating an H.264 bitstream for the server. The uplink in the diagram corresponds to "Uplink: H.264 Bitstream".

[0135] The cloud processing server, acting as a corresponding server, decodes, analyzes, and re-encodes the H.264 bitstream from client A. The H.264 decoding module performs decoding on the H.264 bitstream to recover the original image frame representing a single frame. The "Face Detection & Dynamic Partitioning (Face Analysis)" module detects faces in the original image frame, determines the boundary between the face and background regions, and obtains the target and background regions based on the partitioning results. The differential compression engine, under the configuration of "Face:98%, BG:50%", applies a JPEG encoding configuration with an example quality factor of 98 to the target region and a JPEG encoding configuration with an example quality factor of 50 to the background region. This splits a single original image frame into a high-quality JPEG-encoded image representing the face region and multiple low-quality JPEG-encoded images representing the background region, and generates layout metadata based on the region locations. After processing by the differential compression engine, a downlink data structure of "5-channel JPEG + layout metadata" can be formed. The 5-channel JPEG can correspond to the image encoding results of the face region and four background sub-regions, and the layout metadata is used to indicate the positional relationship of each sub-image in the original image frame.

[0136] Client B, acting as the playback end, is also a resource-constrained device and can serve as a second terminal. Internally, it is equipped with an MCU main controller, a hardware JPEG decoder, and an LCD display. The MCU main controller receives "downlink: 5-channel JPEG + layout metadata" from the cloud processing server via a network interface. It then inputs the 5-channel JPEG encoded images into the hardware JPEG decoder for decoding, obtaining the corresponding face sub-images and various background sub-images. Under the constraints of the layout metadata, the LCD display performs stitching and rendering processing on the decoded multiple sub-images, corresponding to "LCD display (stitching and rendering)," ensuring that the overall display image is presented on the terminal side at the resolution of the original image frames.

[0137] In this interactive process, "Client A (acquisition end) resource-constrained device" and "Client B (playback end) resource-constrained device" respectively undertake the roles of video data acquisition and presentation. The cloud processing server constitutes the intermediate processing link through "H.264 decoding module", "face detection & dynamic partitioning (Face Analysis)" and "differentiated compression engine (Face: 98%, BG: 50%)". Uplink transmits H.264 bitstream and downlink transmits 5-channel JPEG and layout metadata, thereby establishing a video data transmission path based on cloud-based differentiated compression processing between resource-constrained electronic photo frames.

[0138] Figure 5 This is a block diagram of a video data processing system according to an exemplary embodiment. (Refer to...) Figure 5 The system may include: Image acquisition module 501 is used to acquire raw image frames; the raw image frame represents a single frame of image data in the video data stream. The object recognition module 503 is used to identify target objects in the original image frame and determine the target area where the target object is located. Background determination module 505 is used to determine a background region based on the target region; the background region represents the region in the original image frame other than the target region; The parameter acquisition module 507 is used to acquire preset compression parameters; the preset compression parameters include a first compression parameter and a second compression parameter; the image encoding quality corresponding to the first compression parameter is higher than the image encoding quality corresponding to the second compression parameter. The classification encoding module 509 is used to encode the target area and the background area respectively according to the preset compression parameters to obtain the encoded data packet; The video generation module 511 is used to generate target video data based on encoded data packets.

[0139] In one possible implementation, the encoded data packet includes a first encoded data packet and a plurality of second encoded data packets; the classification encoding module 509 includes: The target sub-image determination unit is used to obtain the target sub-image corresponding to the target region based on the target region; The target sub-image encoding unit is used to encode the target sub-image according to the first compression parameters to obtain the first encoded data packet; The boundary acquisition unit is used to acquire the frame boundary of the original image frame and the target boundary of the target region; The background segmentation unit is used to divide the image corresponding to the background region into multiple background sub-images based on the positional relationship between the target boundary and the frame boundary. The background sub-image encoding unit is used to encode multiple background sub-images based on the second compression parameters to obtain multiple second encoded data packets.

[0140] In one possible implementation, the target boundary includes a first target edge and a second target edge opposite each other in a first direction, and a third target edge and a fourth target edge opposite each other in a second direction; the frame boundary of the original image frame includes a first frame edge and a second frame edge opposite each other in a first direction; the background segmentation unit includes: The first background determination sub-unit is used to determine the area between the straight line where the first target edge is located and the edge of the first frame as the first background sub-image; The second background determination sub-unit is used to determine the area between the straight line where the second target edge is located and the edge of the second frame as the second background sub-image; The intermediate region determination sub-unit is used to define the area between the straight line containing the first target edge and the straight line containing the second target edge as the intermediate strip region; the intermediate strip region includes the target sub-image; The third and fourth background determination sub-units are used to determine the third background sub-image and the fourth background sub-image in the middle strip region based on the third target edge and the fourth target edge; The first direction is the height or width direction of the original image frame; the second direction is perpendicular to the first direction.

[0141] In one possible implementation, the server establishes communication connections with both the first terminal and the second terminal; the first terminal is equipped with a first type encoder; the second terminal is equipped with a second type decoder; the image acquisition module 501 includes: A video stream receiving unit is used to receive a first format video stream sent by a first terminal; the first format video stream is generated by the first terminal based on a first type encoder. The decoding unit is used to decode the first format video stream to obtain the original image frames; Classification coding module 509 includes: The protocol acquisition unit is used to acquire the target encoding protocol adapted to the second type of decoder; The protocol and parameter determination encoding unit is used to encode the target region and the background region respectively according to the target encoding protocol and preset compression parameters; It may also include: The video sending module is used to send target video data to the second terminal.

[0142] In one possible implementation, the video generation module 511 includes: The position parameter determination unit is used to obtain the geometric parameters corresponding to the target sub-image and each background sub-image, and to obtain a first position parameter and multiple second position parameters; the geometric parameters represent the coordinate position and image size in the pixel coordinate system of the original image frame; the first position parameter corresponds to the target sub-image; the multiple second position parameters correspond to multiple background sub-images; The layout description generation unit is used to generate layout description information based on a first position parameter and multiple second position parameters; the layout description information is used to indicate the relative positions of the target sub-image and each background sub-image in the pixel coordinate system. The encapsulation unit is used to encapsulate the encoded data packets and layout description information to obtain the target video data.

[0143] In one possible implementation, the object recognition module 503 includes: An object count detection unit is used to detect the number of objects in the original image frame; The area occupancy acquisition unit is used to acquire the area occupancy information of each candidate object in the original image frame when there are multiple candidate objects in the original image frame. The sorting unit is used to sort multiple candidate objects based on the occupied area information to obtain the sorting result; The target region determination unit is used to identify the candidate object that ranks first in the sorting results as the target object and to determine the region where the target object is located as the target region.

[0144] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0145] Figure 6 This is a block diagram illustrating an electronic device for video data processing according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. Electronic devices may include processors, memory, network interfaces, input devices, and output devices connected via a system bus.

[0146] The processor can be a general-purpose processor or a special-purpose processor. For example, the processor can be a CPU (Central Processing Unit), which can be used to execute computer programs and / or data stored in memory. The memory can be used to store the operating system, application programs, and intermediate and configuration data used when executing video data processing methods. The memory can include any one or any combination of volatile memory and non-volatile memory; the volatile memory can be DRAM (Dynamic Random Access Memory), and the non-volatile memory can be flash memory or magnetic storage media.

[0147] A network interface enables electronic devices to communicate with a first terminal and a second terminal via wired or wireless networks. The network interface can support Ethernet access or Wi-Fi (Wireless Fidelity) based LAN access, allowing the electronic device to receive a first-format video stream from the first terminal and send target video data to the second terminal. Input devices can receive input information for configuring and managing the electronic device; these devices may include a keyboard, touchpad, or external input device interface. Output devices can present output information related to the operating status of the electronic device; these devices may include a display, indicator light assembly, or an interface device for outputting operating logs. A system bus connects the processor, memory, network interface, input devices, and output devices, providing a signal transmission channel between these components.

[0148] The memory may store at least one computer program, which may include multiple sets of instructions. When the computer program is executed by the processor, it implements a video data processing method. The video data processing method may include the processes described in the embodiments of this specification: acquiring original image frames, identifying target objects in the original image frames and determining target regions, determining background regions based on target regions, acquiring preset compression parameters, encoding the target regions and background regions according to the preset compression parameters to obtain encoded data packets, and generating target video data based on the encoded data packets. The processor can execute the instructions in the computer program to call the network interface to receive a first-format video stream sent by a first terminal, decode the first-format video stream to obtain original image frames, perform image analysis and encoding parameter configuration on the original image frames, generate target video data based on the encoded data packets, and send it to a second terminal through the network interface.

[0149] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0150] Embodiments of this disclosure also provide a computer storage medium, which can be disposed in a terminal to store at least one instruction or at least one program related to implementing a video data processing method in the method embodiments. The at least one instruction or at least one program is loaded and executed by the processor to implement the video data processing method provided in the above method embodiments.

[0151] Embodiments of this disclosure also provide a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the video data processing method provided in the above-described method embodiments. The computer-readable storage medium may be ROM (Read-Only Memory), RAM (Random Access Memory), flash memory, or other media capable of storing program code.

[0152] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0153] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system, electronic device, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments.

[0154] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0155] The above description is only a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A video data processing method, characterized in that, Applied to a server; the method includes: Acquire raw image frames; the raw image frames represent single frame image data in the video data stream; Identify the target object in the original image frame and determine the target region where the target object is located; Based on the target region, a background region is determined; the background region represents the area in the original image frame other than the target region. Obtain preset compression parameters; the preset compression parameters include a first compression parameter and a second compression parameter; the image encoding quality corresponding to the first compression parameter is higher than the image encoding quality corresponding to the second compression parameter. According to the preset compression parameters, the target region and the background region are encoded respectively to obtain an encoded data packet; Based on the encoded data packet, target video data is generated.

2. The method according to claim 1, characterized in that, The encoded data packet includes a first encoded data packet and multiple second encoded data packets; the process of encoding the target region and the background region according to the preset compression parameters to obtain the encoded data packet includes: Based on the target region, a target sub-image corresponding to the target region is obtained; The target sub-image is encoded according to the first compression parameters to obtain the first encoded data packet; Obtain the frame boundary of the original image frame and the target boundary of the target region; Based on the positional relationship between the target boundary and the frame boundary, the image corresponding to the background region is divided into multiple background sub-images; Based on the second compression parameters, the plurality of background sub-images are encoded respectively to obtain the plurality of second encoded data packets.

3. The method according to claim 2, characterized in that, The target boundary includes a first target edge and a second target edge that are opposite each other in a first direction, and a third target edge and a fourth target edge that are opposite each other in a second direction; the frame boundary of the original image frame includes a first frame edge and a second frame edge that are opposite each other in the first direction. The step of dividing the image corresponding to the background region into multiple background sub-images based on the positional relationship between the target boundary and the frame boundary includes: The region between the straight line containing the first target edge and the edge of the first frame is defined as the first background sub-image; The region between the straight line containing the second target edge and the edge of the second frame is defined as the second background sub-image; The region between the straight line containing the first target edge and the straight line containing the second target edge is defined as the intermediate strip region; the intermediate strip region includes the target sub-image; Based on the third target edge and the fourth target edge, the third background sub-image and the fourth background sub-image in the middle strip region are determined; Wherein, the first direction is the height direction or width direction of the original image frame; the second direction is perpendicular to the first direction.

4. The method according to claim 1, characterized in that, The server establishes communication connections with the first terminal and the second terminal respectively; the first terminal is equipped with a first type encoder. The second terminal is configured with a second type of decoder; the acquisition of the original image frame includes: The system receives a video stream in a first format sent by the first terminal; the first video stream in the first format is generated by the first terminal based on the first type of encoder. The first format video stream is decoded to obtain the original image frame; The step of encoding the target region and the background region according to the preset compression parameters includes: Obtain the target encoding protocol adapted to the second type of decoder; The target region and the background region are encoded according to the target encoding protocol and the preset compression parameters, respectively. After the step of generating target video data based on the encoded data packet, the method includes: The target video data is sent to the second terminal.

5. The method according to claim 2, characterized in that, The step of generating target video data based on the encoded data packet includes: Obtain the geometric parameters corresponding to the target sub-image and each background sub-image to obtain a first position parameter and multiple second position parameters; the geometric parameters represent the coordinate position and image size in the pixel coordinate system of the original image frame; the first position parameter corresponds to the target sub-image; the multiple second position parameters correspond to multiple background sub-images. Based on the first position parameter and the plurality of second position parameters, layout description information is generated; the layout description information is used to indicate the relative positions of the target sub-image and each of the background sub-images in the pixel coordinate system. The encoded data packet and the layout description information are encapsulated to obtain the target video data.

6. The method according to claim 1, characterized in that, The step of identifying the target object in the original image frame and determining the target region where the target object is located includes: Detect the number of objects in the original image frame; When there are multiple candidate objects in the original image frame, obtain the area occupied by each candidate object in the original image frame; Based on the occupied area information, the multiple candidate objects are sorted to obtain a sorting result; The candidate object that ranks first in the sorting results is determined as the target object, and the region where the target object is located is determined as the target region.

7. A video data processing system, characterized in that, include: The image acquisition module is used to acquire raw image frames; the raw image frames represent single frame image data in the video data stream. An object recognition module is used to identify target objects in the original image frame and determine the target region where the target object is located. The background determination module is used to determine the background region based on the target region; The background region represents the area in the original image frame other than the target region; The parameter acquisition module is used to acquire preset compression parameters; The preset compression parameters include a first compression parameter and a second compression parameter; The image encoding quality corresponding to the first compression parameter is higher than the image encoding quality corresponding to the second compression parameter. The classification encoding module is used to encode the target region and the background region respectively according to the preset compression parameters to obtain encoded data packets; The video generation module is used to generate target video data based on the encoded data packet.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the video data processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the video data processing method as described in any one of claims 1 to 6.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the video data processing method according to any one of claims 1 to 6.

Images