A method for processing a boundary of a key region in a video frame and a related device

Abstract

The application discloses a boundary processing method of a key region in a video frame and related equipment, and relates to the field of communication technology. In the method, an initial boundary curve of at least one initial key region in a video frame and a transition zone on both sides are acquired, and the transition zone includes at least one first pixel point. A gradient amplitude response of the first pixel point is calculated by using an edge detection algorithm, and the maximum value of the gradient amplitude response is taken as a local edge intensity. Contrast and information entropy of a local texture block where the first pixel point is located are calculated, and the product of the contrast and the information entropy is taken as a local texture complexity. The local edge intensity and the local texture complexity of the first pixel point are weighted and fused to obtain a boundary attribution confidence of the first pixel point. An energy functional model of any curve in the transition zone is constructed, and the energy functional model is minimized to obtain a boundary curve corresponding to an output minimum value of the energy functional model as a target boundary curve. The application can improve the transmission efficiency of the video frame and save transmission resources.

Description

Technical Field

[0001] This application belongs to the field of communication technology, specifically relating to a method and related equipment for boundary processing of key regions in a video frame. Background Technology

[0002] With the continuous deepening of information technology construction, video communication has become a key information carrier in various event dispatch systems, especially in emergency command and dispatch of sudden events. For example, at disaster sites such as earthquakes, fires, and hazardous chemical leaks, front-line rescue personnel use individual image transmission devices, drones, and other terminals to transmit on-site images back to the rear command center in real time. The rear command personnel then analyze the images and issue disposal instructions.

[0003] Because on-site communication environments are often limited by public network congestion or damage, existing technologies have developed methods to prioritize the transmission of critical visual information within effective bandwidth by differentially encoding different regions within a single video frame.

[0004] However, in existing technologies, the boundary positioning accuracy is only at the coarse-grained level, and the box often contains a large number of non-critical pixels. When the encoding bitrate of critical areas is higher than that of non-critical areas, the resulting video frame encoding data volume is large, resulting in low transmission efficiency and high transmission resource consumption. Summary of the Invention

[0005] This invention provides a method and related equipment for boundary processing of key regions in a video frame, which can improve video frame transmission efficiency and save transmission resources.

[0006] To address the aforementioned problems, this invention discloses a method for boundary processing of key regions in a video frame, the method comprising: The process involves: acquiring initial boundary curves for at least one initial key region in a video frame; obtaining transition bands on both sides of the initial boundary curves, each transition band including at least one first pixel; calculating the gradient magnitude response of the first pixel in multiple directions using an edge detection algorithm, and selecting the largest gradient magnitude response as the local edge intensity of the first pixel; calculating the contrast and entropy of the local texture block where the first pixel is located, and using the product of the contrast and entropy as the local texture complexity of the first pixel; weighted fusing the local edge intensity and local texture complexity of the first pixel to obtain the boundary attribution confidence of the first pixel, which characterizes the probability that the first pixel belongs to the boundary curve of the initial key region; constructing an energy functional model corresponding to any curve in the transition band, and minimizing the energy functional model to obtain the boundary curve corresponding to the minimum output of the energy functional model as the target boundary curve, where the energy functional model is the sum of squared curvatures of each first pixel on any curve minus the boundary attribution confidence of each first pixel on any curve.

[0007] On the other hand, embodiments of the present invention disclose a boundary processing device for key regions in a video frame, the device comprising: The boundary acquisition module is used to acquire the initial boundary curve of at least one initial key region in the video frame, and to acquire the transition bands located on both sides of the initial boundary curve, wherein the transition bands include at least one first pixel.

[0008] The first calculation module is used to calculate the gradient magnitude response of the first pixel in multiple directions through an edge detection algorithm, and select the largest gradient magnitude response as the local edge intensity of the first pixel.

[0009] The second calculation module is used to calculate the contrast and information entropy of the local texture block where the first pixel is located, and to use the product of the contrast and information entropy of the local texture block as the local texture complexity of the first pixel.

[0010] The fusion module is used to perform weighted fusion of the local edge intensity and local texture complexity of the first pixel to obtain the boundary attribution confidence of the first pixel. The boundary attribution confidence is used to characterize the probability that the first pixel belongs to the boundary curve of the initial key region.

[0011] The output module is used to construct the energy functional model corresponding to any curve in the transition zone, and to minimize the energy functional model to obtain the boundary curve corresponding to the minimum value of the energy functional model output as the target boundary curve. The energy functional model is the sum of the squares of the curvature of each first pixel point on any curve minus the boundary assignment confidence of each first pixel point on any curve.

[0012] In another aspect, embodiments of the present invention disclose an electronic device, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, which causes the processor to execute the aforementioned method.

[0013] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described above.

[0014] This invention also discloses a computer program product, including a computer program for implementing the boundary processing method for key regions in the aforementioned video frame.

[0015] The embodiments of this invention offer the following advantages: For each pixel in the transition zone on both sides of the initial critical region, its boundary attribution confidence is evaluated by combining local edge strength and local texture complexity to indicate the probability that it belongs to the target boundary curve. Based on this boundary attribution confidence and an energy functional model, the corrected target boundary curve is obtained from the transition zone, which helps improve the accuracy of the target boundary curve and avoids including some non-critical pixels. When subsequent encoding of the target critical region at a higher bitrate, encoding resources can be effectively saved, the amount of data after video frame encoding can be reduced, thereby improving transmission efficiency and saving transmission resources. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below.

[0017] Figure 1 This is a schematic diagram of a wireless communication scenario provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of a video frame region division provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the relationship between an initial boundary curve, a target boundary curve, and a transition zone, provided in an embodiment of this application. Figure 4 This is a flowchart illustrating the steps of a method for boundary processing of key regions in a video frame, as provided in an embodiment of the present invention. Figure 5 This is a block diagram of a boundary processing device for key regions in a video frame provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0020] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or more. Furthermore, the term "and / or" in the specification and claims is used to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0021] The embodiments of the present invention are applied in wireless communication scenarios. Figure 1 This is a schematic diagram of a wireless communication scenario provided in an embodiment of this application, with reference to... Figure 1 As shown, the on-site terminal equipment communicates wirelessly with the command center server, forming a wireless link. In a typical application scenario, such as earthquakes or fires requiring disaster relief command, the on-site terminal equipment usually needs to send video frames to the command center server so that the command center server can send command instructions to the on-site personnel based on visual information.

[0022] The aforementioned video frames include target critical areas and non-critical areas. Target critical areas are the regions in the video frame that contain important information, such as the face area in a video conference, or the area containing the football and player positions in a live sports broadcast. Non-critical areas are the regions in the video frame that are outside the target critical areas and usually do not contain important information, such as the sky in security monitoring, or the area containing distant trees in autonomous driving. Figure 2 This is a schematic diagram of video frame region division provided in an embodiment of this application. (Refer to...) Figure 2 As shown, the video frame is divided into a critical target region and a non-critical target region, with the critical target region enclosed by the target boundary curve.

[0023] The accuracy of the aforementioned target boundary curves directly impacts transmission efficiency. With limited bandwidth, excessive transmission resources are easily wasted in non-critical areas, leading to poor visual quality in the output image. This is particularly problematic for scenarios requiring remote command, as it can negatively affect decision-making. Therefore, accurately determining the target boundary curves is crucial. Conventional algorithms train models to identify specific object categories within video frames, outputting bounding boxes based on object type, thus identifying different target key regions. These key regions are then differentiated and encoded. However, conventional algorithms only achieve coarse-grained localization accuracy. Target key regions often contain numerous non-critical pixels, and the encoding bitrate for key regions is typically higher than that for non-critical regions. This results in larger video file sizes, lower transmission efficiency, and higher resource consumption.

[0024] To address the aforementioned technical problems, this application proposes a boundary processing method and related equipment for key regions in video frames. The method uses a coarse-grained key region extracted by existing conventional algorithms as the initial key region. Each pixel in the transition zone on both sides of this initial key region is evaluated to obtain a boundary attribution confidence score, indicating the probability that it belongs to the target boundary curve. Based on this boundary attribution confidence score and an energy functional model, the target boundary curve is found from the transition zone. Since the energy functional model describes the difference between the sum of squared curvatures of each pixel of any boundary curve and the boundary attribution confidence score, when this sum is minimized, the corresponding target boundary curve is the curve with the smallest possible sum of squared curvature, most likely belonging to the boundary. A smaller curvature ensures a smoother target boundary curve. Therefore, the target boundary curve in this embodiment is more accurate than the initial boundary curve.

[0025] Figure 3 This is a schematic diagram illustrating the relationship between an initial boundary curve, a target boundary curve, and a transition zone, provided in an embodiment of this application. Figure 3 As shown, before processing, the video frame contains an initial boundary curve and a transition band. The area enclosed by the initial boundary curve is the initial critical region, and the transition band is located on both sides of the initial boundary curve. After calculating the target boundary curve, as shown... Figure 3 The target boundary curve shown is located within the transition zone and is a different curve from the initial boundary curve. The area enclosed by the target boundary curve is the critical target region, while the video frame portions outside the area enclosed by the target boundary curve are non-critical regions. The shapes of the initial boundary curve and the target boundary curve can be arbitrary, including but not limited to the rectangle shown in the illustration.

[0026] Figure 4This is a flowchart illustrating the steps of a method for boundary processing of key regions in a video frame, as provided in an embodiment of the present invention. Figure 4 As shown, the method may include the following steps: S101. Obtain the initial boundary curve of at least one initial key region in the video frame, and obtain the transition band located on both sides of the initial boundary curve, wherein the transition band includes at least one first pixel.

[0027] S102. The gradient magnitude response of the first pixel in multiple directions is calculated by the edge detection algorithm, and the largest gradient magnitude response is selected as the local edge intensity of the first pixel.

[0028] S103. Calculate the contrast and entropy of the local texture block where the first pixel is located, and use the product of the contrast and entropy of the local texture block as the local texture complexity of the first pixel.

[0029] S104. The local edge intensity and local texture complexity of the first pixel are weighted and fused to obtain the boundary attribution confidence of the first pixel. The boundary attribution confidence is used to characterize the probability that the first pixel belongs to the boundary curve of the initial key region.

[0030] S105. Construct the energy functional model corresponding to any curve in the transition zone, and minimize the energy functional model to obtain the boundary curve corresponding to the minimum output of the energy functional model as the target boundary curve. The energy functional model is the sum of the squares of the curvature of each first pixel point on any curve minus the boundary attribution confidence of each first pixel point on any curve.

[0031] Specifically, video frames from a video file in a remote terminal device are first acquired. A lightweight object detection network is deployed on the remote terminal device. After normalization preprocessing, the video frames are input into the lightweight object detection network to extract multi-scale feature maps. A head detection algorithm is used to calculate the offset of a preset anchor box relative to the initial key target bounding box and the category confidence score at each spatial location of the multi-scale feature map. After suppressing and eliminating redundant candidate boxes with an overlap ratio exceeding a preset threshold, the coordinates of the bounding box with the highest confidence score are taken to directly generate a binary image with the same spatial resolution as the input frame. The pixel values ​​of the inner region enclosed by the bounding box are set to 1, and the pixel values ​​of the outer region are set to 0. The resulting binary image is the initial mask. The initial mask is marked with an initial boundary curve containing the target body and the background edge. A transition band covering the initial key region and extending to the outer background region by a certain pixel width is generated through the initial boundary curve. The transition band contains at least one first pixel.

[0032] The first pixel refers to all pixels contained within the transition zone and its edges.

[0033] For example, the lightweight object detection network pre-deployed on the terminal is the MobileNetV3-SSD network. This network performs forward inference on the currently acquired video frame, detects areas where people gather as initial key regions, and outputs a binary image as an initial mask. The boundary of the initial key region in the initial mask is the boundary of a rectangular box containing the areas where people gather, and a transition band of a certain pixel width is generated based on the boundary of the rectangular box.

[0034] For each pixel in the transition zone, i.e., the first pixel, the gradient magnitude response in multiple directions is calculated using an edge detection algorithm. This is achieved by convolving each first pixel with a set of Sobel operator templates with different directional characteristics. The gradient magnitude response measures the degree of grayscale change at each pixel's location in the image. A larger gradient magnitude response indicates that the pixel is located in a region of dramatic texture change, while a smaller gradient magnitude response indicates that the pixel is located in a flat region.

[0035] For example, the Sobel operator template includes four directional templates: a horizontal template, a vertical template, a 45-degree diagonal template, and a 135-degree diagonal template. For any first pixel within the transition band... Its local edge intensity is determined by the maximum value of the gradient magnitude response in four directions:

[0036] in, For local edge intensity, The horizontal gradient magnitude response, The gradient magnitude response is along the 45-degree diagonal direction. The vertical gradient magnitude response, The gradient magnitude response is along the 135-degree diagonal direction.

[0037] For each pixel in the transition zone, a local texture block of a certain size is set, and the frequency of occurrence of adjacent pixel pairs (grayscale value combinations) with specific directions and distances among all pixels in the block is counted to generate the corresponding gray-level co-occurrence matrix (GLCM). Based on the GLCM, the corresponding contrast and information entropy are calculated.

[0038] Contrast refers to the degree of brightness difference between the brightest area (white) and the darkest area (black) in an image. The greater the contrast, the more distinct the image boundaries. The calculation method is as follows: First, normalize the gray-level co-occurrence matrix. Then, traverse all pixels in the gray-level co-occurrence matrix to obtain the gray level (i,j) of each pixel, where i is the gray level of the current pixel and j is the gray level of the adjacent pixel. The square of the difference between i and j is multiplied by the probability P(i,j) of sampling the current pixel pair from the total number of pixel pairs to obtain the single contrast. Finally, the single contrasts of all pixels are summed to obtain the contrast corresponding to the current gray-level co-occurrence matrix.

[0039] Information entropy refers to the regularity of the distribution of image texture (pixel gray levels). The higher the information entropy, the more chaotic the visual presentation of the image. It is calculated as follows: Based on the normalized gray-level co-occurrence matrix, traverse all pixels in the gray-level co-occurrence matrix and determine the value of P(i,j): when its value is 0, the single information entropy value corresponding to that pixel is 0; when its value is greater than 0, the single information entropy value corresponding to that pixel is:

[0040] in, This represents the value of the entropy of a single piece of information.

[0041] Then, sum the individual information entropies of all pixels and take the negative value to obtain the information entropy corresponding to the current gray-level co-occurrence matrix.

[0042] Multiply the output contrast and information entropy, and use the product as the local texture complexity of the first pixel:

[0043] in, For local texture complexity, For contrast, reflecting the depth of texture grooves. Information entropy reflects the degree of texture non-uniformity.

[0044] The obtained local edge strength and local texture complexity are normalized to the interval [0, 1] to eliminate the influence of dimensional differences on subsequent calculations.

[0045] For example, for each pixel within the transition band, convolution operations are performed using Sobel operator templates in four directions: horizontal, vertical, 45-degree diagonal, and 135-degree diagonal, to obtain the gradient magnitudes in each direction. Taking a pixel with coordinates (640, 480) within the transition band as an example, its gradient magnitudes in the four directions are as follows:

[0046] The local edge intensity of the pixel is obtained by taking the maximum value:

[0047] After normalization to the interval [0,1]:

[0048] Simultaneously, a 5×5 pixel local texture block is extracted centered on this pixel, and the gray-level co-occurrence matrix is ​​calculated for this texture block, compressing the gray-level to 16 levels. The contrast attribute is then calculated.

[0049] Calculate the entropy attribute:

[0050] The local texture complexity is:

[0051] After normalization .

[0052] Next, the obtained local edge strength and local texture complexity are weighted and fused to obtain the boundary attribution confidence score of the first pixel. Local edge strength is positively correlated with boundary attribution confidence score, while local texture complexity is negatively correlated. The boundary attribution confidence score represents the probability that the first pixel belongs to the boundary curve of the initial key region, and its value ranges from [0, 1]. The closer the score is to 1, the higher the probability that the pixel belongs to the boundary curve of the initial key region. The weighting coefficients include two components: a first weight coefficient corresponding to the local edge strength and a second weight coefficient corresponding to the local texture complexity.

[0053] For each pixel p within the transition band, its boundary attribution confidence is calculated according to the following expression:

[0054] in, For local edge intensity, For local texture complexity, The first weighting coefficient corresponds to the local edge intensity. This is the second weighting coefficient corresponding to the local texture complexity. and The values ​​of are all real numbers between 0.5 and 2.0, and To maintain scale consistency in the fusion results, the numerator of the weighted fusion expression is used... This allows pixels with low texture complexity to make a higher contribution to fusion, thereby effectively suppressing the false response of texture-rich regions located inside key targets to real boundaries.

[0055] For example, local edge intensity is a weighting coefficient. Set to 1.3, local texture suppression is a weighted coefficient. Setting it to 0.7 satisfies:

[0056] Substitute the local edge intensity and local texture complexity of the pixel into the weighted fusion function

[0057] The calculated boundary assignment confidence score for this pixel is 0.55. This calculation is performed on all approximately 3200 pixels within the transition zone to obtain a confidence score distribution map.

[0058] Construct an energy functional model based on any curve in the transition zone. The value of the energy functional model corresponding to any curve formed by the first pixel is calculated by subtracting the corresponding boundary attribution confidence from the sum of squared curvatures of each first pixel. Define any curve as C, where the input data of the energy functional model includes the spatial coordinates of the first pixel within the transition zone and its corresponding boundary attribution confidence, and the output is the value of the energy functional model corresponding to the current curve. The expression for calculating the energy functional is:

[0059] in, This is the output value of the energy functional. For a data item, it is defined as the negative boundary attribution confidence value corresponding to each first pixel point traversed by any curve C, integrated along the line of any curve C, i.e.:

[0060] in, Let be the coordinates of the first pixel point on any curve C, with arc length parameter s as the independent variable. The boundary attribution confidence at the first pixel obtained from S3 is used to drive any curve C to move towards the region of high confidence scores.

[0061] Here, is the curvature regularization term, representing the sum of squared curvatures corresponding to the first pixel points along any curve C. is the regularization weight coefficient, taking a real number between 0.1 and 0.5. The curvature regularization term is defined as the line integral of the square of the curvature of any curve C along the curve, i.e.:

[0062] in Let be the curvature value of any curve C at the arc length parameter s. This curvature regularization term applies an energy penalty to large curvature bends of the curve to maintain the geometric continuity and topological smoothness of the boundary curve. Regularization weight coefficient. The value of determines the importance of the boundary smoothing constraint relative to the data-driven term. The larger the value, the smoother the boundary curve obtained by evolution. The smaller the value, the closer the curve is to the distribution details of the boundary attribution confidence.

[0063] In the output value of the energy functional model, when the output value is the minimum value, the boundary curve corresponding to this minimum value is the target boundary curve to be obtained.

[0064] For example, based on the boundary assignment confidence values ​​of each pixel output above, an energy functional model is constructed. The regularization weight coefficient is set to 0.25 to balance boundary accuracy and smoothness. The data term is approximated by numerical integration of the boundary assignment confidence on the initial boundary curve, with an initial value of approximately -0.48. The curvature regularization term is approximated by integration of the square of the curve curvature, with an initial value of approximately 0.12. Initial values ​​of the energy functional:

[0065] This yields the initial first prediction curve, and then the minimum value is solved based on the current energy functional model to obtain the target boundary curve.

[0066] In this embodiment of the invention, by obtaining the transition zone on both sides of the initial boundary curve, the local edge intensity of all first pixels within the transition zone is calculated based on the edge detection algorithm. This intensity is then weighted and fused with the local texture complexity calculated based on local texture blocks to obtain the boundary attribution confidence of all first pixels. By constructing an energy functional model of any curve within the transition zone and solving it using a variational level set evolution method, the minimum value of the energy functional model is obtained. The curve corresponding to the minimum value is output as the target boundary curve. This target boundary curve serves as the refined boundary curve, enabling refined extraction of key regions. The accuracy of the target key regions is high, avoiding the inclusion of some non-key pixels. Consequently, when encoding the target key regions at a higher bitrate in the subsequent process, encoding resources can be effectively saved, the amount of data after video frame encoding can be reduced, and transmission efficiency can be improved while saving transmission resources.

[0067] Regarding boundary processing methods for key regions in video frames, one related technique typically prioritizes different regions within a single video frame based on their content and other factors, then performs differentiated encoding based on these priorities to ensure the transmission of critical visual information within the available bandwidth. However, this approach only achieves coarse-grained boundary localization accuracy for key regions, often resulting in the inclusion of numerous non-critical background pixels within these regions. Furthermore, the encoding bitrate for key regions is typically higher than that for non-critical regions, leading to larger video file sizes, lower transmission efficiency, and higher resource consumption. This invention addresses this issue by evaluating the boundary attribution confidence of each pixel in the transition zone on both sides of the initial key region, combining local edge strength and local texture complexity. This evaluation indicates the probability that the pixel belongs to the target boundary curve. Based on this boundary attribution confidence and an energy functional model, a corrected target boundary curve is obtained from the transition zone. This improves the accuracy of the target boundary curve and prevents it from including non-critical pixels. When subsequent encoding of the key region at a higher bitrate, this effectively saves encoding resources, reduces the amount of data after video frame encoding, and thus improves transmission efficiency and conserves transmission resources.

[0068] Optionally, the local edge intensity and local texture complexity of the first pixel are weighted and fused to obtain the boundary attribution confidence of the first pixel, including: Based on the global noise of the video frame, a first weighting coefficient corresponding to the local edge intensity and a second weighting coefficient corresponding to the local texture complexity are determined. The first weighting coefficient is negatively correlated with the global noise, and the second weighting coefficient is negatively correlated with the first weighting coefficient. Based on the first weighting coefficient and the second weighting coefficient, the local edge intensity and the local texture complexity are weighted and fused to obtain the boundary attribution confidence of the first pixel.

[0069] Specifically, the first weighting coefficient corresponding to the aforementioned local edge intensity is determined based on the global noise of the detected video frame. The second weighting coefficient corresponding to the local texture complexity The value of increases with increasing global noise. The value decreases. The value of is adjusted to enhance the noise suppression effect of texture features; as global noise decreases, the value of is reduced. The value increases The value of is used to reinforce the dominant role of the edge gradient response. And based on the above, let and and always maintain .

[0070] Then, based on the obtained first and second weighting coefficients, the local edge strength and local texture complexity are weighted and fused to calculate the boundary attribution confidence score for each pixel within the transition zone:

[0071] The meanings of the parameters are the same as those mentioned above.

[0072] For example, the global noise of the current video frame is obtained. The calculated global noise variance is 32.5, which is lower than the preset noise threshold of 40. Therefore, the edge strength weighting coefficient is adjusted. Set to 1.3, texture suppression weight coefficient Setting it to 0.7 satisfies:

[0073] Substituting the local edge strength and local texture complexity of the pixel into the weighted fusion function, the boundary attribution confidence of the pixel is calculated to be 0.55.

[0074] In this embodiment of the invention, by weighted fusion of local edge intensity and local texture complexity, through 1 The T(p) term is used to suppress the boundary assignment confidence of high-texture pixels, so that pixels with high local edge strength and low local texture complexity can obtain higher boundary assignment confidence, thereby suppressing the interference of background noise and internal texture on the localization of the real boundary.

[0075] Optionally, the energy functional model is minimized to obtain the boundary curve corresponding to the minimum output of the energy functional model, which is used as the target boundary curve of the initial key region, including: Construct a level set function to represent any boundary curve within the transition zone; obtain the first variation of the level set function based on the energy functional model to get the evolution equation of the level set function, which indicates the rule of the level set corresponding to the level set function changing over time; take the initial boundary curve as the zero level set of the level set function; input the zero level set into the evolution equation of the level set function to obtain the evolved level set of the zero level set; obtain the number of different pixels in the evolved level set compared to the zero level set.

[0076] If the ratio of the number of different pixels to the total number of pixels in the video frame is less than a set threshold, the evolved level set is used as the target boundary curve; if the ratio of the number of different pixels to the total number of pixels in the video frame is greater than or equal to the set threshold, the zero level set is updated to the evolved level set, and the process of inputting the zero level set into the evolution equation of the level set function to obtain the evolved level set of the zero level set is returned.

[0077] Specifically, the energy functional model constructed above is minimized using a variational level set evolution algorithm. First, a set of level set functions is constructed, representing the function set of any boundary curve in the transition zone. Then, the first-order variation of the level set functions is obtained based on the energy functional model, yielding the evolution equation for the level set functions. This equation represents the rule governing the change of the level set corresponding to the level set function over time. Here, the level set function exists as an independent variable within the energy functional model, and the evolution equation is the derivative of the energy functional model with respect to this independent variable.

[0078] The minimum value is solved iteratively: the zero level set is input into the evolution equation to calculate the evolved level set. To determine if the evolved level set meets the minimum value criterion, the number of different pixels in the evolved level set and the input zero level set is calculated. The ratio of the number of different pixels to the total number of pixels in the video frame is used to determine if the minimum value is met. If the ratio is less than a set threshold, the condition is met, and the current evolved level set is taken as the target boundary curve. Otherwise, the zero level set is updated to the current evolved level set, and the updated zero level set is used as the input to the next evolution equation to calculate the next evolved level set.

[0079] The zero level set is the curve corresponding to the level set function being equal to 0, and the initial boundary curve is used as the initial zero level set.

[0080] Using the symbolic distance function , as the numerical representation of the level set function, where (x,y) are the spatial coordinates of the image pixels, and t is the evolution time parameter. Curve Implicitly represented as level set function The zero level set, i.e.

[0081] Based on the energy functional model, the first variation of the level set function is obtained, yielding the following partial differential evolution equation:

[0082] in, The spatial gradient of the level set function. For gradient magnitude, For divergence operators, The constant velocity coefficient is a real number between 0.2 and 1.0. The marginal stopping function depends on the boundary attribution confidence, which is constructed from the boundary attribution confidence and is defined as follows:

[0083] in, This represents the boundary attribution confidence of the pixel at coordinates (x, y) obtained from S3. This is a boundary stopping function. It takes a smaller value in regions with high confidence in boundary attribution to slow down the evolution speed and make the curve stop precisely, while it takes a larger value in regions with low confidence in boundary attribution to accelerate the evolution process.

[0084] The two terms added on the right side of the evolution equation are as follows: the first term is the product of the edge stopping function g(C(x,y)) with the curvature term and the gradient magnitude, which corresponds to the role of the curvature regularization term in the energy functional model in the evolution, in order to constrain the smoothness of the boundary and prevent the curve from bending excessively; the second term is the dot product of the gradient of the edge stopping function and the gradient of the level set function, which reflects the spatial modulation of the evolution velocity by the data term in the energy functional model.

[0085] Therefore, during the iteration process, the data terms and curvature regularization terms in the energy functional model are integrated into the numerical solution process of each iteration through two corresponding terms of the evolution equation. Starting from the initial boundary curve, the confidence score and edge stopping function are repeatedly calculated from the current curve position, the level set function is updated, and the zero level set is extracted until the ratio of the number of different pixels to the total number of pixels in the video frame between two consecutive iterations is lower than the set threshold. The resulting evolved level set is the target boundary curve corresponding to the minimum value of the energy functional model.

[0086] For example, after obtaining the energy functional model, the variational level set evolution algorithm is used to solve for the minimum value of the energy functional. Initialize the symbolic distance function. This ensures that the zero level set corresponds to the initial boundary curve. A constant velocity coefficient is set to 0.5. In the first iteration, the edge stopping function is calculated for pixels with a confidence score of 0.55:

[0087] The level set function is updated based on the partial differential evolution equation, and the curve moves about 1.2 pixels towards the region with a high confidence score.

[0088] After 12 iterations, the number of pixels with changed zero-level set positions between consecutive iterations was 84, accounting for 2.6% of the total 3200 pixels to be processed, which had not yet reached the 0.5% convergence threshold. Continuing the iteration to the 23rd iteration, the number of changed pixels decreased to 14, accounting for 0.44%, satisfying the convergence condition, and the iteration stopped. The zero-level set at this point was extracted as a refined target boundary curve. This curve closely matches the true contour of the densely populated area, and compared to the initial rectangular bounding box boundary, the edge localization accuracy was improved by approximately 63%.

[0089] In this embodiment of the invention, an energy functional model constructed using boundary attribution confidence as the data-driven term and combined with curvature regularization term drives the curve to converge to the pixel position with the highest confidence score within the transition zone through a variational level set evolution algorithm. This refines the coarse-grained initial boundary into a target boundary curve with sub-pixel-level positioning accuracy and geometric continuity and smoothness, thus maintaining the robustness of boundary positioning in complex field environments (such as emergency sites) and providing high-quality and highly reliable visual information support for remote command and decision-making.

[0090] Optionally, the method further includes: The video frame is divided according to the target boundary curve to obtain the target critical region and non-critical region of the video frame; the target critical region and non-critical region of the video frame are encoded and transmitted, the encoding bit rate of the target critical region is higher than that of the non-critical region, and the encoding precision of the target critical region is higher than that of the non-critical region.

[0091] Specifically, based on the target boundary curve obtained above, the video frame is divided into two regions: a target critical region and a non-critical region. The first pixel located inside the target boundary curve is assigned to the target critical region, and the first pixel located outside the target boundary curve is assigned to the non-critical region. Based on the video frame after region division, the two regions undergo partitioned encoding: for the target critical region, the encoding bitrate and encoding precision are both higher than those for the non-critical region.

[0092] In this context, bitrate refers to the number of bits per video frame per unit time; a higher bitrate results in clearer video frames. Encoding precision refers to the number of bits required to store one pixel; higher precision allows for more layers of video frames to be recorded, resulting in smoother color transitions.

[0093] In this embodiment of the invention, the target boundary curve is used to divide the video frame into key and non-key regions, which directly drives the differentiated configuration of the quantization parameters and bitrate allocation of the video encoder. This ensures that the key regions of the target receive priority protection that matches the boundary accuracy in terms of encoding quality and transmission scheduling, thereby maximizing the transmission reliability of key visual information in some bandwidth-constrained environments.

[0094] Optionally, the video frame is divided according to the target boundary curve to obtain the target critical region and non-critical region of the video frame, including: For each first pixel that is not on the target boundary curve, determine the ray from the second pixel to any boundary of the video frame, and determine whether the second pixel belongs to the target key region corresponding to the target boundary curve based on the number of intersections between the ray and the target boundary curve. If the number of intersections is odd, the first pixel belongs to the target key region of the target boundary curve; if the number of intersections is even, the first pixel does not belong to the target key region of the target boundary curve. For each first pixel that is on the target boundary curve, determine whether the first pixel belongs to the target key region corresponding to the target boundary curve based on the first area of ​​the third pixel inside the target boundary curve and the second area of ​​the third pixel outside the target boundary curve. Pixels that do not belong to any target key region are classified into non-key regions.

[0095] Specifically, the division of key and non-key regions of a video frame is mainly based on analyzing the spatial relationship between each pixel in the video frame and the target boundary curve. For the second pixel that is not on the target boundary curve, the ray casting method is used: starting from the second pixel, a ray is drawn to any boundary in the video frame, and the number of intersections between the ray and the target boundary curve is calculated. If the number of intersections is odd, the pixel is determined to be inside the target boundary curve, that is, in the target critical region; if the number of intersections is even, the pixel is located outside the target boundary curve, that is, in the non-critical region.

[0096] For the third pixel on the target boundary curve, a sub-pixel-level boundary allocation strategy is adopted: calculate the area ratio of the two sub-regions divided by the target boundary curve. For the first area located inside the target boundary curve, if the proportion of the first area to the total pixel area is greater than the set threshold proportion, then the pixel is located in the target critical region; otherwise, it is located in the non-critical region.

[0097] In this context, the second pixel and the third pixel both belong to the first pixel. The second pixel refers to the first pixel that is not on the target boundary curve, while the third pixel refers to the first pixel that is on the target boundary curve.

[0098] For example, for a pixel not on the target boundary curve, taking the pixel at coordinates (638, 482) as an example, a horizontal ray is drawn to the right from this point. The number of intersections between this ray and the video frame boundary is calculated to be 1, which is an odd number. Therefore, the pixel is determined to be inside the boundary curve and within the target key region, and its corresponding position in the final mask is assigned a value of 1. For a pixel on the target boundary curve, taking the pixel at coordinates (645, 478) as an example, the area of ​​the sub-region inside the target boundary curve is calculated to be 62% of the total pixel area, which is greater than the set threshold ratio of 50%. Therefore, this pixel is determined to belong to the target key region. In the final generated mask, the area of ​​the continuous region with a value of 1 is 23.7% of the total frame area; this region is the target key region.

[0099] In this embodiment of the invention, by using ray casting and a sub-pixel-level boundary allocation strategy, each pixel in the video frame is strictly divided into a target critical region or a non-critical region. This ensures that the target critical region receives priority protection in terms of both encoding quality and transmission scheduling, matching the boundary accuracy. In some bandwidth-constrained environments, this maximizes the transmission reliability of critical visual information.

[0100] Optionally, the target critical regions and non-critical regions of the video frame are encoded and transmitted, including: The target critical area is encoded to obtain first encoded data, and the non-critical area is encoded to obtain second encoded data. Based on the first encoded data, the second encoded data, and the priority of the first encoded data and the priority of the second encoded data, a data packet corresponding to the video frame is generated. The priority of the first encoded data and the priority of the second encoded data are set in the header information of the data packet as supplementary enhancement information, and the priority of the first encoded data is higher than the priority of the second encoded data. The first encoded data and the second encoded data are transmitted according to the priority in the data packet.

[0101] Specifically, based on the video frames after the regions are divided, the two regions are divided into two regions for encoding: different encoding rates and different encoding precisions are used for the target critical regions and non-critical regions respectively.

[0102] For the target critical region, its corresponding coding precision is obtained by applying a negative offset to the frame-level fundamental quantization parameters of the video encoder, so that the actual quantization parameters of the target critical region are:

[0103] in, The actual quantitative parameters for the target key area. These are frame-level basic quantization parameters, representing the basic compression level set by the encoder when processing each frame of video. This is the negative offset, and its value is an integer between -6 and -1. The smaller the quantization parameter value, the higher the encoding precision.

[0104] For non-critical regions, the corresponding coding precision is obtained by applying a positive offset to the frame-level fundamental quantization parameters of the video encoder, so that the actual quantization parameters of the non-critical regions are:

[0105] in, For the actual quantitative parameters of non-critical areas, This is the positive offset, and its value is an integer between +2 and +8.

[0106] The bitrate corresponding to the critical target region is negatively correlated with the bitrate corresponding to the non-critical region, and the two satisfy the following relationship:

[0107] in, The bitrate corresponding to the target key region. The bitrate for non-critical regions. The total target bitrate at the frame level is calculated by the video encoder based on the current channel state parameters, and and The ratio is determined by multiplying the area of ​​the target critical region to the area of ​​the video frame by a priority enhancement factor greater than 1. The value of the priority enhancement factor ranges from 1.5 to 4.0, so as to further tilt the bitrate resources toward the critical region under the constraint of limited total bitrate.

[0108] During the encoding process, the video encoder determines the region type of each encoding block based on the final mask image of the acquired regions. When the number of first pixels within the target critical region of the encoding block is greater than a set threshold, the encoding block is encoded according to the bitrate and encoding precision corresponding to the target critical region; otherwise, it is encoded as a non-critical region. Encoding the target critical region yields the first encoded data, and encoding the non-critical region yields the second encoded data.

[0109] During encoding, region priority identification information is added to two regions: the priority of the first encoded data and the priority of the second encoded data. The priority of the first encoded data corresponds to identifying critical regions, while the priority of the second encoded data corresponds to identifying non-critical regions. The priority of the first encoded data is higher than that of the second encoded data. This region priority identification information is embedded in the header of the video frame as a Supplemental Enhancement Information (SEI) message. This SEI message contains a region priority mapping table synchronized with the video frame, recording the priority of the encoded data corresponding to each coded block.

[0110] The data packets corresponding to the video frames generated by the first encoded data and its priority, and the data packets corresponding to the video frames generated by the second encoded data and its priority, are sent into the transmission channel for transmission, and are transmitted sequentially according to the priority in the data packets.

[0111] For example, a video encoder obtains frame-level basic quantization parameters. The value is 28. Based on the target critical area's area ratio of 23.7% and the priority enhancement factor of 2.0, the offset of the target critical area's quantization parameters is determined. -4 represents the quantization parameter offset for non-critical areas. The value is +5. Therefore, the actual quantization parameter for the target critical area is:

[0112] The actual quantification parameters for non-critical areas are:

[0113] The current frame-level total target bitrate is calculated to be 2.5 Mbps based on the channel conditions. The bitrate corresponding to the target critical region is determined to be 1.2 Mbps, and the bitrate corresponding to the non-critical region is determined to be 1.3 Mbps. The ratio of the bitrate corresponding to the target critical region to the bitrate corresponding to the non-critical region is approximately 0.92, which is significantly higher than the natural ratio of 0.31 corresponding to the target critical region's area share of 23.7%.

[0114] During encoding, the block size is 64×64 pixels. For a certain block located within a target critical region, 76% of its pixels are within the target critical region, exceeding the set threshold of 50%. Therefore, this block is encoded using a quantization parameter of QP=24 and at a bitrate of 1.2Mbps. After encoding, an SEI message is inserted into the bitstream to record the region priority mapping relationship of each block in the frame.

[0115] In this embodiment of the invention, by mapping the refined target boundary curve to the differentiated configuration of encoding quantization parameters and transmission scheduling, and combining the regional priority mapping relationship, the key target region can obtain priority protection matching the boundary accuracy in both encoding quality and transmission scheduling. Thus, in some bandwidth-constrained environments, the transmission reliability of key visual information can be maximized.

[0116] Optionally, the method further includes: Determine the channel communication quality of the transmitted data packets; adjust the coding rate of non-critical areas based on the channel communication quality, as the coding rate of non-critical areas is positively correlated with the channel communication quality.

[0117] Specifically, when data packets are sent into the transmission channel, the channel communication quality can be acquired in real time. Based on the area priority identification information, while allocating transmission resources to data packets corresponding to the target critical area, channel state parameters are simultaneously detected. The channel communication quality is then determined based on these parameters, and the coding rate for non-critical areas is adjusted accordingly. When the channel communication quality is detected to be below the transmission quality threshold, a code rate backoff mechanism for non-critical areas is triggered, meaning the sender actively reduces the code rate for non-critical areas. The worse the channel communication quality, the lower the coding rate for non-critical areas.

[0118] The channel state parameters include various parameters such as signal-to-noise ratio (SNR), reference signal receiving power (RSRP), and Doppler shift. The judgment of channel communication quality is based on setting thresholds according to different channel state parameter values, depending on the actual situation. The bit rate backoff mechanism includes various strategies: reducing resolution, reducing frame rate, and increasing quantization parameters. The choice of backoff strategy is adjusted according to the actual situation.

[0119] For example, the terminal monitors the channel status of the current 4G network transmission link in real time, obtaining a channel bandwidth of 1.8Mbps, a signal strength of -92dBm, a transmission delay of 68ms, and a bit error rate of 0.03%. The encoded video stream is then sent into the transmission channel, and based on the area priority identifier information in the SEI message, data packets corresponding to the target critical area are sent first.

[0120] During transmission, channel conditions fluctuated, signal strength dropped to -105dBm, and transmission latency increased to 142ms, falling below the preset transmission quality threshold. The transmission scheduling module immediately triggered a bitrate backoff mechanism, reducing the bitrate allocation for non-critical areas from 1.3Mbps to 0.7Mbps, while maintaining the bitrate for the target critical area at 1.2Mbps. After the adjustment, video data from the target critical area accounted for 63% of the total transmission bitrate of 1.9Mbps, ensuring clarity and smoothness in images of densely populated areas. The subjective quality score (MOS) for the video from the target critical area received by the command center was 4.2, meeting the requirements for command and decision-making.

[0121] In this embodiment of the invention, the coding bitrate of non-critical areas is adjusted in real time by acquiring the channel communication quality of the transmitted data packets. When the channel communication quality is lower than the transmission quality threshold, a bitrate fallback mechanism for non-critical areas is triggered, while the target critical areas for priority transmission remain unchanged, ensuring that the encoded video data of the target critical areas is guaranteed to be transmitted under limited network conditions.

[0122] Optionally, the transition zone located on both sides of the initial boundary curve is obtained, including: Morphological dilation is performed on the initial boundary curve to obtain a dilated mask; erosion is performed on the initial boundary curve to obtain an eroded mask; XOR operation is performed on the dilated mask and the eroded mask according to the pixel position to obtain the transition zone located on both sides of the initial boundary curve.

[0123] Specifically, morphological dilation is performed on the initial boundary curve using a circular structure with a radius of r pixels. The initial boundary curve is scanned and dilated pixel by pixel, where r is an integer between 3 and 8. The output boundary width is a dilated mask image with an increase of r pixels based on the initial boundary curve.

[0124] Then, an erosion operation is performed on the initial boundary curve. A circular structure with the same radius of r pixels is used to perform pixel-by-pixel scanning and erosion operations on the initial boundary curve. The output boundary width is an erosion mask image with a width reduced by r pixels based on the initial boundary curve.

[0125] Perform a pixel-by-pixel XOR operation on the dilation mask and the erosion mask. The difference between the dilation mask and the erosion mask is exactly the annular region that covers the initial boundary curve, which shrinks inward by r pixels and expands outward by r pixels. This forms a transition zone with a range of r pixels on each side, and its radial width is 2r pixels.

[0126] The value of radius r is determined by the resolution of the video frame in the remote terminal device. When the resolution is greater than the resolution threshold, the value is 7 to 8, and when the resolution is less than or equal to the resolution threshold, the value is 3 to 5, so as to adaptively match the actual width of the transition band under different resolutions.

[0127] For example, for the boundary of a rectangular frame containing a densely populated area, the expansion radius is determined based on the current video frame resolution of 1920×1080. The value of is 4. A circular structuring element with a radius of 4 pixels is used to perform a morphological dilation operation on the boundary of the rectangular frame, generating a ring-shaped region with a width of 2r = 8 pixels. This region is the transition zone, which contains all possible positions of the boundary of the real people gathering area.

[0128] In this embodiment of the invention, morphological dilation and erosion are performed on the initial boundary curve, and combined with XOR operation, a transition zone containing the possible positions of the target boundary curve is constructed. This allows for the subsequent structural representation of each pixel within the transition zone, thereby effectively distinguishing the target boundary curve from background interference textures under various conditions such as sudden changes in lighting and smoke occlusion.

[0129] Reference Figure 5 The diagram illustrates a block diagram of a boundary processing device for key regions in a video frame according to an embodiment of the present invention. Figure 5 As shown, the device 200 may specifically include: The boundary acquisition module 201 is used to acquire the initial boundary curve of at least one initial key region in the video frame, and to acquire the transition band located on both sides of the initial boundary curve, wherein the transition band includes at least one first pixel.

[0130] The first calculation module 202 is used to calculate the gradient magnitude response of the first pixel in multiple directions through an edge detection algorithm, and select the largest gradient magnitude response as the local edge intensity of the first pixel.

[0131] The second calculation module 203 is used to calculate the contrast and information entropy of the local texture block where the first pixel is located, and to use the product of the contrast and information entropy of the local texture block as the local texture complexity of the first pixel.

[0132] The fusion module 204 is used to perform weighted fusion of the local edge intensity and local texture complexity of the first pixel to obtain the boundary attribution confidence of the first pixel. The boundary attribution confidence is used to characterize the probability that the first pixel belongs to the boundary curve of the initial key region.

[0133] The output module 205 is used to construct the energy functional model corresponding to any curve in the transition zone, and to minimize the energy functional model to obtain the boundary curve corresponding to the minimum output value of the energy functional model as the target boundary curve. The energy functional model is the sum of the squares of the curvature of each first pixel point on any curve minus the boundary attribution confidence of each first pixel point on any curve.

[0134] Optionally, the fusion module 204 is also used for: Based on the global noise of the video frame, a first weighting coefficient corresponding to the local edge intensity and a second weighting coefficient corresponding to the local texture complexity are determined. The first weighting coefficient is negatively correlated with the global noise, and the second weighting coefficient is negatively correlated with the first weighting coefficient. Based on the first weighting coefficient and the second weighting coefficient, the local edge intensity and the local texture complexity are weighted and fused to obtain the boundary attribution confidence of the first pixel.

[0135] Optionally, the output module 205 is also used for: Construct a level set function to represent any boundary curve within the transition band; obtain the first variation of the level set function based on the energy functional model to obtain the evolution equation of the level set function, which indicates the rule of the level set corresponding to the level set function changing over time; take the initial boundary curve as the zero level set of the level set function; input the zero level set into the evolution equation of the level set function to obtain the evolved level set of the zero level set; obtain the number of different pixels in the evolved level set compared to the zero level set; if the ratio of the number of different pixels to the total number of pixels in the video frame is less than a set threshold, then take the evolved level set as the target boundary curve; if the ratio of the number of different pixels to the total number of pixels in the video frame is greater than or equal to the set threshold, then update the zero level set to the evolved level set, and return to the step of inputting the zero level set into the evolution equation of the level set function to obtain the evolved level set of the zero level set.

[0136] Optionally, the device further includes a partitioning module 206, used for: The video frame is divided according to the target boundary curve to obtain the target critical region and non-critical region of the video frame; the target critical region and non-critical region of the video frame are encoded and transmitted, the encoding bit rate of the target critical region is higher than that of the non-critical region, and the encoding precision of the target critical region is higher than that of the non-critical region.

[0137] Optionally, the partition module 206 is also used for: For each first pixel that is not on the target boundary curve, determine the ray from the second pixel to any boundary of the video frame, and determine whether the second pixel belongs to the target key region corresponding to the target boundary curve based on the number of intersections between the ray and the target boundary curve. If the number of intersections is odd, the first pixel belongs to the target key region of the target boundary curve; if the number of intersections is even, the first pixel does not belong to the target key region of the target boundary curve. For each first pixel that is on the target boundary curve, determine whether the first pixel belongs to the target key region corresponding to the target boundary curve based on the first area of ​​the third pixel inside the target boundary curve and the second area of ​​the third pixel outside the target boundary curve. Pixels that do not belong to any target key region are classified into non-key regions.

[0138] Optionally, the device further includes an encoding module 207, used for: The target critical area is encoded to obtain first encoded data, and the non-critical area is encoded to obtain second encoded data. Based on the first encoded data, the second encoded data, and the priority of the first encoded data and the priority of the second encoded data, a data packet corresponding to the video frame is generated. The priority of the first encoded data and the priority of the second encoded data are set in the header information of the data packet as supplementary enhancement information, and the priority of the first encoded data is higher than the priority of the second encoded data. The first encoded data and the second encoded data are transmitted according to the priority in the data packet.

[0139] Optionally, the encoding module 207 is also used for: Determine the channel communication quality of the transmitted data packets; adjust the coding rate of non-critical areas based on the channel communication quality, as the coding rate of non-critical areas is positively correlated with the channel communication quality.

[0140] Optionally, the boundary acquisition module 201 is also used for: Morphological dilation is performed on the initial boundary curve to obtain a dilated mask; erosion is performed on the initial boundary curve to obtain an eroded mask; XOR operation is performed on the dilated mask and the eroded mask according to the pixel position to obtain the transition zone located on both sides of the initial boundary curve.

[0141] In summary, the boundary processing device for key regions in video frames provided in this embodiment of the invention evaluates the boundary attribution confidence of each pixel in the transition zone on both sides of the initial key region by combining local edge strength and local texture complexity, thereby indicating the probability that it belongs to the target boundary curve. Based on this boundary attribution confidence and an energy functional model, the corrected target boundary curve is obtained from the transition zone, which helps to improve the accuracy of the target boundary curve and avoid including some non-key pixels. When encoding the target key region at a higher bitrate in the subsequent process, encoding resources can be effectively saved, the amount of data after video frame encoding can be reduced, and transmission efficiency can be improved while saving transmission resources.

[0142] Reference Figure 6 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. For example... Figure 6 As shown, the electronic device includes: a processor 302, a memory 301, a communication interface 304, and a communication bus 303.

[0143] The processor, memory, and communication interface communicate with each other via a communication bus; the memory stores at least one executable instruction, which causes the processor to execute the code optimization method described in the preceding embodiments. The executable instructions can form a program.

[0144] This invention provides a machine-readable medium storing instructions that, when executed by one or more processors, enable the processors to perform the code optimization method described in the foregoing embodiments. The machine-readable medium may be one or more processors.

[0145] This invention also discloses a computer program product, including a computer program for implementing the above-described domestic sewage treatment control method.

[0146] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0147] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0149] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0150] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing terminal device to operate in a predictive manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0152] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0153] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0154] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0155] The present invention has been described in detail above as a method, an apparatus, an electronic device, and a readable medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for boundary processing of key regions in a video frame, characterized in that, include: Obtain the initial boundary curve of at least one initial key region in the video frame, and obtain the transition bands located on both sides of the initial boundary curve, wherein the transition bands include at least one first pixel. The gradient magnitude response of the first pixel in multiple directions is calculated by the edge detection algorithm, and the largest gradient magnitude response is selected as the local edge intensity of the first pixel. Calculate the contrast of the local texture block where the first pixel is located and the information entropy of the local texture block, and use the product of the contrast and information entropy of the local texture block as the local texture complexity of the first pixel. The local edge intensity and local texture complexity of the first pixel are weighted and fused to obtain the boundary attribution confidence of the first pixel. The boundary attribution confidence is used to characterize the probability that the first pixel belongs to the boundary curve of the initial key region. Construct an energy functional model corresponding to any curve in the transition zone, and minimize the energy functional model to obtain the boundary curve corresponding to the minimum output value of the energy functional model as the target boundary curve. The energy functional model is the sum of squares of curvature of each first pixel on any curve minus the boundary assignment confidence of each first pixel on any curve.

2. The method according to claim 1, characterized in that, The local edge intensity and local texture complexity of the first pixel are weighted and fused to obtain the boundary attribution confidence of the first pixel, including: Based on the global noise of the video frame, a first weighting coefficient corresponding to the local edge intensity and a second weighting coefficient corresponding to the local texture complexity are determined. The first weighting coefficient is negatively correlated with the global noise, and the second weighting coefficient is negatively correlated with the first weighting coefficient. Based on the first weighting coefficient and the second weighting coefficient, the local edge intensity and the local texture complexity are weighted and fused to obtain the boundary attribution confidence of the first pixel.

3. The method according to claim 1, characterized in that, Minimize the energy functional model to obtain the boundary curve corresponding to the minimum output value of the energy functional model, which is then used as the target boundary curve of the initial critical region, including: Construct a level set function, which is used to represent any boundary curve within the transition zone; The first variation of the level set function is obtained based on the energy functional model to obtain the evolution equation of the level set function. The evolution equation is used to indicate the rule of the level set corresponding to the level set function changing with time. The initial boundary curve is used as the zero level set of the level set function; The zero level set is input into the evolution equation of the level set function to obtain the evolved level set of the zero level set; Obtain the number of different pixels in the evolved level set compared to the zero level set; If the ratio of the number of different pixels to the total number of pixels in the video frame is less than a set threshold, then the evolved horizontal set is used as the target boundary curve. If the ratio of the number of different pixels to the total number of pixels in the video frame is greater than or equal to a set threshold, then the zero level set is updated to the evolved level set, and the step of inputting the zero level set into the evolution equation of the level set function to obtain the evolved level set of the zero level set is returned.

4. The method according to any one of claims 1 to 3, characterized in that, Also includes: The video frame is divided according to the target boundary curve to obtain the target key region and non-key region of the video frame; The target critical region and the non-critical region of the video frame are encoded and transmitted, wherein the encoding bitrate of the target critical region is higher than that of the non-critical region, and the encoding precision of the target critical region is higher than that of the non-critical region.

5. The method according to claim 4, characterized in that, The video frame is divided according to the target boundary curve to obtain the target key region and non-key region of the video frame, including: For each of the first pixels that is not on the target boundary curve, determine the ray from the second pixel to any boundary of the video frame, and determine whether the second pixel belongs to the target key region corresponding to the target boundary curve based on the number of intersections between the ray and the target boundary curve. If the number of intersections is odd, the first pixel belongs to the target key region of the target boundary curve; if the number of intersections is even, the first pixel does not belong to the target key region of the target boundary curve. For each of the first pixels, the third pixel on the target boundary curve is determined to be either the first pixel or the target key region corresponding to the target boundary curve, based on the first area of ​​the third pixel inside the target boundary curve and the second area of ​​the third pixel outside the target boundary curve. Pixels that do not belong to any of the target critical regions are assigned to the non-critical regions.

6. The method according to claim 5, characterized in that, Encoding and transmitting the target critical region and the non-critical region of the video frame includes: The target critical region is encoded to obtain first encoded data, and the non-critical region is encoded to obtain second encoded data; Based on the first encoded data, the second encoded data, and the priorities of the first encoded data and the second encoded data, a data packet corresponding to the video frame is generated. The priorities of the first encoded data and the second encoded data are set in the header information of the data packet as supplementary enhancement information, and the priority of the first encoded data is higher than the priority of the second encoded data. The first encoded data and the second encoded data are transmitted according to the priority in the data packet.

7. The method according to claim 6, characterized in that, Also includes: Determine the channel communication quality for transmitting the data packets; The coding rate of the non-critical region is adjusted according to the channel communication quality, and the coding rate of the non-critical region is positively correlated with the channel communication quality.

8. The method according to claim 1, characterized in that, Obtaining the transition zone located on both sides of the initial boundary curve includes: Morphological dilation is performed on the initial boundary curve to obtain a dilated mask image; The initial boundary curve is eroded to obtain an erosion mask image; The dilation mask and the erosion mask are XORed according to their pixel positions to obtain the transition zone located on both sides of the initial boundary curve.

9. A boundary processing device for key regions in a video frame, characterized in that, include: A boundary acquisition module is used to acquire the initial boundary curve of at least one initial key region in a video frame, and to acquire the transition bands located on both sides of the initial boundary curve, wherein the transition bands include at least one first pixel. The first calculation module is used to calculate the gradient magnitude response of the first pixel in multiple directions through an edge detection algorithm, and select the largest gradient magnitude response as the local edge intensity of the first pixel. The second calculation module is used to calculate the contrast of the local texture block where the first pixel is located and the information entropy of the local texture block, and to use the product of the contrast and information entropy of the local texture block as the local texture complexity of the first pixel. The fusion module is used to perform weighted fusion of the local edge intensity and the local texture complexity of the first pixel to obtain the boundary attribution confidence of the first pixel. The boundary attribution confidence is used to characterize the probability that the first pixel belongs to the boundary curve of the initial key region. The output module is used to construct the energy functional model corresponding to any curve in the transition zone, and to minimize the energy functional model to obtain the boundary curve corresponding to the minimum output value of the energy functional model as the target boundary curve. The energy functional model is the sum of the squares of the curvature of each first pixel point on any curve minus the boundary assignment confidence of each first pixel point on any curve.

10. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store executable instructions that cause the processor to perform the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1 to 8.

Images