A method of expanding a foreground region

Abstract

The application provides a method for expanding a foreground region, which draws a foreground contour map on the basis of a foreground binary map, frames a minimum rectangular frame of the contour map, compares w points farthest and nearest to the rectangular frame in a width w direction, draws an ellipse with the intersection point coordinates of the farthest and nearest points as the center points of the ellipse, sets all pixel points from the inside of the ellipse to the foreground contour part as foreground points, and thus obtains an expanded foreground binary map. Since the farthest and nearest points do not necessarily contain the whole height h direction of the rectangular frame, many pairs of the farthest and nearest points are obtained, and each pair of the farthest and nearest points draws an ellipse. On the basis of frame difference, the foreground region is expanded, so that the foreground region which cannot be displayed due to the same foreground and background colors is normally displayed.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method for expanding the foreground region. Background Technology

[0002] Foreground detection in image processing is an important research direction in the field of computer vision. Computer vision is a science that studies how to enable computers to "understand" images and videos, thereby comprehending and processing visual information. Its goal is to simulate the function of the human visual system through computers, enabling them to extract useful information from images or multidimensional data and make corresponding judgments and decisions. Computer vision involves the acquisition, processing, analysis, and understanding of digital images, and the extraction of high-dimensional data for further processing. It combines knowledge from multiple disciplines, including computer science, artificial intelligence, signal processing, and neuroscience.

[0003] Foreground detection is one of the fundamental tasks in computer vision. It focuses on identifying regions of interest in images or video sequences while simultaneously distinguishing them from the background. Its goal is to accurately separate foreground objects (such as pedestrians and vehicles) from the background in complex image or video scenes. This technology plays a crucial role in subsequent advanced vision tasks, such as object tracking, behavior analysis, and scene understanding. It is widely used in video surveillance, motion analysis, autonomous driving, and many other fields.

[0004] Techniques used in foreground detection include: inter-frame difference method, background modeling, optical flow method, etc.

[0005] Inter-frame difference is a method for obtaining the contour of a moving target by performing a difference operation on two consecutive frames of a video image sequence. When a target moves in a scene, there will be a significant difference between two adjacent frames. By subtracting the two frames, the absolute value of the difference in pixel values ​​at corresponding positions in the images is obtained. It is then determined whether this difference exceeds a certain threshold, and the motion characteristics of the object in the image sequence can be analyzed.

[0006] Therefore, the shortcomings of existing technology are:

[0007] When using the inter-frame difference method to obtain a foreground binary map, if the pixel values ​​of moving objects in the image are similar to or equal to the pixel values ​​of the background, an accurate foreground map cannot be obtained. This will result in the moving object appearing partially missing in the image. However, methods such as dilation cannot achieve the desired effect with a small number of dilation operations.

[0008] Furthermore, the terminology used in this art includes:

[0009] Foreground detection: Foreground detection refers to the process of distinguishing between dynamic or highly salient parts (foreground) and relatively static or unchanging parts (background) in a video or image sequence using certain algorithms and techniques.

[0010] Binary image: This refers to an image where each pixel has only two possible values. In other words, the grayscale value of each pixel in the image is either 0 or 255, representing black and white respectively. Summary of the Invention

[0011] To address the aforementioned issues, the purpose of this application is to augment the foreground binary image obtained through the inter-frame difference method, thereby supplementing the portion of the foreground contour that disappears due to the foreground and background having the same color. This enhances the contour of moving objects in the subsequently processed image.

[0012] Specifically, the present invention provides a method for expanding the foreground region. The method involves drawing a foreground outline map based on the obtained foreground binary map, outlining the smallest rectangle of the outline map, comparing the farthest and nearest w points of the foreground outline to the rectangle in the width w direction, drawing an ellipse with the intersection of the farthest and nearest points as the center point of the ellipse, and setting all pixels inside the ellipse and extending to the foreground outline as foreground points, thereby obtaining the expanded foreground binary map. Since the two nearest and farthest points may not necessarily include the entire height h direction of the rectangle, many pairs of nearest and farthest points will be obtained, and an ellipse will be drawn for each pair of nearest and farthest points.

[0013] The method further includes the following steps:

[0014] S1. First, obtain a binary image of the foreground using the inter-frame difference method. This image is a binary image with a black background and a white foreground. Enclose the foreground part with a rectangle and obtain the (x,y) coordinates of the upper left corner of the rectangle and the width w and height h of the rectangle.

[0015] S2, Assuming there is a rectangular frame containing the outline of an object, calculate the data for the left half of the outline and proceed to step S3; calculate the data for the right half of the outline and proceed to step S4; the calculation of the left and right half of the outline has no order and can be performed simultaneously.

[0016] S3, calculates the data for the left half of the contour, including:

[0017] S3.1 Calculate the coordinates of the width w of the left half of the outline within the height h interval of the rectangle, finding the closest and farthest points from the rectangle. Store these coordinates in xy_coor_left. Retrieve the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_left, which are (height of the farthest point, farthest point) and (height of the closest point, closest point). Since we are determining the farthest and closest distances from the rectangle's width, the height coordinates determine the farthest and farthest points. After finding the widest point, draw an ellipse based on its height. Determine if the ellipse can encompass the left half of the foreground outline within the range h_far to h_near. If not, compare the coordinates (x, y) of the point not included by the ellipse with the existing (h_far, w_far) and (h_near, w_near) coordinates, set the (x, y) coordinates as the farthest or nearest point, and update the coordinate information in xy_coor_left.

[0018] S3.2, compare h_far and h_near to obtain h_min and h_max. If h_min is not equal to the box_h_min of the rectangle, compare box_h_min and h_min again to calculate the new coordinates of the nearest and farthest points in the w direction from the rectangle. Store these coordinates in xy_coor_left. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_left. Draw an ellipse based on these coordinates. Determine whether the ellipse can contain the left half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point that the ellipse cannot contain with the existing (h_far, w_far) and (h_near, w_near). Set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_left.

[0019] S3.3, recursively call step S3.2 until h_min is the same as box_h_min of the rectangle;

[0020] S3.4, such as Figure XAs shown, compare h_far and h_near to obtain h_min and h_max. If h_max is not equal to box_h_max of the rectangular box, then compare and calculate the new coordinates of the nearest and farthest points in the w direction between h_max and box_h_max again, save the coordinates in xy_coor_left, take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_left, draw an ellipse according to these coordinates, and judge whether the ellipse can enclose the left half of the foreground contour within the interval from h_far to h_near. If not, compare the coordinate points (x, y) that the ellipse cannot enclose with the existing (h_far, w_far) and (h_near, w_near), and set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_left;

[0021] S3.5, recursively call step S3.4 until h_max and box_h_max are the same;

[0022] S3.6, loop: The loop code is implemented as follows: for(int j = 0; j < xy_coor_left.size(); j++), because multiple pairs of coordinates are saved in xy_coor_left, loop through xy_coor_left here, take out the coordinate pairs (h_far, w_far) and (h_near, w_near) of the nearest and farthest points stored in each pair of coordinates in xy_coor_left. With (h_near, w_far) as the center and w_far - w_near as the length of the w axis, judge the coordinate point information. If the condition is satisfied: h_near < h_far && w_near < w_far, draw an ellipse with h_far - h_near as the length of the h axis; if the condition is satisfied: h_near > h_far && w_near < w_far, draw an ellipse with h_near - h_far as the length of the h axis; within the interval from h_near to h_far, set all the pixel points from the inside of the ellipse to the foreground part of the contour as foreground points;

[0023] S3.7, the left half contour part ends;

[0024] S4, right contour:

[0025] S4.1 Calculate the coordinates of the right half of the outline in the h-interval of the rectangle, specifically the coordinates in the w-direction that are closest to and farthest from the rectangle. Store these coordinates in xy_coor_right. Retrieve the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_right. Draw an ellipse based on these coordinates. There are two coordinates here. Use the intersection of these two coordinates as the center. Then, use the distances from these two points to the center as the major and minor axes of the ellipse, respectively. Use the intersection of the purple lines as the center, and the two purple lines as the major and minor axes, respectively. Determine whether the ellipse can contain the right half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) that the ellipse cannot contain with the existing (h_far, w_far) and (h_near, w_near). Set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_left.

[0026] S4.2, compare h_far and h_near to obtain h_min and h_max. If h_min is not equal to the box_h_min of the rectangle, compare box_h_min and h_min again to calculate the new coordinates of the nearest and farthest points in the w direction from the rectangle. Store the coordinates in xy_coor_right. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_right. Draw an ellipse based on the coordinates. Determine whether the ellipse can contain the right half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point that the ellipse cannot contain with the existing (h_far, w_far) and (h_near, w_near). Set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_right.

[0027] S4.3, recursively call step S4.2 until h_min is the same as box_h_min of the rectangle;

[0028] S4.4, compare h_far and h_near to obtain h_min and h_max. If h_max is not equal to the box_h_max of the rectangle, compare h_max and box_h_max again to calculate the new coordinates of the nearest and farthest points in the w direction from the rectangle. Store the coordinates in xy_coor_right. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_right. Draw an ellipse based on the coordinates. Determine whether the ellipse can contain the right half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point that the ellipse cannot contain with the existing (h_far, w_far) and (h_near, w_near). Set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_right.

[0029] S4.5, recursively call step S4.4 until h_max and box_h_max are the same;

[0030] S4.6 Iterate through the nearest and farthest point coordinate pairs (h_far, w_far) and (h_near, w_near) in xy_coor_right. Using (h_near, w_far) as the center and w_far - w_near as the length of the w-axis, check the coordinate information of the points. If the condition is met: h_near<h_far&&w_near> w_far, draw an ellipse with h_far-h_near as the length of the h-axis; if the condition is met: h_near>h_far&&w_near>w_far, draw an ellipse with h_near-h_far as the length of the h-axis; within the interval from h_near to h_far, set all pixels inside the ellipse to the foreground outline as the foreground point.

[0031] S4.7, the right half of the outline ends.

[0032] In step S1, the findContours function in OpenCV is used to draw the outline of the foreground part, and then the boundingRect function in OpenCV is used to draw a bounding box that tightly surrounds the outline.

[0033] In step S3.1:

[0034] The step of drawing an ellipse based on the coordinates includes:

[0035] To draw an ellipse, you need to determine the center point, major axis, and minor axis. Let's assume that after determining the nearest point on the left side of the width, let's call it w_left_near and the farthest point on the left side of the width, let's call it w_left_far1. Draw straight lines from these two points into the frame. The point where the two lines intersect is the center point of the ellipse. Then, the straight-line distances from the two points to the center point are used as the major axis and minor axis, respectively. This way, you can draw an ellipse.

[0036] The determination that the interval is from h_far to h_near includes:

[0037] We currently have a contour plot and an ellipse plot, both filled with numbers. If the ellipse can completely contain the contour, then assuming we check the value of the (x,y) point, the value of the ellipse is not 0, and the value of the contour is not 0 either. However, if the value of the (x,y) point in the ellipse is 0, but the value of the (x,y) point in the contour is not 0, then it means that the current ellipse cannot contain the contour point.

[0038] The conditional statement is: if(ellipse_value==0&&contour_value!=0), where ellipse_value is the value of the (x,y) point of the ellipse and contour_value is the value of the (x,y) point of the contour.

[0039] In step S3.2:

[0040] The comparison between h_far and h_near is performed as follows: if (h_far > h_near) { h_min = h_near; h_max = h_far;} else { h_min = h_far; h_max = h_near;};

[0041] Because `far` and `near` in `h_far` and `h_near` are used to match the width, representing the `h` values ​​corresponding to the farthest and closest points in the width, respectively, and do not represent the height, the height needs to be determined here. The specific meaning is as follows:

[0042] Suppose we initially obtain (h_left_near, w_left_near) and (h_left_far1, w_left_far1). Since w_left_far1 is farther from the width of the rectangle than w_left_far2, the farthest point obtained initially is w_left_far1. After obtaining these two points, we skip the step of determining whether the ellipse can contain the outline. At this step, for comparison, if h_near is smaller than h_far, then we consider h_near to be h_min. If h_near is still some distance from the h-side of the rectangle, then we obtain another set of coordinates (h_left_near, w_left_near) and (h_left_far2, w_left_far2) between h_near and the h-side of the rectangle.

[0043] The step of drawing an ellipse based on the coordinates includes:

[0044] Here we have two coordinates. We use the intersection of these two coordinates as the center of the circle. Then, the distances from these two points to the center point are used as the major and minor axes of the ellipse, respectively. The intersection of the yellow lines is the center of the circle, and the two yellow lines are used as the major and minor axes, respectively.

[0045] In step S3.4:

[0046] The process involves comparing h_far and h_near, with the smaller value being h_min and the larger value being h_max, to obtain new coordinates (h_left_near, w_left_near) and (h_left_far2, w_left_far2), which are then saved into xy_coor_left.

[0047] The method involves drawing an ellipse based on these coordinates. There are two coordinates, and the intersection of these two coordinates is used as the center of the circle. The distances from these two points to the center point are used as the major and minor axes of the ellipse, respectively. The intersection of the yellow lines is used as the center of the circle, and the two yellow lines are used as the major and minor axes, respectively.

[0048] Therefore, this application has the following advantages:

[0049] Based on inter-frame difference, the foreground area is expanded so that the foreground area that cannot be displayed because the foreground and background colors are the same can be displayed normally. Attached Figure Description

[0050] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0051] Figure 1 This is a flowchart illustrating the method.

[0052] Figure 2 This is a flowchart illustrating step S3 in this method.

[0053] Figure 3 This is a flowchart illustrating step S4 in this method.

[0054] Figure 4 This is a schematic diagram of a specific embodiment of the method.

[0055] Figure 5 This is a schematic diagram of the code implementation of step S1 of this method.

[0056] Figure 6 This is a schematic diagram of the code implementation of step S3.1 of this method.

[0057] Figure 7 This is a schematic diagram of the code implementation of step S3.2 of this method.

[0058] Figure 8 This is a schematic diagram of the code implementation of step S3.4 of this method.

[0059] Figure 9 This is a schematic diagram of the code implementation of step S3.6 of this method.

[0060] Figure 10 This is a schematic diagram of the code implementation of step S4.1 of this method.

[0061] Figure 11 This is a schematic diagram of the code implementation for step S4.2 of this method.

[0062] Figure 12 This is a schematic diagram of the code implementation of step S4.4 of this method.

[0063] Figure 13 This is a schematic diagram of the code implementation of step S4.6 of this method. Detailed Implementation

[0064] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0065] This method primarily involves drawing a foreground outline based on the obtained foreground binary image. The smallest rectangle within the outline is defined. Along the width (w) direction, the furthest and nearest (w points) of the foreground outline are compared to the rectangle (since the furthest and nearest points may not encompass the entire height (h) of the rectangle, many pairs of nearest and farthest points are obtained, and each pair is represented by an ellipse). The center point of the ellipse is determined by the intersection of the furthest and nearest points. Pixels within the ellipse extending to the foreground outline are then designated as foreground pixels. This process yields the expanded foreground binary image.

[0066] Specifically, such as Figure 1 As shown, the steps of this method include the following:

[0067] S1. First, obtain a binary image of the foreground using the inter-frame differencing method. This image is a binary image with a black background and a white foreground. Use the `findContours` function in OpenCV to draw the outline of the foreground. Then, use the `boundingRect` function in OpenCV to draw a bounding box that tightly surrounds the outline. Obtain the (x, y) coordinates of the top-left corner of the bounding box and the width and height of the bounding box. Figure 5 The image shows the code implementation of step S1. S2: Assuming there's a rectangular frame containing the outline of an item, calculate the data for the left half of the outline, then proceed to step S3; as shown... Figure 4 As shown, there is an object outline in the box. When calculating, the left side can be calculated first; if the data of the right half of the outline is calculated, then proceed to step S4; the calculation of the left half of the outline and the right half of the outline does not have a specific order and can be performed simultaneously.

[0068] like Figure 2 The diagram illustrates the process of calculating the left half contour data in step S3, including: S3.1, calculating the coordinates of the width w direction of the left half contour within the height h interval of the rectangle, which is closest to and farthest from the rectangle, and storing the coordinates in xy_coor_left. Retrieving the last set of coordinates (h_far, w_far) and (h_near, w_near) from xy_coor_left, i.e., (height of the farthest width point, farthest width point) and (height of the closest width point, closest width point), since it determines the farthest and closest distances from the rectangle's width, the height coordinates are the height corresponding to the point where the farthest width point is determined. An ellipse is drawn based on these coordinates. Drawing the ellipse requires determining the center point, major axis, and minor axis. Assuming that after determining... After denoteing the nearest point on the left width as w_left_near and the farthest point on the left width as w_left_far1, draw straight lines from these two points into the box. The point where the two lines intersect is the center point of the ellipse. The straight-line distances from the two points to the center point are then used as the major and minor axes, respectively, thus drawing an ellipse. We now have a contour map and an ellipse map, both filled with numbers. If the ellipse completely contains the contour, then, assuming the value of the (x,y) point is checked, the value of both the ellipse and the contour is not 0. However, if the value of the (x,y) point in the ellipse is 0, but the value of the (x,y) point in the contour is not 0, then the current ellipse cannot contain the contour point.

[0069] The conditional statement is: if(ellipse_value==0&&contour_value!=0), where ellipse_value is the value of the (x,y) point of the ellipse graph, and contour_value is the value of the (x,y) point of the contour graph.

[0070] If the ellipse cannot encompass the left half of the foreground outline, then the coordinates (x, y) of the point not included by the ellipse are compared with the existing (h_far, w_far) and (h_near, w_near). The (x, y) coordinates are then set as the farthest or nearest point, and the coordinate information in xy_coor_left is updated. Figure 6 The image shown is a schematic diagram of the code implementation for step S3.1.

[0071] S3.2, compare h_far and h_near to obtain h_min and h_max. The comparison of h_far and h_near is performed as follows: if (h_far > h_near) { h_min = h_near; h_max = h_far;} else { h_min = h_far; h_max = h_near;};

[0072] Because `far` and `near` in `h_far` and `h_near` are used to match the width, representing the `h` corresponding to the farthest point and the closest point in the width, respectively, and do not represent the height, the height needs to be determined here. The specific meaning is as follows: Figure 4 ,

[0073] Assuming we initially obtain (h_left_near, w_left_near) and (h_left_far1, w_left_far1), since w_left_far1 is farther from the width of the rectangle than w_left_far2, the farthest point obtained initially is w_left_far1. After obtaining these two points, the step of determining whether the ellipse contains the outline is skipped. At this stage, assuming h_near is smaller than h_far, we consider h_near to be h_min. Assuming h_near still has a distance from the h-side of the rectangle, we obtain another set of coordinates (h_left_near, w_left_far1) between h_near and the h-side of the rectangle. (h_left_near) and (h_left_far2, w_left_far2); if h_min is not equal to the box_h_min of the rectangle, then compare box_h_min and h_min again to calculate the new coordinates of the closest and farthest w-direction from the rectangle, and save these coordinates in xy_coor_left. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_left, and draw an ellipse based on these coordinates. Here there are two coordinates, and the intersection of these two coordinate points is used as the center of the circle. Then the distances from these two points to the center point are used as the major axis and minor axis of the ellipse, respectively. Figure 4 The intersection of the yellow lines is the center of the circle, and the two yellow lines are the major and minor axes, respectively. Determine if the ellipse within the interval h_far to h_near can encompass the left half of the foreground contour. If not, compare the coordinates (x, y) of the point not included by the ellipse with the existing (h_far, w_far) and (h_near, w_near) coordinates, and set the (x, y) coordinates as the farthest or nearest point, updating the coordinate information in xy_coor_left. Figure 7 The image shown is a schematic diagram of the code implementation for step S3.2.

[0074] S3.3, recursively call step S3.2 until h_min is the same as box_h_min of the rectangle;

[0075] S3.4, compare h_far and h_near to obtain h_min and h_max. That is, compare h_far and h_near again. The smaller one is h_min and the larger one is h_max. This is the same as the meaning in step S4. After obtaining the new coordinates (h_left_near, w_left_near) and (h_left_far2, w_left_far2), save them to xy_coor_left.

[0076] If h_max and box_h_max of the rectangular box are not equal, then calculate the new coordinates of the nearest and farthest points in the w direction between h_max and box_h_max again. Save the coordinates in xy_coor_left. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_left. Draw an ellipse based on these coordinates. Here there are two coordinates. Use the intersection point of these two coordinate points as the center of the circle, and then the distances from these two points to the center point are used as the major axis and minor axis of the ellipse respectively. As Figure 4 shown, the intersection point of the yellow lines is the center of the circle, and the two yellow lines are used as the major axis and minor axis respectively; judge whether the ellipse can enclose the left half of the foreground contour in the interval from h_far to h_near. If not, compare the coordinate points (x, y) that the ellipse cannot enclose with the existing (h_far, w_far) and (h_near, w_near), and set this (x, y) as the farthest point coordinate or the nearest point coordinate to update the coordinate information in xy_coor_left; As Figure 8 shown, it is a schematic diagram of the code implementation of this step S3.4.

[0077] S3.5, Recursively call step S3.4 until h_max and box_h_max are the same;

[0078] S3.6, Loop: The loop code implementation is as follows: for(int j = 0; j < xy_coor_left.size(); j++), because multiple pairs of coordinates are saved in xy_coor_left, loop through xy_coor_left here, take out the coordinate pairs (h_far, w_far) and (h_near, w_near) of the nearest and farthest points stored in each pair of coordinates in xy_coor_left. Use (h_near, w_far) as the center of the circle and w_far - w_near as the length of the w axis. Judge the coordinate point information. If the condition is satisfied: h_near < h_far && w_near < w_far, draw an ellipse with h_far - h_near as the length of the h axis; if the condition is satisfied: h_near > h_far && w_near < w_far, draw an ellipse with h_near - h_far as the length of the h axis; In the interval from h_near to h_far, set all the pixel points inside the ellipse to foreground points for the foreground contour part; As Figure 9 shown, it is a schematic diagram of the code implementation of this step S3.6.

[0079] S3.7, The left half contour part ends;

[0080] As Figure 3The diagram shows the calculation process for the right-hand contour in step S4, including:

[0081] S4.1, within the h-interval of the rectangle, calculate the coordinates of the right half of the outline in the w-direction, indicating the closest and farthest points from the rectangle. Store these coordinates in xy_coor_right. Retrieve the last set of coordinates (h_far, w_far) and (h_near, w_near) from xy_coor_right. Draw an ellipse based on these coordinates. Since there are two coordinates, use the intersection of these two points as the center. Then, use the distances from these two points to the center point as the major and minor axes of the ellipse, respectively. Figure 4 The intersection of the purple lines is the center of the circle, and the two purple lines are the major and minor axes, respectively. It is determined whether the ellipse within the interval from h_far to h_near can encompass the right half of the foreground outline. If not, the coordinates (x, y) of the point not included by the ellipse are compared with the existing (h_far, w_far) and (h_near, w_near), and the (x, y) is set as the farthest or nearest point coordinates, updating the coordinate information in xy_coor_left. Figure 10 The image shown is a code diagram for step S4.1.

[0082] S4.2, compare h_far and h_near to obtain h_min and h_max. If h_min is not equal to the bounding box's box_h_min, then compare box_h_min and h_min again to calculate the new coordinates of the nearest and farthest points in the w direction from the bounding box, and save the coordinates in xy_coor_right. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_right, and draw an ellipse based on these coordinates. Determine whether the ellipse can contain the right half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point not contained by the ellipse with the existing (h_far, w_far) and (h_near, w_near), and set the (x, y) as the farthest or nearest point coordinates, and update the coordinate information in xy_coor_right; Figure 11 The image shown is a code diagram for step S4.2. S4.3 recursively calls step S4.2 until h_min and the rectangle's box_h_min are the same;

[0083] S4.4, compare h_far and h_near to obtain h_min and h_max. If h_max is not equal to the rectangle's box_h_max, then compare h_max and box_h_max again to calculate the new coordinates of the nearest and farthest points in the w direction from the rectangle, and save the coordinates in xy_coor_right. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_right, and draw an ellipse based on these coordinates. Determine whether the ellipse can contain the right half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point not contained by the ellipse with the existing (h_far, w_far) and (h_near, w_near), and set the (x, y) as the farthest or nearest point coordinates, and update the coordinate information in xy_coor_right; Figure 12 The image shown is a code diagram for step S4.4. S4.5 recursively calls step S4.4 until h_max and box_h_max are the same;

[0084] S4.6 Iterate through the nearest and farthest point coordinate pairs (h_far, w_far) and (h_near, w_near) in xy_coor_right. Using (h_near, w_far) as the center and w_far - w_near as the length of the w-axis, check the coordinate information of the points. If the condition is met: h_near<h_far&&w_near> `w_far` draws an ellipse with `h_far - h_near` as the length of the h-axis; if the condition `h_near > h_far && w_near > w_far` is met, draw an ellipse with `h_near - h_far` as the length of the h-axis; within the interval from `h_near` to `h_far`, set all pixels inside the ellipse up to the foreground outline as foreground pixels; for example... Figure 13 The image shown is a code diagram for step S4.6.

[0085] S4.7, the right half of the outline ends.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for expanding a foreground region, characterized in that, The method, based on the obtained foreground binary image, draws a foreground outline image, outlines the smallest rectangle of the outline image, compares the farthest and nearest w points of the foreground outline to the rectangle in the width w direction, draws an ellipse with the intersection of the farthest and nearest points as the center point of the ellipse, and sets all pixels inside the ellipse to the foreground outline as foreground points, thus obtaining an expanded foreground binary image. Since the two nearest and farthest points may not necessarily include the entire height h direction of the rectangle, many pairs of nearest and farthest points will be obtained, and an ellipse will be drawn for each pair of nearest and farthest points.

2. The method for expanding the foreground region according to claim 1, characterized in that, The method further Includes the following steps: S1. First, obtain a binary image of the foreground using the inter-frame difference method. This image is a binary image with a black background and a white foreground. Enclose the foreground part with a rectangle and obtain the (x,y) coordinates of the upper left corner of the rectangle and the width w and height h of the rectangle. S2, Assuming there is a rectangular frame containing the outline of an object, calculate the data for the left half of the outline and proceed to step S3; calculate the data for the right half of the outline and proceed to step S4; the calculation of the left and right half of the outline has no order and can be performed simultaneously. S3, calculates the data for the left half of the contour, including: S3.1 Calculate the coordinates of the width w of the left half of the outline within the height h interval of the rectangle, finding the closest and farthest points from the rectangle. Store these coordinates in xy_coor_left. Retrieve the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_left, which are (height of the farthest point, farthest point) and (height of the closest point, closest point). Since we are determining the farthest and closest distances from the rectangle's width, the height coordinates determine the farthest and farthest points. After finding the widest point, draw an ellipse based on its height. Determine if the ellipse can encompass the left half of the foreground outline within the range h_far to h_near. If not, compare the coordinates (x, y) of the point not included by the ellipse with the existing (h_far, w_far) and (h_near, w_near) coordinates, set the (x, y) coordinates as the farthest or nearest point, and update the coordinate information in xy_coor_left. S3.2, compare h_far and h_near to obtain h_min and h_max. If h_min is not equal to the box_h_min of the rectangle, compare box_h_min and h_min again to calculate the new coordinates of the nearest and farthest points in the w direction from the rectangle. Store these coordinates in xy_coor_left. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_left. Draw an ellipse based on these coordinates. Determine whether the ellipse can contain the left half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point that the ellipse cannot contain with the existing (h_far, w_far) and (h_near, w_near). Set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_left. S3.3, recursively call step S3.2 until h_min is the same as box_h_min of the rectangle; S3.4, compare h_far and h_near to obtain h_min and h_max. If h_max is not equal to the box_h_max of the rectangle, compare h_max and box_h_max again to calculate the new coordinates of the nearest and farthest points in the w direction from the rectangle. Store the coordinates in xy_coor_left. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_left. Draw an ellipse based on the coordinates. Determine whether the ellipse can contain the left half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point that the ellipse cannot contain with the existing (h_far, w_far) and (h_near, w_near). Set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_left. S3.5, recursively call step S3.4 until h_max and box_h_max are the same; S3.6, Loop: The loop code is implemented as follows: for(int j = 0; j < xy_coor_left.size(); j++). Since xy_coor_left stores multiple pairs of coordinates, loop through xy_coor_left here to retrieve the coordinate pairs (h_far, w_far) and (h_near, w_near) of the nearest and farthest points stored in each pair of coordinates in xy_coor_left. With (h_near, w_far) as the center and w_far - w_near as the length of the w-axis, judge the coordinate point information. If the conditions are met: h_near < h_far && w_near < w_far, draw an ellipse with h_far - h_near as the length of the h-axis; if the conditions are met: h_near > h_far && w_near < w_far, draw an ellipse with h_near - h_far as the length of the h-axis; within the interval from h_near to h_far, set all pixel points from the inside of the ellipse to the foreground contour part as foreground points. S3.7, The left half contour part ends; S4, Right contour: S4.1, Calculate the nearest and farthest w-direction coordinates of the right half contour within the h interval of the rectangular frame, save the coordinates in xy_coor_right, retrieve the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_right, and draw an ellipse based on these coordinates. Here there are two coordinates, use the intersection point of these two coordinate points as the center, and then the distances from these two points to the center point are used as the major axis and minor axis of the ellipse respectively. Use the intersection point of the purple lines as the center and the two purple lines as the major axis and minor axis respectively. Judge whether the ellipse can enclose the right half of the foreground contour within the interval from h_far to h_near. If not, compare the coordinate points (x, y) that the ellipse cannot enclose with the existing (h_far, w_far) and (h_near, w_near), and set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_left. S4.2, compare h_far and h_near to obtain h_min and h_max. If h_min is not equal to the box_h_min of the rectangle, compare box_h_min and h_min again to calculate the new coordinates of the nearest and farthest points in the w direction from the rectangle. Store the coordinates in xy_coor_right. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_right. Draw an ellipse based on the coordinates. Determine whether the ellipse can contain the right half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point that the ellipse cannot contain with the existing (h_far, w_far) and (h_near, w_near). Set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_right. S4.3, recursively call step S4.2 until h_min is the same as box_h_min of the rectangle; S4.4, compare h_far and h_near to obtain h_min and h_max. If h_max is not equal to the box_h_max of the rectangle, compare h_max and box_h_max again to calculate the new coordinates of the nearest and farthest points in the w direction from the rectangle. Store the coordinates in xy_coor_right. Take out the last set of coordinates (h_far, w_far) and (h_near, w_near) in xy_coor_right. Draw an ellipse based on the coordinates. Determine whether the ellipse can contain the right half of the foreground outline within the interval from h_far to h_near. If not, compare the coordinates (x, y) of the point that the ellipse cannot contain with the existing (h_far, w_far) and (h_near, w_near). Set the (x, y) as the farthest point coordinate or the nearest point coordinate, and update the coordinate information in xy_coor_right. S4.5, recursively call step S4.4 until h_max and box_h_max are the same; S4.6 Iterate through the nearest and farthest point coordinate pairs (h_far, w_far) and (h_near, w_near) in xy_coor_right. Using (h_near, w_far) as the center and w_far - w_near as the length of the w-axis, check the coordinate information of the points. If the condition is met: h_near<h_far&&w_near> w_far, draw an ellipse with h_far-h_near as the length of the h-axis; if the condition is met: h_near>h_far&&w_near>w_far, draw an ellipse with h_near-h_far as the length of the h-axis; within the interval from h_near to h_far, set all pixels inside the ellipse to the foreground outline as the foreground point. S4.7, the right half of the outline ends.

3. The method for expanding the foreground region according to claim 2, characterized in that, In step S1, the findContours function in OpenCV is used to draw the outline of the foreground part, and then the boundingRect function in OpenCV is used to draw a bounding box that tightly surrounds the outline.

4. The method for expanding the foreground region according to claim 2, characterized in that, In step S3.1: The step of drawing an ellipse based on the coordinates includes: To draw an ellipse, you need to determine the center point, major axis, and minor axis. Let's assume that after determining the nearest point on the left side of the width, let's call it w_left_near and the farthest point on the left side of the width, let's call it w_left_far1. Draw straight lines from these two points into the frame. The point where the two lines intersect is the center point of the ellipse. Then, the straight-line distances from the two points to the center point are used as the major axis and minor axis, respectively. This way, you can draw an ellipse. The determination that the interval is from h_far to h_near includes: We currently have a contour plot and an ellipse plot, both filled with numbers. If the ellipse can completely contain the contour, then assuming we check the value of the (x,y) point, the value of the ellipse is not 0, and the value of the contour is not 0 either. However, if the value of the (x,y) point in the ellipse is 0, but the value of the (x,y) point in the contour is not 0, then it means that the current ellipse cannot contain the contour point. The conditional statement is: if(ellipse_value==0&&contour_value!=0), where ellipse_value is the value of the (x,y) point of the ellipse and contour_value is the value of the (x,y) point of the contour.

5. The method for expanding the foreground region according to claim 2, characterized in that, In step S3.2: The comparison between h_far and h_near is performed as follows: if (h_far > h_near) { h_min = h_near; h_max = h_far;} else { h_min = h_far; h_max = h_near;}; Because `far` and `near` in `h_far` and `h_near` are used to match the width, representing the `h` values ​​corresponding to the farthest and closest points in the width, respectively, and do not represent the height, the height needs to be determined here. The specific meaning is as follows: Suppose we initially obtain (h_left_near, w_left_near) and (h_left_far1, w_left_far1). Since w_left_far1 is farther from the width of the rectangle than w_left_far2, the farthest point obtained initially is w_left_far1. After obtaining these two points, we skip the step of determining whether the ellipse can contain the outline. At this step, for comparison, if h_near is smaller than h_far, then we consider h_near to be h_min. If h_near is still some distance from the h-side of the rectangle, then we obtain another set of coordinates (h_left_near, w_left_near) and (h_left_far2, w_left_far2) between h_near and the h-side of the rectangle. The step of drawing an ellipse based on the coordinates includes: Here we have two coordinates. We use the intersection of these two coordinates as the center of the circle. Then, the distances from these two points to the center point are used as the major and minor axes of the ellipse, respectively. The intersection of the yellow lines is the center of the circle, and the two yellow lines are used as the major and minor axes, respectively.

6. The method for expanding the foreground region according to claim 2, characterized in that, In step S3.4: The process involves comparing h_far and h_near, with the smaller value being h_min and the larger value being h_max, to obtain new coordinates (h_left_near, w_left_near) and (h_left_far2, w_left_far2), which are then saved into xy_coor_left. The method involves drawing an ellipse based on these coordinates. There are two coordinates, and the intersection of these two coordinates is used as the center of the circle. The distances from these two points to the center point are used as the major and minor axes of the ellipse, respectively. The intersection of the yellow lines is used as the center of the circle, and the two yellow lines are used as the major and minor axes, respectively.

Images