Video static region identification method and device

Abstract

The invention discloses a video static region recognition method and device, which are used for improving the video static region recognition effectiveness and enabling a static region extraction algorithm to have the static state recognition capability of video total elements while ensuring the effectiveness. The method comprises the following steps: acquiring a to-be-processed video frame sequence; performing full segmentation on each video frame in the video frame sequence through a pre-trained full segmentation model to obtain a multi-target mask image corresponding to the video frame, the multi-target mask image representing an area where each target is located in the corresponding video frame; constructing a multi-target matrix corresponding to the video frame based on the multi-target mask image, wherein any slice of the multi-target matrix in a preset dimension represents an area where one target is located in one video frame; performing target matching on the multi-target matrix corresponding to the plurality of continuous video frames based on the video frame sequence to obtain a target association relationship corresponding to the plurality of continuous video frames; and determining an area where the static target is located in the plurality of continuous video frames according to the target association relationship.

Description

Technical Field

[0001] This application relates to video data processing methods, and more particularly to a method and apparatus for identifying static regions in a video. Background Technology

[0002] In video static target recognition applications, specific third-party engines are often relied upon to identify specific targets and extract static targets. For example, if a face recognition engine is used, it can only identify whether a face is static. If a target recognition engine is used for trees, flowers, houses, or other targets, it can only identify whether the corresponding specified target is static. In real-world applications, static areas are often complex and diverse, making them difficult to pre-define, which leads to inaccurate and incomplete static target recognition.

[0003] This kind of static recognition method with pre-defined targets can often only be applied to specific scenarios, such as surveillance cameras in fixed locations, to identify static objects such as roads and streetlights. Its application scenarios are limited and it lacks wide applicability.

[0004] How to improve the effectiveness of static region recognition in videos is the technical problem that this application aims to solve. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for identifying static regions in videos, so as to improve the effectiveness of identifying static regions in videos.

[0006] Firstly, a method for identifying static regions in a video is provided, including: Obtain the video frame sequence to be processed; By performing full segmentation on each video frame in the video frame sequence using a pre-trained full segmentation model, a multi-target mask image corresponding to the video frame is obtained. The multi-target mask image represents the region where each target is located in the corresponding video frame. Based on the multi-target mask image, a multi-target matrix corresponding to the video frame is constructed. Any slice of the multi-target matrix in a preset dimension represents the region where a target is located in a video frame. Based on the video frame sequence, target matching is performed on the multi-target matrix corresponding to multiple consecutive video frames to obtain the target association relationship corresponding to multiple consecutive video frames; Based on the target association relationship, the location of the static target is determined in multiple consecutive video frames.

[0007] Secondly, a video static region recognition device is provided, comprising: The acquisition module acquires the video frame sequence to be processed; The segmentation module performs full segmentation on each video frame in the video frame sequence using a pre-trained full segmentation model to obtain a multi-target mask image corresponding to the video frame. The multi-target mask image represents the region where each target is located in the corresponding video frame. The construction module constructs a multi-target matrix corresponding to the video frame based on the multi-target mask image. Any slice of the multi-target matrix in a preset dimension represents the region where a target is located in a video frame. The matching module performs target matching on the multi-target matrix corresponding to multiple consecutive video frames based on the video frame sequence to obtain the target association relationship corresponding to multiple consecutive video frames; The determination module determines the location of a static target in multiple consecutive video frames based on the target association relationship.

[0008] Thirdly, an electronic device is provided, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method of the first aspect.

[0009] Fourthly, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the steps of the method of the first aspect.

[0010] Fifthly, a computer program product is provided, comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of the method of the first aspect.

[0011] In this embodiment, firstly, a sequence of video frames to be processed is acquired. Then, a pre-trained full segmentation model is used to perform full segmentation on each video frame in the video frame sequence, obtaining a multi-target mask image corresponding to each video frame. The multi-target mask image represents the region where each target is located in the corresponding video frame. In this scheme, full segmentation of the video frames is performed using a full segmentation model, eliminating the need for pre-specified prompts during the segmentation step, and enabling comprehensive analysis and segmentation of the entire video frame. Next, a multi-target matrix corresponding to the video frame is constructed based on the multi-target mask image. Any slice of the multi-target matrix in a preset dimension represents the region where a target is located in a video frame. By constructing a multi-target matrix, the region where each target is located in the video frame can be represented by a multi-dimensional matrix, facilitating target matching and association in subsequent steps. Subsequently, target matching is performed on the multi-target matrices corresponding to multiple consecutive video frames based on the video frame sequence, obtaining the target association relationship between multiple consecutive video frames. The target association relationship obtained through matching can represent the correspondence between targets in consecutive video frames, thereby indicating the location of the same target in multiple consecutive video frames. Finally, the location of the static target is determined in multiple consecutive video frames based on the target association relationship. Based on the above target association relationship, it can be determined whether the same target changes in multiple consecutive video frames, thereby efficiently and accurately determining whether the target is a static target. The solution provided by this application embodiment enables the static region extraction algorithm to have the ability to recognize the static state of all video elements while ensuring effectiveness, thus breaking through the limitations of specific scenarios and achieving broader domain adaptation. This solution can achieve static region recognition for various targets in video frames without pre-specifying the categories of things to be recognized, and is especially suitable for video frames with changing scene content. By performing comparison on the target location region in consecutive video frames, the effectiveness of static region recognition can be improved. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1a This is one of the flowcharts illustrating a video static region recognition method according to an embodiment of this application; Figure 1b This is a schematic diagram of a video frame sequence illustrating a video static region identification method according to an embodiment of this application; Figure 1c This is a schematic diagram of a single video frame of a video static region identification method according to an embodiment of this application; Figure 1d This is a schematic diagram of a multi-target mask image of a video static region recognition method according to an embodiment of this application; Figure 1e This is a schematic diagram illustrating the full segmentation execution of a video static region recognition method according to an embodiment of this application; Figure 2a This is a second schematic flowchart of an embodiment of a video static region recognition method according to this application; Figure 2b This is a schematic diagram illustrating the execution of a video static region recognition method based on a multi-target mask image to generate a target slice image, according to an embodiment of this application. Figure 2c This is a schematic diagram illustrating the execution of a video static region recognition method based on generating a multi-target matrix from a target slice image, according to an embodiment of this application. Figure 3a This is a third schematic flowchart of an embodiment of the present application of a video static region recognition method; Figure 3b This is a schematic diagram illustrating the conversion of a multi-channel image into a single-channel image using a video static region recognition method according to an embodiment of this application. Figure 3c This is a schematic diagram illustrating the execution of a video static region recognition method according to an embodiment of this application, which performs segmentation on a single-channel image to obtain a target slice image; Figure 4 This is a fourth flowchart illustrating an embodiment of a video static region recognition method according to this application; Figure 5 This is the fifth flowchart of an embodiment of the present application of a video static region recognition method; Figure 6 This is a schematic flowchart of a video static region recognition method according to an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a video static area recognition device according to an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. The drawing numbers in this application are only used to distinguish the various steps in the solution and are not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0014] The essence of video static region identification is to compare consecutive frames in a video frame sequence to identify areas that remain static and do not undergo significant changes. These areas are typically the background of the video. For example, in surveillance videos, static regions may include relatively fixed areas such as buildings, roads, and the sky within the monitored area. Alternatively, in live broadcasts featuring people, the people are often dynamic, while the background behind them is often static. In practical applications, video static region identification can be used to extract video backgrounds or extract regions of interest (ROIs).

[0015] In one related technology, a third-party engine can be used to identify specific targets to achieve static region recognition. For example, a face recognition engine can only identify whether a face is static. If a target recognition engine is used for trees, flowers, houses, or other targets, it can only identify whether these preset targets are static. However, in many real-world scenarios, static regions are often complex and diverse, making extraction difficult by pre-specifying targets. Therefore, this approach has limited application scenarios, cannot handle targets that are not pre-specified or lack clear semantic labels, and lacks broad applicability. Judging target status solely based on comparing the overlap deviation of target selection boxes between consecutive frames with a fixed threshold is a simplistic method, easily affected by slight target movement or brief occlusion, leading to misjudgments and insufficient accuracy. Furthermore, the fixed threshold is difficult to adapt to different video scenarios, resulting in poor versatility. In complex backgrounds, it lacks a mechanism to eliminate background interference, exhibiting weak adaptability and poor scalability due to its reliance on specific third-party engines.

[0016] One related technique determines whether something is foreground or background by comparing pixel values ​​from consecutive frames. However, this method struggles with distinguishing similar colors, often failing to accurately differentiate between foreground and background when their colors are similar. For example, in a scene, the color of a foreground object might be very close to the background color. Even though the foreground object is moving, the pixel value deviation might be small, leading to the foreground object being misclassified as part of the background. In scenes with complex and dynamically changing backgrounds, such as wind blowing leaves, shimmering water, or constantly changing lighting, the actual value of each pixel may frequently deviate from its initial model, significantly reducing model accuracy. Furthermore, this method requires complex, non-storable statistical calculations and analysis of each pixel in the video to construct the background model, resulting in enormous computational and memory consumption, making each static target extraction extremely time-consuming.

[0017] In one related technique, a static and dynamic distinction is made between a target based on known foreground and background within a pre-defined scene. This method is limited by scene constraints, requiring pre-defined foreground and background information, and is often difficult to extend.

[0018] To address the problems existing in related technologies and improve the accuracy, versatility, applicability, and execution efficiency of video static region recognition, this application provides a video static region recognition method, such as... Figure 1a As shown, it includes: S11: Obtain the video frame sequence to be processed.

[0019] In this step, the input video is first read and then broken down into a series of consecutive video frames according to a predetermined frame rate or time interval. These ordered video frames form a video frame sequence, such as... Figure 1b As shown, it is used for subsequent identification and processing.

[0020] Video reading can be achieved using video processing libraries (such as OpenCV). In practical applications, video frames can be extracted continuously to improve recognition accuracy, or extracted at preset intervals to reduce computational costs and save storage space, depending on the specific needs.

[0021] In the step of extracting video frames based on a preset interval, the video frames can be extracted using a method based on the preset time interval. For example, the system's time module can be used to record the time interval for reading frames, extracting one frame every 0.1 seconds, and arranging the extracted images sequentially to construct a video frame sequence. This method of extracting and generating video frames based on time intervals (such as 0.1 seconds) reduces the storage and computational burden caused by extracting all frames, achieving the effects of saving storage space and improving computational efficiency.

[0022] S12: Perform full segmentation on each video frame in the video frame sequence using a pre-trained full segmentation model to obtain a multi-target mask image corresponding to the video frame. The multi-target mask image represents the region where each target is located in the corresponding video frame.

[0023] In this step, a pre-trained full segmentation model is used to perform full segmentation on each video frame. The pre-trained full segmentation model can be, for example, SAM (Segment Anything Model), which is trained on a large amount of image data. Specifically, SAM is divided into three pre-trained models: "ViT-H", "ViT-B", and "ViT-1". This application example uses the "ViT-H" model.

[0024] In practical applications, to improve operational efficiency, the required runtime environment for SAM can be further prepared. SAM supports Python 3.7 and above, and PyTorch 1.7 and above. It requires at least an NVIDIA graphics card with 8GB of dedicated video memory, and the necessary software packages for the graphics card must be installed.

[0025] In this step, the use of the pre-trained "ViT-H" model and a specific operating environment can effectively solve the problems of model adaptation and performance bottlenecks, ensuring the segmentation effect and running efficiency.

[0026] The aforementioned SAM can implement two segmentation methods: cue-based segmentation and full segmentation. Cue-based segmentation requires specific cue information to be input into the incoming image. This cue information guides the model to focus on specific regions or targets for segmentation. Full segmentation, on the other hand, does not require specific cue information; the model performs a comprehensive analysis and segmentation of the entire image, automatically identifying each target and generating a corresponding segmentation mask. This scheme selects the full segmentation method to perform full segmentation on the extracted video frames, fully extracting each target from the video frames.

[0027] Among them, the full segmentation function based on the SAM model can perform frame-by-frame comparison analysis of all targets in the video, thereby filtering out static targets, breaking through the limitations of specific targets and engines, improving the detection capability of various targets, adopting a more accurate and diverse target state judgment method, enhancing accuracy, constructing an effective strategy to deal with complex background interference, and having good scalability, which can be flexibly optimized and expanded to adapt to the ever-changing video processing needs.

[0028] For example, a single video frame in a video frame sequence, such as Figure 1c As shown. In this step, SAM is used to... Figure 1c Perform full segmentation to obtain the multi-target mask image corresponding to the video frame, as shown below. Figure 1d As shown, different target areas are distinguished by different colors. It should be understood that in practical applications, other identification methods can also be used to distinguish the areas of different targets within a video frame.

[0029] In this step, full segmentation is performed on each video frame in the video frame sequence, such as... Figure 1e As shown, each video frame in the video frame sequence is processed into a corresponding multi-target mask image.

[0030] Optionally, SAM can effectively segment targets within video frames. In practical applications, to improve the accuracy of contour information in target regions, this step can also be combined with other algorithms. For example, one or more methods such as Canny operator, Sobel operator, Prewitt operator, threshold segmentation, and semantic segmentation can be used to extract target contour information, thereby improving the recognition accuracy of each target region. In practical applications, multiple algorithms can be flexibly combined to implement this step based on the required accuracy, computing resources, and other conditions, meeting the actual needs of different application scenarios and enhancing algorithm flexibility.

[0031] S13: Construct a multi-target matrix corresponding to the video frame based on the multi-target mask image. Any slice of the multi-target matrix in a preset dimension represents the region where a target is located in a video frame.

[0032] In this step, a multi-target matrix is ​​used to represent the regions containing multiple targets within a video frame in a multi-dimensional way. The multi-target matrix includes multiple slices along a preset dimension, and any slice can represent the region containing a target within a video frame. Optionally, a slice can only represent the target's position within the video frame; for example, the slice's dimensions match the video frame's dimensions, using 1 to indicate the target's region within the video frame and 0 to indicate a region not belonging to the target. Alternatively, the slice can also include the pixel values ​​of the target's region to store the true color within that region. For example, the pixel values ​​of the target within the video frame are the target's true pixel values, while the pixel values ​​of pixels not belonging to the target are 0. Furthermore, other methods can be used to represent the target's region within the slice, such as simply representing the outline of the target's region.

[0033] This step uses a multi-target matrix to pinpoint the location of each target within a video frame, solving the problems of target information management and data organization, and enabling the orderly storage and processing of multiple independent targets. The multiple slices of the multi-target matrix along preset dimensions facilitate efficient target comparison in subsequent steps, improving processing efficiency.

[0034] S14: Based on the video frame sequence, perform target matching on the multi-target matrix corresponding to multiple consecutive video frames to obtain the target association relationship corresponding to multiple consecutive video frames.

[0035] In this step, relationships are established between the discrete targets obtained from segmentation, that is, the target associations between consecutive frames are established. Target associations are used to represent the correlation between the regions where targets are located in different video frames corresponding to the same target.

[0036] Since the time interval between consecutive adjacent video frames is short, the displacement of the same target is often small. Therefore, one or more methods such as coordinates, color values, and contour shapes can be used to compare targets based on similarity.

[0037] In this step, target association relationships can be constructed based on preset rules. For example, based on a slice in a multi-target matrix of a video frame in a preset dimension, a similarity comparison is performed with each slice in the multi-target matrix of adjacent video frames in the preset dimension. An association relationship is constructed between two slices with a similarity greater than a preset similarity, thereby generating target association relationships based on the group of slices with association relationships.

[0038] Target slices that cannot be associated with each other in this step can be considered dynamic targets. The target associations constructed in this step can be directly used to determine static targets. Alternatively, based on the target associations, further judgments can be made according to the static target determination rules to more strictly determine which targets are static targets.

[0039] S15: Determine the location of the static target in multiple consecutive video frames based on the target association relationship.

[0040] In this step, the location of static targets is determined across multiple consecutive video frames based on target association relationships, and the pixel coordinates of the static target's location are ultimately output. Specifically, for any given target, slices associated with that target are acquired, and consistency comparisons are used to determine the degree of change in the target's location across multiple video frames. Targets with a change less than a preset threshold are identified as static targets. For any static target, the slices associated with it can be combined and output together to show the location of the static target across multiple video frames. In this step, all identified static target slices can be grouped and output, with each group of slices identifying the location of a static target across multiple video frames.

[0041] The solution provided in this application can achieve static region recognition of various targets in video frames without pre-specifying the categories of objects to be recognized, making it particularly suitable for video frames with variable content. By performing comparisons on the target's location in consecutive video frames, the effectiveness of static region recognition can be improved. This solution utilizes the SAM model to achieve full segmentation of all targets in the video, without relying on specific semantic labels or preset targets. Through continuous video frame target matching and staticity determination, it achieves high accuracy, high efficiency, and has wide applicability and scalability.

[0042] This solution achieves full target segmentation based on the SAM model, effectively improving static target recognition capabilities and overcoming interference from similar colors and complex backgrounds. Furthermore, leveraging the versatility and efficiency of the SAM model, it eliminates the need for complex parameter tuning or retraining for specific scenarios, avoiding individual statistical modeling of each pixel. This allows for rapid analysis and processing of video content, significantly improving the accuracy and efficiency of static target extraction and better meeting the needs of video processing in various scenarios.

[0043] Compared to related technologies, this solution overcomes the limitation of only recognizing preset targets. Utilizing a large SAM model, it achieves accurate full segmentation of all targets in the video, with a wide range of applications, covering static regions without clear semantic labels. Through multi-frame joint analysis, it significantly enhances the accuracy and stability of static target detection, reducing the probability of false positives. Based on a multi-target matrix, a multi-dimensional static judgment method improves recognition accuracy and can adapt to complex backgrounds and occlusion conditions. It avoids the enormous computational burden of pixel-level single-point statistics such as Gaussian mixture models, achieving efficient extraction of static video regions. Its performance in video surveillance, intelligent security, autonomous driving, video compression and enhancement, and other fields demonstrates significant commercial value.

[0044] The solution provided by the embodiments of this application enables the static region extraction algorithm to have the ability to recognize the static state of all elements in the video while ensuring effectiveness, thereby breaking through the limitations of specific scenarios and achieving wider domain adaptation.

[0045] Based on the solutions provided in the above embodiments, optionally, such as Figure 2a As shown, in step S13 above, constructing a multi-target matrix corresponding to the video frame based on the multi-target mask image includes: S21: Based on the region where each target is represented by the multi-target mask image, the corresponding video frame is segmented to generate a target slice image corresponding to each target.

[0046] See Figure 2b The multi-target mask image obtained by performing full segmentation on a video frame using SAM is as follows: Figure 2b As shown on the left, each grid represents a pixel, and the color of the grid indicates the target to which the pixel belongs. In other words, pixels belonging to the same target area are shown in the same color.

[0047] In this step, the region where each target is located is segmented based on the multi-target mask image, such as... Figure 2b As shown on the right. For example, the region where the target is located (yellow) is identified from a multi-target mask image, and a first target slice image is generated accordingly. Pixels in the target slice image that belong to the target region are identified as the first marker (e.g., GRAY), and pixels that do not belong to the target region are identified as the second marker (e.g., 0).

[0048] Optionally, in generating a target slice image, the region containing a target is first identified from the multi-target mask image. For example, the region containing yellow pixels in the target slice image is determined. Then, the portion of the region containing yellow pixels that overlaps with the video frame is extracted to construct the target slice image of that target. Pixels not covered by the region containing yellow pixels are filled with a second marker (e.g., 0). In the resulting target slice image, the color values ​​of the original pixels in the target region can be preserved, thus fully retaining the features of the original image.

[0049] Similarly, segmentation is performed on each target in the target mask image to obtain multiple target slice images, with each target slice image corresponding to the region where one target is located. This step performs video frame segmentation based on multiple target mask images. S22: Stack the target slice images corresponding to each target in the multi-target mask image based on the preset dimension to obtain the multi-target matrix.

[0050] See Figure 2c In this step, based on the multiple target slice images obtained in step S21 above (such as...), Figure 2c (As shown on the left) Construct a multi-objective matrix (such as) Figure 2c (As shown on the right). Specifically, target slice images corresponding to multiple targets are stacked according to a preset dimension to construct a multi-dimensional multi-target matrix. Here, a slice of the multi-target matrix in the preset dimension is a single target slice image.

[0051] The solution provided in this application performs segmentation and integration on a multi-target mask image, generating a multi-target matrix that can represent the location of each target in a video frame within a preset dimension. Integrating the target locations using a multi-dimensional matrix improves the efficiency of subsequent target matching. Furthermore, the multi-dimensional matrix helps preserve the original pixel features of the video frame; these features stored in the matrix can be used in subsequent steps to improve the effectiveness of target matching.

[0052] Based on the solutions provided in the above embodiments, optionally, such as Figure 3a As shown, before step S21 above, that is, before segmenting the corresponding video frames based on the regions where each target is represented by the multi-target mask image to generate target slice images corresponding to each target, the method further includes: S31: If the video frame corresponding to the multi-target mask image is a multi-channel image, then the multi-channel image is converted into a single-channel image by weighted averaging.

[0053] In this embodiment, converting a multi-channel image into a single-channel image effectively reduces the dimensionality of image features and improves the processing efficiency of subsequent steps. For example, see... Figure 3bFor RGB three-channel images, this step converts the three-channel image of each image frame into a grayscale single-channel image. This step simplifies the data, thereby improving computational efficiency. Converting to a single channel also further reduces the impact of lighting and color variations, thus enhancing the algorithm's accuracy in identifying static targets.

[0054] In this step, a weighted average method is used to achieve single-channel conversion, which is implemented through the following formula (3-1): (3-1) In the above formula, R, G, and B represent the pixel values ​​of the red, green, and blue channels of the color image, respectively. The weighting coefficients of 0.3, 0.59, and 0.11 for the pixel values ​​are only examples and can be flexibly adjusted according to actual needs in practical applications. The Gray value obtained by formula (3-1) is the output grayscale value. This grayscale value is obtained by weighted averaging, which solves the interference of color and illumination changes, improves computational efficiency, and enhances the accuracy of static target identification.

[0055] In step S21 above, segmentation is performed on the corresponding video frames based on the regions where each target is represented by the multi-target mask image, generating target slice images corresponding to each target, including: S32: Based on the regions where each target is represented by the multi-target mask image, the single-channel image is segmented to generate target slice images corresponding to each target.

[0056] See Figure 3c In this step, a single-channel image is segmented based on a multi-target mask image to obtain target slice images corresponding to each target. Specifically, the single-channel image is segmented according to the region corresponding to each target in the multi-target mask image, and a corresponding target slice image is generated for each segmented single-channel region image. This target slice image includes the portion of the image corresponding to the target in the single-channel image, which can represent the region where the target is located in the single-channel image and the pixel grayscale value of the target in the single-channel image.

[0057] The solution provided in this application converts multi-channel images into single-channel images, which can effectively reduce the computing power required for processing, improve the generation efficiency of target slice images and multi-target matrices, and reduce the negative impact of color changes caused by illumination on subsequent steps.

[0058] Based on the solutions provided in the above embodiments, optionally, such as Figure 4 As shown, in step S14 above, target matching is performed on the multi-target matrix corresponding to multiple consecutive video frames based on the video frame sequence to obtain the target association relationship corresponding to multiple consecutive video frames, including: S41: Based on the first target slice image of the multi-target matrix corresponding to the k-th frame in the video frame sequence, determine the similarity with each target slice image of the multi-target matrix of the (k+1)-th frame, wherein the similarity is determined according to the Euclidean distance of the center point of the target region and / or the overlap of the target region.

[0059] In this step, according to the order in the video frame sequence, for adjacent k-th and k+1-th frames, using the first target slice image in the k-th frame as a reference, a similarity comparison is performed with multiple target slice images in the k+1-th frame. The similarity comparison can be determined by the Euclidean distance between the center points and / or the overlap of the target regions.

[0060] The following explains how to perform similarity comparison using Euclidean distance from the application center point.

[0061] In this step, the center point of the target region in the first target slice image in frame k and the center points of the target regions in each of the multiple target slice images in frame (k+1) are determined. For any target slice image, the center point of the target can be calculated using the coordinates of the target's outer contour line.

[0062] Specifically, for a target slice image, all coordinate points on the outer contour line of the target are traversed. The coordinate values ​​of these points in the horizontal direction (x-axis) and vertical direction (y-axis) are summed respectively. Then, the sum of the summed x-coordinates is divided by the number of contour points to obtain the center coordinate value in the x-direction. Similarly, the sum of the summed y-coordinates is divided by the number of contour points to obtain the center coordinate value in the y-direction. Thus, the x-coordinate and y-coordinate of the center point are obtained, as shown in formula (4-1).

[0063] (4-1) In the formula, n x is the number of points on the outer contour of the target. i y i Let x and y be the pixel coordinates of the i-th outer contour point, respectively, along the x and y axes. c y c The coordinates are the center coordinates of the target.

[0064] After determining the center point of the target region in the first target slice image of frame k and the center points of the target regions in multiple target slice images of frame k+1, the Euclidean distance is calculated between the center point of the target in the first target slice image and the center point of each target in frame k+1. This Euclidean distance represents the correlation between the location of each target in frame k+1 and the location of the target in the first target slice image. The Euclidean distance between the center points can be calculated using formula (4-2): (4-2) In the formula, the center point coordinates of the target in the k-th frame are (x1, y1), the center point coordinates of a target in the (k+1)-th frame are (x2, y2), and the calculated Euclidean distance between the center points is d.

[0065] The Euclidean distances between the center point of target A in frame k and the center points of each target in frame (k+1) are calculated using the above method, forming the distance matrix d of target A. n To facilitate the identification of the most similar targets, the distance matrix d can be used. n The targets are arranged in ascending order of distance value. Since the time difference between adjacent video frames is very small, the center point of the same target in the same video often overlaps significantly in adjacent frames. In this step, if the minimum value d in the Euclidean distance matrix... min The similarity threshold d is less than or equal to the preset distance threshold. t If the following equation (4-3) is satisfied, then the (k+1)th frame is determined to be related to d. t The corresponding objective is the same as objective A mentioned above.

[0066] (4-3) The following explains how to compare the similarity of the overlapping areas of the application target.

[0067] For target A in frame k and all targets B in frame (k+1) n The segmentation mask corresponding to the target is obtained from the target slice image. The segmentation mask of target A is represented as M. A Target set B n The segmentation mask for a specific target B in the image is represented as M. B The target overlap (IOU) can then be calculated using formula (4-4).

[0068] (4-4) The symbol | * | represents the number of elements in the set (i.e., the number of pixels). Calculate target A in frame k and all targets B in frame (k+1). n IOU (Interval Overlap) ABn In practical applications, the same target in adjacent frames often largely overlaps; therefore, based on IOU... ABn For all targets B in the (k+1)th frame n Arrange from smallest to largest, if the largest item has the highest IOU ABmax greater than the preset overlap threshold IOU t Then determine the target A in the k-th frame and the target A in the (k+1)-th frame with IOU. ABmax Corresponding target Bmax For the same goal.

[0069] In this step, the similarity between targets can be expressed solely based on the center point coordinates or the region overlap, or the center point coordinates and the region overlap can be combined into a single value using a weighted summation method to represent the similarity between targets.

[0070] S42: If the similarity between the first target slice image and the second target slice image in the (k+1)th frame satisfies a preset association condition, then it is determined that the first target slice image in the kth frame and the second target slice image in the (k+1)th frame correspond to the same target.

[0071] The preset association conditions can be flexibly set according to actual needs. For example, the preset association conditions can be "the Euclidean distance between the center points is less than or equal to the preset distance similarity judgment threshold, or the region overlap is greater than the preset overlap threshold", which can determine that the target corresponding to the first target slice image and the target corresponding to the second target slice image are the same target.

[0072] The solution provided in this application achieves target association matching and recognition between adjacent video frames through similarity, efficiently constructing target associations between different video frames. By associating targets between adjacent frames, the transient error problem of single-frame judgment is solved by using center point coordinate fitting and region overlap matching thresholds, achieving more stable and accurate static target recognition. Furthermore, similarity can achieve region feature expression at the image pixel level, effectively improving the accuracy of target association. In practical applications, if a target in the k-th frame cannot be associated with any target in the (k+1)-th frame, this target in the k-th frame can be temporarily identified as a dynamic target.

[0073] Based on the solutions provided in the above embodiments, optionally, such as Figure 5 As shown, in step S15 above, determining the location of a static target in multiple consecutive video frames based on the target association relationship includes: S51: If target slice images corresponding to the same target to be determined in at least a preset number of consecutive video frames satisfy the static target determination condition, then the target to be determined is determined to be a static target in the at least a preset number of consecutive video frames.

[0074] The preset number n can be flexibly adjusted according to actual needs (e.g., set to 100 frames). The larger the value, the stricter the determination of static targets. In this step, if there is a target A that meets the static target determination criteria within n consecutive video frames, then target A is determined to be a static target.

[0075] The static target determination criteria can be flexibly set according to actual needs. Optionally, based on the solution provided in the above embodiments, such as... Figure 6 As shown, the static target determination criteria include at least one of the following: In multiple target slice images corresponding to the same target to be determined, the slope of the fitted straight line of the center point of the region where the target to be determined is located in the preset coordinate direction is less than the preset slope. In multiple target slice images corresponding to the same target to be determined, the overlap of the regions where the target to be determined is located is greater than a preset overlap. In multiple target slice images corresponding to the same target to be determined, the pixel value similarity of the region where the target to be determined is located in each target slice image is greater than a preset similarity.

[0076] In this embodiment, for multiple target slice images corresponding to the same target to be determined, static determination is achieved from at least one aspect: the degree of change in the center point, the degree of change in regional overlap, and the degree of change in pixel values. By using at least one of the above conditions, it can be determined whether the target has changed based on the characteristics of the target's appearance, thereby determining whether the target belongs to a static target.

[0077] The following section explains the method of fitting the center point coordinates.

[0078] In this example, the center point of target A in n frames of video is calculated, and the coordinates (x, y, y) of the center point in each frame are... i ,y i (where i = 1, 2, ..., n) are used for fitting. Specifically, the least squares method can be used for linear fitting, or other fitting methods can be used. If the slope of the fitted line in the horizontal x-direction and the slope of the fitted line in the vertical y-direction approach 0 after linear fitting, it indicates that the position of the center point of target A changes very little within these n frames. The result of this center point coordinate fitting is one of the important criteria for judging whether the target is static. It can indicate that the center position of this target should be relatively stable in multiple consecutive frames, without obvious linear displacement changes.

[0079] The following section explains the method of using regional overlap.

[0080] In this example, the Intersection over Union (IOU) of target A in n frames of video is calculated. If the average IOU is greater than a set threshold (e.g., 0.8) over n consecutive frames, it indicates that target A has a high degree of overlap between the n frames, and its shape and position change very little.

[0081] The following explains the method of using pixel values.

[0082] Since the grayscale values ​​of individual pixels within a static target remain unchanged over n consecutive frames, under stable lighting and other environmental conditions, the grayscale values ​​of each pixel should be relatively independent and stable. Based on this, in this example, we can determine whether a target is static by calculating the covariance of pixel values ​​point by point. For example, suppose the grayscale value sequence of a certain pixel over n consecutive frames is h. j (i) (where j represents the specific pixel number, i = 1, 2, ..., n), h is calculated using the following formula (5-1). j Covariance of (i): (5-1) If the covariance value is lower than the threshold t (t can be a value close to 0), it indicates that the grayscale value of the pixel does not change much within n frames, which is consistent with the characteristics of a static target. Based on this, if within n consecutive frames, the proportion of static pixels in target A exceeds the threshold k... t If target A is considered a static target, then k is considered a static target. t It can be a value close to 1.

[0083] S52: Determine the region where the static target is located based on the target slice images corresponding to the static target in at least a preset number of consecutive video frames.

[0084] In this step, based on the static target determination results described above, the region where the static target is located can be represented by an output set. Specifically, a static target image set can be constructed from multiple target slice images associated with the determined static target in a consecutive preset number of video frames, and the static target image set can be output to represent the region where the corresponding static target is located.

[0085] Alternatively, multiple target slice images associated with the identified static target can be obtained from a predetermined number of consecutive video frames. These multiple target slice images can then be integrated and output as a multidimensional matrix. Alternatively, the multiple target slice images can be integrated into a single image, with the target's location within the image represented by its pixel values.

[0086] Below, we provide an alternative example for using pixel values.

[0087] In the actual process of acquiring video images, numerous interference factors often arise, such as noise from the image acquisition device itself, minor fluctuations in ambient light, and noise generated during image transmission. These interferences can cause unnecessary and chaotic fluctuations in the grayscale values ​​of the target area. If subsequent fitting operations (whether linear fitting of the average grayscale value or point-by-point calculation of covariance) are performed directly based on such noisy grayscale data, misjudgment is likely to occur, making it impossible to accurately determine whether the target is static.

[0088] In this embodiment, filtering is performed before pixel matching, for example, mean filtering, median filtering, Gaussian filtering, etc.

[0089] Mean filtering replaces the original gray value of a target pixel by calculating the average gray value of the pixels in the neighborhood of the target pixel. It can smooth the image to a certain extent and reduce gray value abrupt changes caused by noise.

[0090] Median filtering selects the median gray value of a neighboring pixel as the replacement value, and it is particularly effective in removing impulse noise such as salt-and-pepper noise.

[0091] Gaussian filtering assigns different weights to pixels in the neighborhood based on the Gaussian function, and then updates the gray value of the pixels by weighted averaging. It can effectively suppress noise while preserving details such as image edges, making the image smoother.

[0092] Optionally, this solution can use a spatial filter to implement filtering. Alternatively, other filters can be selected, such as high-pass and low-pass filters based on Fourier transform, band-pass filters, etc.

[0093] After filtering, the covariance is calculated point-by-point using pixel values ​​as described in the above embodiments. This scheme solves the problem of misjudgment caused by image noise and illumination fluctuations through filtering, achieving noise suppression and image smoothing, and improving the reliability of pixel matching. The covariance threshold and static pixel ratio threshold address the quantitative requirements of static pixel point judgment, realizing accurate quantification of the static target judgment process.

[0094] Below, in conjunction with Figure 6 This plan will be further explained.

[0095] This application provides a method for identifying static regions in a video, which can flexibly identify static regions in a video without being affected by the target type.

[0096] First, input the video that needs to be recognized, and then perform video processing through the following steps: (a) Video frame extraction: Extract consecutive video frames at a predetermined frame rate or time interval (e.g., every 0.1 seconds). Interval extraction of video frames can effectively save computational costs.

[0097] (b) Single-frame SAM full segmentation: Use a pre-trained SAM (e.g., ViT-H model) to perform full segmentation on each video frame to obtain high-quality masks for each target.

[0098] (c) Target storage: Convert RGB images to grayscale images, segment grayscale video frames using masks and extract targets, and construct a multi-target matrix.

[0099] (d) Target matching between consecutive frames: The target correspondence between consecutive frames is established by calculating the Euclidean distance between the target center points (judged by a threshold) and matching the target mask overlap (IOU).

[0100] (e) Static target assessment: Within n consecutive frames (n≥100), the stability of the matched target is assessed, specifically including: (e-1) Linear fitting of the center point coordinates determines displacement stability; (e-2) Overlap degree matching determines morphological stability; (e-3) Filtered pixel matching determines the stability of pixel grayscale changes.

[0101] If the above three conditions are met and the proportion of static pixels exceeds the threshold, the target is determined to be static.

[0102] (f) Video static region extraction: Output the union of all static target masks in n consecutive frames to form the final video static region mask.

[0103] If target A, which was matched in step d, is determined to be a static target in step e, output the union of all segmentation masks for A in n frames, as the final static mask for static target A. The union of all final static masks determined to be static targets within n frames constitutes the static region within the video. Return and output the static region mask.

[0104] The solution provided in this application introduces the SAM model to accurately segment individual targets in an image. Regardless of whether the target has low contrast with the background, partial occlusion, or irregular target shape, it can accurately delineate the target outline and obtain high-quality mask information, providing accurate and clear target range definition for subsequent static region judgment. This has significant advantages over traditional segmentation methods.

[0105] In practical applications, methods for determining whether a target is static using edge information suffer from limitations imposed by the quality of filtering algorithms. This results in excessive noise in the extracted edge information, leading to low accuracy. While segmentation models, whether threshold-based or semantic segmentation, can extract sufficiently accurate edge contours, thresholding is limited by the number of thresholds, and semantic segmentation by the label type. They cannot segment all targets in a video frame; they can only determine if a preset target is static. Therefore, static target identification based on traditional segmentation methods can only determine if the region of interest (ROI) is static, not if the non-ROI is static. This solution utilizes a SAM model to extract all target information to perform static region extraction. This ensures that the identification of static regions simultaneously includes both ROI and non-ROI while remaining unaffected by noise, resulting in superior technical performance compared to the aforementioned methods.

[0106] This scheme uses a full segmentation model to separate all targets in a video frame from the overall image, forming independent entities for separate judgment. This separation allows the analysis of each target to be performed without being affected by other targets, avoiding judgment errors caused by factors such as mutual occlusion and motion correlation between targets. Furthermore, the full segmentation model can traverse and judge all targets in the video frame, rather than only judging whether the pre-defined segmented targets are static, making it more versatile.

[0107] This scheme employs the SAM model for target extraction. By accurately segmenting all targets in the image using the SAM model, it can identify and judge all targets in the video image, rather than the preset targets of ordinary target recognition models. This scheme compares image features across n frames of the video (n at least greater than 100). This multi-frame comparison method greatly enhances the accuracy and stability of static target detection. By analyzing image features from more frames, it can more comprehensively capture the state changes of the target over a period of time, reducing misjudgments caused by factors such as single-frame noise or local occlusion.

[0108] In this scheme, the accuracy and reliability of target detection are further improved by comparing the contour and pixel features of the target within n frames through center point coordinate fitting, overlap matching, and pixel matching. Center point coordinate fitting can more accurately locate the trend of the target's center position change. Even if the target undergoes slight displacement or rotation, it can be effectively tracked and judged by the changes in center point coordinates over multiple consecutive frames. Overlap matching takes into account factors such as the target's shape and size. For targets that are partially occluded or have slightly changed shapes, it can still accurately determine whether they are the same static target, greatly enhancing the system's adaptability to complex scenes. Pixel matching analyzes the pixel changes of the target over multiple frames at a more microscopic level. It can keenly capture subtle changes in the target's surface texture, color, and other features, which plays a crucial role in distinguishing similar targets and identifying whether the target has undergone substantial state changes.

[0109] In the solution provided in this application embodiment, because the SAM large model is used to perform high-precision full segmentation of video frames without the need for preset semantic labels, the contours of all targets in the video can be accurately extracted, achieving coverage and accurate segmentation of complex and diverse targets. Because time-interval-based video frame extraction is used, redundant data is reduced and computational efficiency is improved. Converting RGB images to grayscale images and applying spatial filters reduces illumination and noise interference, enhancing the accuracy of target staticity judgment. By constructing target matching relationships between consecutive frames and utilizing the Euclidean distance of the center point and the mask overlap (IOU), target consistency is confirmed from a spatial perspective. Combining multi-dimensional static judgment conditions such as center point coordinate fitting, overlap matching, and pixel covariance matching of continuous targets across multiple frames enhances the robustness and accuracy of recognition. Simultaneously, it avoids the traditional single-frame or few-frame-based static judgment single mode, improving stability and versatility. In summary, these key technical features work together to make this solution superior in terms of static region accuracy, adaptability, efficiency, and versatility.

[0110] In this embodiment, a full segmentation model is used to perform full segmentation of video frames. No pre-specified prompts are needed in the segmentation steps, enabling comprehensive analysis and segmentation of the entire video frame. By constructing a multi-target matrix, the location of each target in the video frame can be represented in a multi-dimensional matrix, facilitating target matching and association in subsequent steps. The target association relationships obtained through matching can represent the correspondence between targets in consecutive video frames, thus indicating the location of the same target in multiple consecutive video frames. Based on the target association relationships, it can be determined whether the same target has changed in multiple consecutive video frames, thereby efficiently and accurately determining whether the target is a static target. The solution provided in this embodiment enables the static region extraction algorithm to maintain effectiveness while possessing the ability to recognize the static state of all video elements, thereby overcoming the limitations of specific scenarios and achieving broader domain adaptability. This solution can achieve static region recognition of various targets in video frames without pre-specifying the categories of things to be recognized, and is particularly suitable for video frames with variable content. By comparing the target locations in consecutive video frames, the effectiveness of static region recognition can be improved.

[0111] The solution provided in this application can extract static regions from videos, addressing needs across various fields. Its algorithm has high demand and covers a wide range of sectors; if successfully commercialized, it will generate significant economic value and market influence across multiple industries.

[0112] In fields such as intelligent security and traffic management, which require monitoring systems, video surveillance systems can quickly locate objects or scenes that remain in a fixed state for extended periods by extracting static areas from the video feed. Examples include vehicles that have been parked in a parking lot for a long time without being moved, or suspicious packages that have been left untouched in a corner of a building for an extended period.

[0113] Dynamic regions can also be extracted by excluding static regions, which is crucial for accurately monitoring moving targets. This is of paramount importance in fields such as intelligent security, traffic management, industrial production, and logistics, enabling more efficient detection of abnormal personnel movements, vehicle violations, and other dynamic behaviors.

[0114] Furthermore, static region extraction algorithms can also be applied to intelligent driving. For example, in target recognition and tracking algorithms, by accurately extracting static regions first, fixed background interference factors can be eliminated, providing a "cleaner" input image for subsequent target recognition models. In intelligent driving assistance systems, accurate extraction of static regions helps identify static elements such as roads, traffic signs, and fixed roadside buildings, providing crucial environmental information for vehicle path planning, obstacle detection, and other functions.

[0115] Static region extraction can also be used in video compression and video enhancement algorithms. In video compression, since the pixel values ​​of static regions remain largely unchanged across multiple frames, it's unnecessary to fully encode and transmit the static portion of each frame. This can significantly improve video compression rates and reduce video storage costs. In video enhancement, static region extraction algorithms select or remove static regions from a video, thereby enhancing the region of interest.

[0116] In summary, this algorithm has broad market demand and huge commercial potential, and can become a key technical support in multiple industries.

[0117] To address the problems existing in related technologies, this application also provides a video static region recognition device 70, such as... Figure 7 As shown, it includes: Module 71 acquires the video frame sequence to be processed; The segmentation module 72 performs full segmentation on each video frame in the video frame sequence using a pre-trained full segmentation model to obtain a multi-target mask image corresponding to the video frame. The multi-target mask image represents the region where each target is located in the corresponding video frame. Construction module 73 constructs a multi-target matrix corresponding to the video frame based on the multi-target mask image, wherein any slice of the multi-target matrix in a preset dimension represents the region where a target is located in a video frame; Matching module 74 performs target matching on the multi-target matrix corresponding to multiple consecutive video frames based on the video frame sequence to obtain the target association relationship corresponding to multiple consecutive video frames; The determination module 75 determines the location of the static target in multiple consecutive video frames based on the target association relationship.

[0118] The apparatus provided in this application first acquires a sequence of video frames to be processed. Then, a pre-trained full segmentation model is used to perform full segmentation on each video frame in the video frame sequence, obtaining a multi-target mask image corresponding to each video frame. The multi-target mask image represents the region where each target is located in the corresponding video frame. In this scheme, full segmentation of the video frames is performed using a full segmentation model, eliminating the need for pre-specified prompts during the segmentation step, enabling comprehensive analysis and segmentation of the entire video frame. Next, a multi-target matrix corresponding to the video frame is constructed based on the multi-target mask image. Any slice of the multi-target matrix in a preset dimension represents the region where a target is located in a video frame. By constructing the multi-target matrix, the region where each target is located in the video frame can be represented by a multi-dimensional matrix, facilitating target matching and association in subsequent steps. Subsequently, target matching is performed on the multi-target matrices corresponding to multiple consecutive video frames based on the video frame sequence, obtaining the target association relationship between multiple consecutive video frames. The target association relationship obtained through matching can represent the correspondence between targets in consecutive video frames, thereby indicating the location of the same target in multiple consecutive video frames. Finally, the location of the static target is determined in multiple consecutive video frames based on the target association relationship. Based on the above target association relationship, it can be determined whether the same target changes in multiple consecutive video frames, thereby efficiently and accurately determining whether the target is a static target. The solution provided by this application embodiment can achieve static region recognition for various targets in video frames without pre-specifying the categories of things to be recognized, and is particularly suitable for video frames with frequently changing content. By performing comparison of the target's location in consecutive video frames, the effectiveness of static region recognition can be improved.

[0119] In this application, the modules in the apparatus provided can also implement the method steps provided in the method embodiments. Alternatively, the apparatus provided in this application may include other modules besides those described above to implement the method steps provided in the method embodiments. Furthermore, the apparatus provided in this application can achieve the technical effects achievable by the method embodiments.

[0120] Preferably, this application embodiment also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described embodiment of the video static region recognition method and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0121] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described embodiment of the video static region recognition method and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0122] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program is operable to cause a computer to perform some or all of the steps of the above-described embodiment of a video static region recognition method, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0127] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0128] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0129] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0130] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0131] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for identifying static regions in a video, characterized in that, include: Obtain the video frame sequence to be processed; By performing full segmentation on each video frame in the video frame sequence using a pre-trained full segmentation model, a multi-target mask image corresponding to the video frame is obtained. The multi-target mask image represents the region where each target is located in the corresponding video frame. Based on the multi-target mask image, a multi-target matrix corresponding to the video frame is constructed. Any slice of the multi-target matrix in a preset dimension represents the region where a target is located in a video frame. Based on the video frame sequence, target matching is performed on the multi-target matrix corresponding to multiple consecutive video frames to obtain the target association relationship corresponding to multiple consecutive video frames; Based on the target association relationship, the location of the static target is determined in multiple consecutive video frames.

2. The method as described in claim 1, characterized in that, Constructing a multi-target matrix corresponding to a video frame based on the multi-target mask image includes: Based on the region where each target is represented by the multi-target mask image, the corresponding video frame is segmented to generate a target slice image corresponding to each target. The multi-target matrix is ​​obtained by stacking the target slice images corresponding to each target in the multi-target mask image based on the preset dimension.

3. The method as described in claim 2, characterized in that, Before segmenting the corresponding video frames based on the regions where each target is represented by the multi-target mask image, and generating target slice images corresponding to each target, the method further includes: If the video frame corresponding to the multi-target mask image is a multi-channel image, then the multi-channel image is converted into a single-channel image by weighted averaging. Specifically, based on the region where each target is represented by the multi-target mask image, the corresponding video frame is segmented to generate a target slice image corresponding to each target, including: The single-channel image is segmented based on the region where each target is represented by the multi-target mask image, generating target slice images corresponding to each target.

4. The method as described in claim 2 or 3, characterized in that, Based on the video frame sequence, target matching is performed on the multi-target matrix corresponding to multiple consecutive video frames to obtain the target association relationship corresponding to multiple consecutive video frames, including: Based on the first target slice image of the multi-target matrix corresponding to the k-th frame in the video frame sequence, the similarity with each target slice image of the multi-target matrix of the (k+1)-th frame is determined, wherein the similarity is determined according to the Euclidean distance of the center point of the target region and / or the overlap of the target regions. If the similarity between the first target slice image and the second target slice image in the (k+1)th frame satisfies a preset association condition, then it is determined that the first target slice image in the kth frame and the second target slice image in the (k+1)th frame correspond to the same target.

5. The method as described in claim 4, characterized in that, Determining the location of a static target across multiple consecutive video frames based on the target association relationship includes: If target slice images corresponding to the same target to be determined in at least a preset number of consecutive video frames satisfy the static target determination condition, then the target to be determined is determined to be a static target in the at least preset number of consecutive video frames; The region where the static target is located is determined based on the target slice images corresponding to the static target in at least a preset number of consecutive video frames.

6. The method as described in claim 5, characterized in that, The static target determination criteria include at least one of the following: In multiple target slice images corresponding to the same target to be determined, the slope of the fitted straight line of the center point of the region where the target to be determined is located in the preset coordinate direction is less than the preset slope. In multiple target slice images corresponding to the same target to be determined, the overlap of the regions where the target to be determined is located is greater than a preset overlap. In multiple target slice images corresponding to the same target to be determined, the pixel value similarity of the region where the target to be determined is located in each target slice image is greater than a preset similarity.

7. A video static region recognition device, characterized in that, include: The acquisition module acquires the video frame sequence to be processed. The segmentation module performs full segmentation on each video frame in the video frame sequence using a pre-trained full segmentation model to obtain a multi-target mask image corresponding to the video frame. The multi-target mask image represents the region where each target is located in the corresponding video frame. The construction module constructs a multi-target matrix corresponding to the video frame based on the multi-target mask image. Any slice of the multi-target matrix in a preset dimension represents the region where a target is located in a video frame. The matching module performs target matching on the multi-target matrix corresponding to multiple consecutive video frames based on the video frame sequence to obtain the target association relationship corresponding to multiple consecutive video frames; The determination module determines the location of a static target in multiple consecutive video frames based on the target association relationship.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform the steps of the method as described in any one of claims 1 to 6.

Images