A method, device, electronic device and readable storage medium for occluding frame correction

Abstract

The present application provides a method, apparatus, electronic device, and readable storage medium for occluding frame correction. The method includes: obtaining first occlusion data and second occlusion data of a target video, where the first occlusion data represents the position of an occlusion frame at time T, the second occlusion data represents the position of the occlusion frame at time T+1, and T is a positive integer; processing the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information; and correcting the position of the occlusion frame at time T+1 according to the correction information. The present application inputs the position of the occlusion frame of each video frame in the target video and the position of the occlusion frame of the corresponding previous video frame into a Kalman filter to obtain correction information for correcting the position of the occlusion frame of each video frame output by the target tracking algorithm, reducing the position offset between different occlusion frames corresponding to adjacent video frames, ensuring the picture coherence during the playback of the target video, and improving the playback effect of the target video.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and in particular, to an occlusion box correction method, apparatus, electronic device, and readable storage medium. Background Art

[0002] In the intelligent logo occlusion scenario, it is necessary to occlude specific trademarks, clothing, etc. that appear in the video to avoid potential legal risks.

[0003] Currently, related technologies mostly use target tracking algorithms to track the objects to be occluded included in the video to be occluded, and perform blurring, virtualization, etc. on the objects to be occluded based on the tracking results. It is found in applications that the occlusion boxes output by the existing target tracking algorithms have position jitter, that is, there is a large deviation between the occlusion box position in the previous video frame and the occlusion box position in the subsequent video frame. The above-mentioned occlusion box jitter will destroy the picture coherence during video playback. That is to say, after the occlusion processing of the video to be occluded based on related technologies, the playback effect of the video is poor. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide an occlusion box correction method, apparatus, and related devices, which are used to solve the problem that the playback effect of the video is poor after the occlusion processing of the video to be occluded by related technologies.

[0005] In a first aspect, the embodiments of the present application provide an occlusion box correction method, and the method includes:

[0006] Obtain first occlusion data and second occlusion data of a target video, where the first occlusion data is used to represent the occlusion box position in the video frame at time T of the target video, and the second occlusion data is used to represent the occlusion box position in the video frame at time T + 1 of the target video, and T is a positive integer;

[0007] Process the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information;

[0008] Correct the occlusion box position in the video frame at time T + 1 of the target video according to the correction information.

[0009] In a second aspect, the embodiments of the present application further provide an occlusion box correction apparatus, and the apparatus includes:

[0010] A data acquisition module, configured to obtain first occlusion data and second occlusion data of a target video, where the first occlusion data is used to represent the occlusion box position in the video frame at time T of the target video, and the second occlusion data is used to represent the occlusion box position in the video frame at time T + 1 of the target video, and T is a positive integer;

[0011] A smoothing processing module, configured to process the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information;

[0012] A position correction module, configured to correct the position of the occlusion box in the video frame at time T+1 of the target video according to the correction information.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, including a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the above occlusion box correction method are implemented.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above occlusion box correction method are implemented.

[0015] In the embodiment of the present application, after determining the position of the occlusion box in each video frame of the target video by using the target tracking algorithm, the position of the occlusion box in each video frame of the target video and the position of the occlusion box in the corresponding previous video frame are input into the Kalman filter to obtain correction information for correcting the position of the occlusion box in each video frame output by the target tracking algorithm, reducing the position offset between different occlusion boxes corresponding to adjacent video frames, ensuring the picture coherence during the playback of the target video, and improving the playback effect of the target video. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for description in the embodiments of the present application will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 is a flowchart of an occlusion box correction method provided by an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of an occlusion box provided by an embodiment of the present application;

[0019] Figure 3 is a schematic structural diagram of an occlusion box correction device provided by an embodiment of the present application;

[0020] Figure 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0022] An embodiment of the present application provides a method for correcting an occlusion box. Refer to Figure 1 , Figure 1 which is a flowchart of the method for correcting an occlusion box provided by the embodiment of the present application. As Figure 1 shown, it includes the following steps:

[0023] Step 101, obtain first occlusion data and second occlusion data of a target video.

[0024] Among them, the first occlusion data is used to represent the position of the occlusion box in the video frame at time T of the target video, and the second occlusion data is used to represent the position of the occlusion box in the video frame at time T + 1 of the target video, where T is a positive integer.

[0025] After the customer uploads the video resource to the video playback platform, the platform party will review the video resource uploaded by the customer. During the review process, in addition to deleting or taking down the video including prohibited content, the platform party will also perform occlusion processing on the video that may have legal risks (for example: occluding unauthorized products or trademarks appearing in the video). The aforementioned target video can be the video detected by the platform party during the video review process and to be subjected to occlusion processing.

[0026] After determining the target video, a target detection algorithm or a trained target detection model can be applied to perform target detection on multiple video frames included in the target video, so as to detect the video frames including the objects to be occluded in the target video, and mark the objects to be occluded included in the video frame in the form of detection boxes.

[0027] It should be noted that since the system overhead of applying the target detection algorithm or the target detection model is large and the detection process is time-consuming, in actual application, the frame extraction method can be adopted to reduce the number of video frames that the target detection algorithm or the target detection model needs to process. After obtaining the target detection result, with the target detection result as a reference, the target tracking algorithm is applied to perform tracking detection on the remaining video frames in the target video that have not been extracted, so as to complete the target detection of all the video frames included in the target video.

[0028] For example, if the video duration of the target video is set to 1 hour and the frame density is 24 frames per second, to complete the object detection for all the video frames included in the target video, the target video can be first frame-sampled at a frame-sampling density of 1 frame per second. After frame-sampling, 3,600 first video frames are obtained. Subsequently, the 3,600 first video frames are processed according to the object detection algorithm or object detection model, and a detection box is used to mark the object to be occluded included in each first video frame. The first video frame marked with the detection box is called the second video frame. Then, according to the object tracking algorithm, using the corresponding second video frame per second as a reference, the remaining 23 video frames that are not frame-sampled within each second are tracked and detected, and a detection box is used to mark the object to be occluded included in the video frames that are not frame-sampled. Thus, the object detection for 86,400 video frames within the target video is completed.

[0029] The above frame-sampling density can be adjusted based on actual needs. For example, the frame-sampling density can be set to 2 frames per second, etc. The present application does not limit the specific value of the frame-sampling density.

[0030] As described above, by applying the combination of object detection and object tracking, the position of the detection box for each video frame within the target video can be determined, that is, the position of the occlusion box for each video frame within the target video can be determined. Based on the position of the occlusion box for each video frame within the target video, the first occlusion data and the second occlusion data can be obtained.

[0031] The first occlusion data includes the center point coordinates, box height, and box width of the corresponding occlusion box. Alternatively, the first occlusion data includes the coordinates of the lower left endpoint and the upper right endpoint (or the upper left endpoint and the lower right endpoint) of the corresponding occlusion box. In applications, other representation methods can also be selected to indicate the position of the occlusion box corresponding to the first occlusion data. The embodiments of the present application do not limit this.

[0032] The data representation method of the second occlusion data is the same as that of the first occlusion data. To avoid repetition, it will not be elaborated here.

[0033] Step 102: Process the first occlusion data and the second occlusion data according to the Kalman filter to obtain correction information.

[0034] It should be noted that time T and time T + 1 are within the target time period. The target time period is any one of the multiple time periods included in the target video. The durations of the multiple time periods included in the target video are the same. For example, if the video duration of the target video is set to 1 hour and the duration of each time period is 1 second, then 3,600 time periods can be obtained based on the target video.

[0035] The processing procedure of step 102 can be understood as follows: in each of the aforementioned multiple time periods, the Kalman filter is applied to smooth multiple video frames (excluding the first video frame in the time period) included in the time period. Specifically, the occlusion data indicating the occlusion box position in the target video frame is smoothed based on the occlusion data indicating the occlusion box position in the reference video frame, where the reference video frame and the target video frame are adjacent and the reference video frame is before the target video frame.

[0036] Exemplarily, when the duration of each time period is 1 second, the value range of T is [1, N - 1], the parameter N is the frame density of the target video, and preferably the first video frame per second is extracted as the first video frame.

[0037] It should be noted that the correction information can indicate the first occlusion position or the position offset between the first occlusion position and the second occlusion position. The first occlusion position can be understood as the occlusion box position determined after estimating the occlusion box position in the video frame at time T + 1 using the Kalman Filter, and the second occlusion position can be understood as the occlusion box position indicated by the aforementioned second occlusion data.

[0038] Step 103: Correct the occlusion box position in the video frame at time T + 1 according to the correction information.

[0039] As described above, the processing procedure of step 103 can be understood as correcting the occlusion box in the video frame at time T + 1 from the second occlusion position to the first occlusion position based on the correction information.

[0040] In the embodiment of the present application, after determining the occlusion box position of each video frame in the target video using the object detection algorithm and the object tracking algorithm, the occlusion box position of each video frame in the target video and the occlusion box position of the corresponding previous video frame are input into the Kalman filter to obtain the correction information for correcting the occlusion box position of each video frame output by the object tracking algorithm. This can complete the position correction of the occlusion box with relatively low processing time consumption, reduce the position offset between different occlusion boxes corresponding to adjacent video frames, ensure the picture coherence during the playback of the target video, and even keep the occluded object "stationary" in the same scene, reduce the user's perception of the occluded object during the playback of the target video, and improve the user's video viewing experience, that is, improve the playback effect of the target video.

[0041] It should be noted that after the position of the occlusion box for each video frame corresponding to the target video is corrected by using the method described in this application, the area enclosed by the occlusion box can be processed through image processing methods such as blurring and defocusing to complete the occlusion processing of the target video; the target video after the occlusion processing can be uploaded to a video playback platform and viewed by users who meet the playback conditions in the video playback platform.

[0042] Optionally, the processing of the first occlusion data and the second occlusion data according to the Kalman filter to obtain correction information includes:

[0043] Determine state information according to the first occlusion data and the second occlusion data, where the state information includes the position of the occlusion box in the video frame at time T + 1 of the target video, and the position offset between the position of the occlusion box in the video frame at time T of the target video and the position of the occlusion box in the video frame at time T + 1 of the target video;

[0044] Perform smoothing processing on the state information according to the Kalman filter to obtain correction information.

[0045] The state information can be understood as the feature information concerned by the Kalman filter. In this application, the state information not only includes the position of the occlusion box in the video frame at time T + 1 of the target video, but also includes the position offset between the position of the occlusion box in the video frame at time T of the target video and the position of the occlusion box in the video frame at time T + 1 of the target video. Through the above settings, that is, introducing the position offset between the position of the occlusion box in the video frame at time T and the position of the occlusion box in the video frame at time T + 1 into the state information, the influence degree of the feature information of the reference time on the feature information of the target time estimated by the Kalman filter is enhanced, so as to improve the occlusion box position correction effect, further reduce the position offset between the position of the occlusion box in the video frame at time T and the position of the occlusion box in the video frame at time T + 1 (after correction), and then improve the playback effect of the target video. Among them, the reference time and the target time are adjacent, and the reference time is before the target time.

[0046] Optionally, the performing smoothing processing on the state information according to the Kalman filter to obtain correction information includes:

[0047] Obtain the target state transition matrix and the target state estimation covariance matrix corresponding to time T + 1 of the target video;

[0048] Perform prior estimation according to the target state transition matrix and the state information to obtain prediction information;

[0049] According to the prediction information and the residual information, correction information is obtained, wherein the residual information is obtained according to the Kalman gain information corresponding to the (T + 1)-th moment of the target video, and the Kalman gain information corresponding to the (T + 1)-th moment of the target video is obtained according to the target state estimation covariance matrix.

[0050] As described above, after the state information is determined, the Kalman filter performs a prior estimation according to the target state transition matrix corresponding to the (T + 1)-th moment and the determined state information, so as to obtain prediction information for predicting the position of the occlusion box of the target video at the (T + 1)-th moment, wherein the target state transition matrix corresponding to the (T + 1)-th moment is used to characterize the position change trend of the occlusion box between the T-th moment and the (T + 1)-th moment;

[0051] Subsequently, the Kalman gain information is calculated based on the target state estimation covariance matrix corresponding to the (T + 1)-th moment, and the residual information is determined based on the calculated Kalman gain information. By aggregating the residual information and the prediction information, the state estimation information output by the Kalman filter can be obtained, that is, the aforementioned correction information can be obtained.

[0052] Among them, the target state estimation covariance matrix corresponding to the (T + 1)-th moment is used to characterize the influence degree of the prediction information at the (T + 1)-th moment and the position of the occlusion box of the detected target video at the (T + 1)-th moment on the aforementioned correction information.

[0053] It should be noted that after the state estimation information at the (T + 1)-th moment is obtained, the Kalman filter will also update the state estimation covariance matrix and the state transition matrix according to the state estimation information at the (T + 1)-th moment, so as to obtain the state estimation covariance matrix and the state transition matrix at the (T + 2)-th moment. When T is 1, that is, when the Kalman filter performs the first smoothing process within the target time period, the state estimation covariance matrix and the state transition matrix can be arbitrarily selected.

[0054] The state information further includes the first image feature corresponding to the object to be occluded in the video frame at the (T + 1)-th moment of the target video, and the feature difference between the second image feature corresponding to the object to be occluded in the video frame at the T-th moment of the target video and the first image feature.

[0055] As described above, introducing the first image feature and the feature difference between the first image feature and the second image feature into the status information, that is, introducing the image feature information tracked by the target tracking algorithm, to further enhance the influence degree of the image feature information at the reference moment on the feature information at the target moment estimated by the Kalman filter during the Kalman filtering process, reduce the occurrence probability of the problem of unstable tracking frames caused by the deformation of the object to be occluded, and suppress the noise interference caused by the background change in the video frame, so as to reduce the position offset between different occlusion frames corresponding to adjacent video frames, ensure the picture coherence during the playback of the target video, and improve the playback effect of the target video.

[0056] Optionally, the processing of the first occlusion data and the second occlusion data according to the Kalman filter to obtain correction information includes:

[0057] Obtain reference occlusion data, where the reference occlusion data is determined based on the occlusion frame positions in multiple consecutive video frames included in the target video, and the multiple consecutive video frames include the video frame at time T and the video frame at time T + 1;

[0058] According to the reference occlusion data and the second occlusion data, obtain an intersection over union parameter, where the intersection over union parameter is used to represent the overlap degree between the occlusion frames in the multiple consecutive video frames and the occlusion frame in the video frame at time T + 1;

[0059] When the intersection over union parameter is greater than or equal to the overlap threshold, process the first occlusion data and the second occlusion data according to the Kalman filter to obtain correction information.

[0060] Among them, the multiple consecutive video frames included in the target video can be understood as multiple consecutive video frames within a target time period, and the reference occlusion data is used to indicate the occlusion frame positions within the target time period.

[0061] In some embodiments, the occlusion frame positions in multiple consecutive video frames can be represented by the detection frame positions determined by a target detection algorithm or a target detection model within a target time period.

[0062] In some embodiments, the average value of the occlusion box positions in multiple consecutive video frames may also be calculated to obtain reference occlusion data. For example, if the reference occlusion data includes the center coordinates (x, y) of the occlusion box, the box width w, and the box height h, and the multiple consecutive video frames include video frame A1, video frame A2, and video frame A3, where the center coordinates, box width, and box height of the occlusion box in video frame A1 are {x1, y1, w1, h1} respectively, the center coordinates, box width, and box height of the occlusion box in video frame A2 are {x2, y2, w2, h2} respectively, and the center coordinates, box width, and box height of the occlusion box in video frame A3 are {x3, y3, w3, h3} respectively, then the center coordinates, box width, and box height of the occlusion box included in the reference occlusion data are {(x1 + x2 + x3) / 3, (y1 + y2 + y3) / 3, (w1 + w2 + w3) / 3, (h1 + h2 + h3) / 3}.

[0063] By calculating the intersection over union (IoU) parameter to determine whether the image features tracked at time T + 1 conform to the image features expected to be tracked by the target tracking algorithm, where the image features expected to be tracked by the target tracking algorithm are referred to by the reference occlusion data, and the image features tracked at time T + 1 are referred to by the second occlusion data.

[0064] The intersection over union parameter being greater than or equal to the overlap threshold can be understood as: the image features tracked at time T + 1 conform to the image features expected to be tracked by the target tracking algorithm, and subsequent correction operations for the occlusion box position can be performed.

[0065] The intersection over union parameter being less than the overlap threshold can be understood as: the image features tracked at time T + 1 do not conform to the image features expected to be tracked by the target tracking algorithm. Subsequently, the correction operation for the occlusion box position will be interrupted, and the occlusion box position indicated by the aforementioned reference occlusion data will be used as the occlusion box position at this moment, and the Kalman filter processing will be restarted.

[0066] As described above, the IoU judgment method is applied to avoid the interference of tracking results with large errors on the Kalman filter processing and ensure the accuracy of the correction information.

[0067] Exemplarily, if it is assumed that the target video includes consecutive video frames b1, b2, b3, b4, b5, and b6; the occlusion data of video frame b1 is obtained based on a target detection algorithm or a target detection model, and the occlusion data of video frames b2 - b6 is obtained based on a target tracking algorithm. Among them, video frame b5 fails the IoU judgment, and video frames b2, b3, b4, and b6 all pass the IoU judgment. Then, during the application of the Kalman filter, video frames b1 - b4 correspond to one Kalman filtering process, and video frames b5 - b6 correspond to another Kalman filtering process. Moreover, the other Kalman filtering process corresponding to video frames b5 - b6 starts with the occlusion data of video frame b1 as the starting data.

[0068] Optionally, the obtaining of the reference occlusion data includes:

[0069] Processing the target video according to a target detection algorithm to obtain third occlusion data corresponding to the starting video frame and fourth occlusion data corresponding to the ending video frame, where the starting video frame is the first video frame among the multiple consecutive video frames, and the ending video frame is the last video frame among the multiple consecutive video frames;

[0070] Performing a mean process on the third occlusion data and the fourth occlusion data to obtain the reference occlusion data.

[0071] As above, for a target time period, applying a target detection algorithm with a higher confidence level (compared with the target tracking algorithm) to perform target detection on the first video frame and the last video frame of the target time period to respectively obtain the third occlusion data and the fourth occlusion data, and performing a mean process on the third occlusion data and the fourth occlusion data, so as to improve the accuracy of the obtained reference occlusion data while maintaining the system overhead and processing time at a relatively low level, thereby ensuring the accuracy of the subsequent IoU judgment and improving the correction effect of the occlusion box.

[0072] Among them, the third occlusion data is set to correspond to the first video frame of the target time period, and the fourth occlusion data is set to correspond to the last video frame of the target time period to maximize the time difference between the third occlusion data and the fourth occlusion data, and further improve the accuracy of the obtained reference occlusion data.

[0073] It should be noted that in practical applications, the frame extraction density for frame extraction processing of the target time period can be 2 frames per second, 3 frames per second, etc. At this time, the reference occlusion data can be determined by performing a mean process on the occlusion data corresponding to 2 video frames or 3 video frames extracted within 1 second.

[0074] Optionally, correcting the position of the occlusion box in the video frame at time T+1 according to the correction information includes:

[0075] Correcting at least one of the center point coordinates, the occlusion box width, and the occlusion box height of the occlusion box in the video frame at time T+1 according to the correction information.

[0076] In this embodiment, at least one of the center point coordinates, the occlusion box width, and the occlusion box height of the occlusion box in the video frame at time T+1 is adjusted based on the correction information to complete the correction operation of correcting the occlusion box in the video frame at time T+1 from the second occlusion position to the first occlusion position.

[0077] As Figure 2 shown, Figure 2 the outermost solid box in Figure 2 can be the aforementioned video frame, Figure 2 the dashed box in Figure 2 can be the aforementioned occlusion box, Figure 2 the point (x, y) in

[0078] can be the center point coordinates of the occlusion box, Figure 3 , Figure 3 is a structural diagram of an occlusion box correction device 300 provided by an embodiment of the present application. As Figure 3 shown, the occlusion box correction device 300 includes:

[0079] A data acquisition module 301, configured to acquire first occlusion data and second occlusion data of a target video, where the first occlusion data is used to represent the position of the occlusion box in the video frame at time T of the target video, and the second occlusion data is used to represent the position of the occlusion box in the video frame at time T+1 of the target video, and T is a positive integer;

[0080] A smoothing processing module 302, configured to process the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information;

[0081] A position correction module 303, configured to correct the position of the occlusion box in the video frame at time T+1 of the target video according to the correction information.

[0082] Optionally, the smoothing processing module 302 includes;

[0083] An information determination sub-module, configured to determine status information according to the first occlusion data and the second occlusion data, where the status information includes the position of the occlusion box in the video frame at time T+1 of the target video, and the position offset between the position of the occlusion box in the video frame at time T of the target video and the position of the occlusion box in the video frame at time T+1 of the target video;

[0084] A smoothing processing sub-module, configured to perform smoothing processing on the status information according to a Kalman filter to obtain corrected information.

[0085] Optionally, the smoothing processing sub-module includes:

[0086] A matrix acquisition unit, configured to acquire a target state transition matrix and a target state estimation covariance matrix corresponding to time T+1 of the target video;

[0087] A prior estimation unit, configured to perform prior estimation according to the target state transition matrix and the status information to obtain prediction information;

[0088] An information determination unit, configured to obtain corrected information according to the prediction information and residual information, where the residual information is obtained according to the Kalman gain information corresponding to time T+1 of the target video, and the Kalman gain information corresponding to time T+1 of the target video is obtained according to the target state estimation covariance matrix.

[0089] Optionally, the status information further includes the first image feature corresponding to the object to be occluded in the video frame at time T+1 of the target video, and the feature difference between the second image feature corresponding to the object to be occluded in the video frame at time T of the target video and the first image feature.

[0090] Optionally, the smoothing processing module 302 includes:

[0091] A reference data acquisition sub-module, configured to acquire reference occlusion data, where the reference occlusion data is determined based on the positions of the occlusion boxes in a plurality of consecutive video frames included in the target video, and the plurality of consecutive video frames include the video frame at time T and the video frame at time T+1;

[0092] A parameter calculation sub-module, configured to acquire an intersection over union parameter according to the reference occlusion data and the second occlusion data, where the intersection over union parameter is used to characterize the overlap degree between the occlusion boxes in the plurality of consecutive video frames and the occlusion box in the video frame at time T+1 of the target video;

[0093] A threshold judgment sub-module, configured to process the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information when the intersection-over-union parameter is greater than or equal to an overlap threshold.

[0094] Optionally, the reference data acquisition sub-module includes:

[0095] A data acquisition unit, configured to process the target video according to a target detection algorithm to obtain third occlusion data corresponding to a starting video frame and fourth occlusion data corresponding to an ending video frame, where the starting video frame is the first video frame among the multiple consecutive video frames, and the ending video frame is the last video frame among the multiple consecutive video frames;

[0096] An average processing unit, configured to perform average processing on the third occlusion data and the fourth occlusion data to obtain reference occlusion data.

[0097] Optionally, the position correction module 303 is specifically configured to:

[0098] Correct at least one of the center point coordinates, the width of the occlusion box, and the height of the occlusion box in the video frame at the T+1 moment according to the correction information.

[0099] The occlusion box correction device 300 provided in the embodiments of the present application can implement each process in the above method embodiments. To avoid repetition, it will not be elaborated here.

[0100] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of an electronic device provided in the embodiments of the present application. As Figure 4 shown, the electronic device may include a processor 401, a memory 402, and a program 4021 stored on the memory 402 and executable on the processor 401.

[0101] When the program 4021 is executed by the processor 401, it can implement Figure 1 any step in the corresponding method embodiment and achieve the same beneficial effects. Details are not described herein again.

[0102] Those of ordinary skill in the art can understand that all or part of the steps for implementing the method in the above embodiments can be completed by hardware related to program instructions, and the program can be stored in a readable medium.

[0103] The embodiments of the present application further provide a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can implement any step in the above Figure 1 corresponding method embodiment and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0104] The computer-readable storage medium of the embodiments of the present application may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0105] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0106] The program code contained on the storage medium may be transmitted by any appropriate medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above.

[0107] The computer program code for performing the operations of the present application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or terminal. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., connected through the Internet using an Internet service provider).

[0108] The above is the preferred implementation mode of the embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle described in the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for correcting an occlusion box, characterized in that The method includes: Obtaining first occlusion data and second occlusion data of a target video, where the first occlusion data is used to characterize the positions of occlusion frames in the video frame at time T of the target video, and the second occlusion data is used to characterize the positions of occlusion frames in the video frame at time T+1 of the target video, and T is a positive integer; Processing the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information; Correcting the positions of the occlusion frames in the video frame at time T+1 according to the correction information; The processing the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information includes: Obtaining reference occlusion data, where the reference occlusion data is determined based on the positions of occlusion frames in a plurality of consecutive video frames included in the target video, and the plurality of consecutive video frames include the video frame at time T and the video frame at time T+1; Obtaining an intersection over union parameter according to the reference occlusion data and the second occlusion data, where the intersection over union parameter is used to characterize the overlap degree between the occlusion frames in the plurality of consecutive video frames and the occlusion frames in the video frame at time T+1; When the intersection over union parameter is greater than or equal to an overlap threshold, processing the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information; The obtaining the reference occlusion data includes: Processing the target video according to an object detection algorithm to obtain third occlusion data corresponding to a starting video frame and fourth occlusion data corresponding to an ending video frame, where the starting video frame is the first video frame in the plurality of consecutive video frames, and the ending video frame is the last video frame in the plurality of consecutive video frames; Performing a mean process on the third occlusion data and the fourth occlusion data to obtain reference occlusion data.

2. The method according to claim 1, wherein The processing the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information includes: Determining state information according to the first occlusion data and the second occlusion data, where the state information includes the positions of the occlusion frames in the video frame at time T+1 of the target video, and the position offset between the positions of the occlusion frames in the video frame at time T of the target video and the positions of the occlusion frames in the video frame at time T+1 of the target video; Performing a smoothing process on the state information according to a Kalman filter to obtain correction information.

3. The method according to claim 2, wherein The performing a smoothing process on the state information according to a Kalman filter to obtain correction information includes: Obtaining a target state transition matrix and a target state estimation covariance matrix corresponding to time T+1 of the target video; Performing a prior estimation according to the target state transition matrix and the state information to obtain prediction information; Obtaining correction information according to the prediction information and residual information, where the residual information is obtained according to the Kalman gain information corresponding to time T+1 of the target video, and the Kalman gain information corresponding to time T+1 of the target video is obtained according to the target state estimation covariance matrix.

4. The method according to claim 2, wherein The state information further includes the first image features corresponding to the object to be occluded in the video frame at time T+1 of the target video, and the feature difference between the second image features corresponding to the object to be occluded in the video frame at time T of the target video and the first image features.

5. The method according to claim 1, wherein The correcting the position of the occlusion box in the video frame at time T+1 according to the correction information includes: Correcting at least one of the center point coordinates, the width, and the height of the occlusion box in the video frame at time T+1 according to the correction information.

6. An occlusion box correction device, characterized in that, including: A data acquisition module, configured to acquire first occlusion data and second occlusion data of a target video, where the first occlusion data is used to represent the position of the occlusion box in the video frame at time T of the target video, and the second occlusion data is used to represent the position of the occlusion box in the video frame at time T+1 of the target video, and T is a positive integer; A smoothing processing module, configured to process the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information; A position correction module, configured to correct the position of the occlusion box in the video frame at time T+1 of the target video according to the correction information; The smoothing processing module includes: A reference data acquisition sub-module, configured to acquire reference occlusion data, where the reference occlusion data is determined based on the positions of the occlusion boxes in a plurality of consecutive video frames included in the target video, and the plurality of consecutive video frames include the video frame at time T and the video frame at time T+1; A parameter calculation sub-module, configured to obtain an intersection over union parameter according to the reference occlusion data and the second occlusion data, where the intersection over union parameter is used to represent the overlap degree between the occlusion boxes in the plurality of consecutive video frames and the occlusion box in the video frame at time T+1 of the target video; A threshold judgment sub-module, configured to process the first occlusion data and the second occlusion data according to a Kalman filter to obtain correction information when the intersection over union parameter is greater than or equal to an overlap threshold; The reference data acquisition sub-module includes: A data acquisition unit, configured to process the target video according to a target detection algorithm to obtain third occlusion data corresponding to a starting video frame and fourth occlusion data corresponding to an ending video frame, where the starting video frame is the first video frame in the plurality of consecutive video frames, and the ending video frame is the last video frame in the plurality of consecutive video frames; An average processing unit, configured to perform average processing on the third occlusion data and the fourth occlusion data to obtain reference occlusion data.

7. An electronic device, characterized in that, including a processor, a memory, and a computer program stored on the memory and executable on the processor, where when the computer program is executed by the processor, the steps of the occlusion box correction method according to any one of claims 1 to 5 are implemented.

8. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the steps of the occlusion box correction method according to any one of claims 1 to 5 are implemented.

Images