Video processing method and device, medium and computer equipment

Abstract

The invention discloses a video processing method and device, a medium and computer equipment. The method comprises the following steps: detecting an operation object in a current video frame collected from an operation environment; and if the detection fails, predicting a first position of the operation object in the current video frame based on a first tracking result of the operation object, and when the distance between the first position and a second position of an operator of the operation object in the current video frame is smaller than a preset distance threshold, determining that the operation object is in the current video frame. Adding the first position to the first tracking result; the first tracking result is obtained by tracking the operation object in at least one historical video frame collected from the operation environment; and if the detection is successful, carrying out quality detection on a pixel area where the operation object is located in the current video frame, and determining whether the current video frame is used for carrying out quality monitoring on the operation object or not based on a quality detection result.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video processing method and apparatus, medium and computer equipment. Background Art

[0002] When an operator operates an operation object, it is often necessary to collect video frames of the operation object and monitor the quality of the operation object based on the video frames of the operation object. In the related art, the operation object is easily blocked. After the operation object is blocked, if the operation object reappears in the video frame, the operation object will be determined as a new operation object, so that the video frame collected after the operation object reappears cannot be used to monitor the quality of the operation object. Summary of the invention

[0003] In a first aspect, an embodiment of the present disclosure provides a video processing method, the method comprising: detecting an operation object in a current video frame collected from an operation environment; if the detection fails, predicting a first position of the operation object in the current video frame based on a first tracking result of the operation object, and adding the first position to the first tracking result when the distance between the first position and a second position of an operator of the operation object in the current video frame is less than a preset distance threshold; the first tracking result is obtained by tracking the operation object in at least one historical video frame collected from the operation environment; if the detection is successful, performing quality detection on a pixel area in the current video frame where the operation object is located, and determining whether to use the current video frame for quality monitoring of the operation object based on the quality detection result.

[0004] In some embodiments, the first tracking result includes the position of the operation object in each historical video frame of the at least one historical video frame; and predicting the first position of the operation object in the current video frame based on the first tracking result of the operation object includes: obtaining a target historical video frame in the at least one historical video frame; and predicting the first position of the operation object in the current video frame based on the position of the operation object in the target historical video frame.

[0005] In some embodiments, predicting the first position of the operation object in the current video frame based on the position of the operation object in the target historical video frame includes: detecting the operator in the current video frame to obtain the position of the operator in the current video frame; determining the offset between the position of the operator in the current video frame and the position of the operator in the target historical video frame; and predicting the first position of the operation object in the current video frame based on the offset and the position of the operation object in the target historical video frame.

[0006] In some embodiments, the first position of the operation object in the current video frame is predicted based on the offset and the position of the operation object in the target historical video frame, including: if the offset is less than a preset offset threshold, determining the first position of the operation object in the current video frame based on the position of the operation object in the target historical video frame; if the offset is greater than or equal to the preset offset threshold, determining the first position of the operation object in the current video frame based on the sum of the position of the operation object in the target historical video frame and the offset.

[0007] In some embodiments, the first tracking result includes the position of the operation object in each historical video frame of the at least one historical video frame, and the operation object corresponding to each position in the first tracking result is assigned the same tracking ID; if the detection fails, adding the first position to the first tracking result includes: extending the tracking ID assigned to the operation object in the at least one historical video frame to the current video frame, so as to add the first position to the first tracking result based on the tracking ID; if the detection is successful, the method also includes: extending the tracking ID assigned to the operation object in the at least one historical video frame to the current video frame, so as to add the position of the operation object detected from the current video frame to the first tracking result based on the tracking ID.

[0008] In some embodiments, the quality detection of the pixel area where the operation object is located in the current video frame includes: obtaining a first quality detection result of the pixel area based on the clarity and completeness of the operation object in the pixel area; obtaining a second quality detection result of the pixel area based on a pre-trained quality detection model; and obtaining a quality detection result of the current video frame based on the first quality detection result and the second quality detection result.

[0009] In some embodiments, determining whether to use the current video frame for quality monitoring of the operation object based on the quality detection result includes: comparing the quality detection result of the video frame in the cache with the quality detection result of the current video frame, the video frame in the cache being any historical video frame containing the operation object; if the quality detection result of the current video frame is better than the quality detection result of the video frame in the cache, replacing the video frame in the cache with the current video frame; wherein the video frame in the cache is used for quality monitoring of the operation object.

[0010] In some embodiments, replacing the video frames in the cache with the current video frame includes: if the quality detection result of the current video frame is better than the preset quality detection result, replacing the video frames in the cache with the current video frame, and no longer performing quality detection on subsequent video frames containing the operation object; if the quality detection result of the current video frame is better than the quality detection result of the historical video frame, but not better than the preset quality detection result, replacing the video frame in the cache with the current video frame.

[0011] In some embodiments, the method also includes: if a preset condition is met, deleting the first tracking result and outputting the video frame in the cache to a business database; the preset condition includes at least one of the following: the duration of the failure of the operation object detection reaches a first preset duration threshold; the duration of the operation object being obscured by other operation objects reaches a second preset duration threshold; the distance between the position of the operation object in the current video frame and the position of the operation object in the previous video frame of the current video frame is greater than a preset distance threshold; the position of the operator exceeds a preset operation area.

[0012] In a second aspect, an embodiment of the present disclosure provides a video processing device, comprising: a detection module, configured to detect an operation object in a current video frame collected from an operation environment; an adding module, configured to predict a first position of the operation object in the current video frame based on a first tracking result of the operation object if the detection fails, and add the first position to the first tracking result if the distance between the first position and a second position of an operator of the operation object in the current video frame is less than a preset distance threshold; the first tracking result is obtained by tracking the operation object in at least one historical video frame collected from the operation environment; and a determination module, configured to perform quality detection on a pixel area in the current video frame where the operation object is located if the detection is successful, and determine whether to use the current video frame for quality monitoring of the operation object based on the quality detection result.

[0013] In some embodiments, the first tracking result includes the position of the operation object in each historical video frame of the at least one historical video frame; the adding module is specifically used to: obtain the target historical video frame in the at least one historical video frame; and predict the first position of the operation object in the current video frame based on the position of the operation object in the target historical video frame.

[0014] In some embodiments, the adding module is specifically used to: detect the operator in the current video frame to obtain the position of the operator in the current video frame; determine the offset between the position of the operator in the current video frame and the position of the operator in the target historical video frame; and predict the first position of the operation object in the current video frame based on the offset and the position of the operation object in the target historical video frame.

[0015] In some embodiments, the adding module is specifically used to: if the offset is less than a preset offset threshold, determine the first position of the operation object in the current video frame based on the position of the operation object in the target historical video frame; if the offset is greater than or equal to the preset offset threshold, determine the first position of the operation object in the current video frame based on the sum of the position of the operation object in the target historical video frame and the offset.

[0016] In some embodiments, the first tracking result includes the position of the operation object in each historical video frame of the at least one historical video frame, and the operation object corresponding to each position in the first tracking result is assigned the same tracking ID; the adding module is specifically used to: extend the tracking ID assigned to the operation object in the at least one historical video frame to the current video frame, so as to add the first position to the first tracking result based on the tracking ID; if the detection is successful, the device also includes: a continuation module, used to extend the tracking ID assigned to the operation object in the at least one historical video frame to the current video frame, so as to add the position of the operation object detected from the current video frame to the first tracking result based on the tracking ID.

[0017] In some embodiments, the determination module is specifically used to: obtain a first quality detection result of the pixel area based on the clarity and completeness of the operation object in the pixel area; obtain a second quality detection result of the pixel area based on a pre-trained quality detection model; and obtain a quality detection result of the current video frame based on the first quality detection result and the second quality detection result.

[0018] In some embodiments, the determination module is specifically used to: compare the quality detection result of the video frame in the cache with the quality detection result of the current video frame, the video frame in the cache is any historical video frame containing the operation object; if the quality detection result of the current video frame is better than the quality detection result of the video frame in the cache, replace the video frame in the cache with the current video frame; wherein the video frame in the cache is used to monitor the quality of the operation object.

[0019] In some embodiments, the determination module is specifically used to: if the quality detection result of the current video frame is better than the preset quality detection result, replace the video frame in the cache with the current video frame, and no longer perform quality detection on subsequent video frames containing the operation object; if the quality detection result of the current video frame is better than the quality detection result of the historical video frame, but not better than the preset quality detection result, replace the video frame in the cache with the current video frame.

[0020] In some embodiments, the device also includes: a deletion module, which is used to delete the first tracking result and output the video frame in the cache to a business database if a preset condition is met; the preset condition includes at least one of the following: the duration of the failure of the operation object detection reaches a first preset duration threshold; the duration of the operation object being blocked by other operation objects reaches a second preset duration threshold; the distance between the position of the operation object in the current video frame and the position of the operation object in the previous video frame of the current video frame is greater than a preset distance threshold; the position of the operator exceeds a preset operation area.

[0021] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present disclosure.

[0022] In a fourth aspect, an embodiment of the present disclosure provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present disclosure when executing the program.

[0023] In the embodiment of the present disclosure, when the detection of the operating object in the current video frame fails, the first position of the operating object in the current video frame can be predicted, and when the distance between the predicted first position and the second position of the operator of the operating object in the current video frame is less than a preset distance threshold, the first position is added to the first tracking result of the operating object. Therefore, even if the operating object is blocked in the current video frame, the position of the operating object in the current video frame can be continued to the first tracking result of the operating object, so that when the operating object reappears, the operating object will not be misjudged as a new operating object. In this way, the video frame collected after the operating object is blocked can still be used to monitor the quality of the operating object. When the detection of the operating object in the current video frame is successful, the pixel area where the operating object is located in the current video frame can be directly quality-checked to determine whether the current video frame is used to monitor the quality of the operating object.

[0024] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into the specification and constitute a part of the present disclosure. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0026] Figure 1 It is a schematic diagram of an application scenario of an embodiment of the present disclosure.

[0027] Figure 2 is a flowchart of a video processing method according to an embodiment of the present disclosure.

[0028] Figure 3 It is a schematic diagram of a process of determining a target video frame according to an embodiment of the present disclosure.

[0029] Figure 4 It is an overall flow chart of an embodiment of the present disclosure.

[0030] Figure 5 is a block diagram of a video processing device according to an embodiment of the present disclosure.

[0031] Figure 6 is a schematic diagram of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0033] The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of.

[0034] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings.

[0036] When the operator is operating the object, the object needs to be quality-monitored. For example, in food processing scenarios, the food quality needs to be monitored to ensure the safety of the food; in material processing scenarios, the material production process needs to be quality-monitored to ensure that the material is produced according to the correct process flow; in experimental scenarios, the experimental process needs to be quality-monitored to ensure the effectiveness of the experiment. In addition to the above-mentioned scenarios, there are other scenarios that require quality monitoring, which are not listed here one by one.

[0037] In the related art, quality monitoring is generally performed based on the video frame of the operation object. However, the operation object may be blocked and reappear after the blockage. The reappearing operation object may be misjudged as a new operation object. In this way, the image captured after the operation object reappears may not be selected as the image of the operation object, so that the quality monitoring of the operation object cannot be performed based on the image captured after the operation object reappears.

[0038] The operation objects of the embodiment of the present disclosure may include but are not limited to food (e.g., steamed buns, pizza Z, fruits, drinks, etc.), handicrafts, experimental equipment, etc. For ease of understanding and explanation, the following takes the operation object being pizza Z as an example to illustrate the scheme of the embodiment of the present disclosure. Accordingly, the operator may be referred to as pizza Z chef U (referred to as chef U).

[0039] Figure 1 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure, wherein the application scenario includes an operating environment, which includes an operating table D, a pizza Z placed on the operating table D, a chef U standing next to the operating table D, and a camera 11 arranged on the operating table D. The chef U can make pizza Z on the operating table D according to a certain production process. For example, the chef U can first prepare the dough, make the dough into a specified shape (for example, a circle), spread the stuffing on the dough of the specified shape, and then put the dough with the stuffing into the oven for baking. After baking, the chef U can take out the pizza Z and place it on the operating table D for slicing. In the above process, the video frame of the operating table D can be collected by the camera 11, and the video frame can be sent to the processing device 12 for video processing. The processing device 12 can select at least one target video frame from the collected video frames for quality monitoring of the pizza Z production process. In the process of the chef U making pizza Z, the pizza Z may be temporarily blocked by the chef U, so that the pizza Z cannot be detected in one or more video frames, and in other video frames after this one or more video frames, the pizza Z can be detected again. The re-detected pizza Z will be considered as a different pizza from the pizza Z detected before the occlusion, resulting in the quality monitoring of pizza Z being performed only based on the video frames captured before the occlusion. If the quality of the video frames captured before the occlusion is low, the quality monitoring effect of pizza Z is likely to be poor.

[0040] Understandably, Figure 1 The application scenarios shown are only exemplary and are not intended to limit the present disclosure. For example, in other application scenarios, the operator and the operation object may be of other types, and the operation environment may only include the operator and the operation object, but not the operation console D.

[0041] To solve the above problems, the present disclosure provides a video processing method, see Figure 2 , the method comprising:

[0042] Step S1: detecting an operation object in a current video frame collected from an operation environment;

[0043] Step S2: if the detection fails, predicting a first position of the operation object in the current video frame based on the first tracking result of the operation object, and adding the first position to the first tracking result when the distance between the first position and a second position of the operator of the operation object in the current video frame is less than a preset distance threshold; the first tracking result is obtained by tracking the operation object in at least one historical video frame collected from the operation environment;

[0044] Step S3: If the detection is successful, perform quality detection on the pixel area where the operation object is located in the current video frame, and determine whether to use the current video frame for quality monitoring of the operation object based on the quality detection result.

[0045] Combine the following Figure 1 The application scenario shown is used to illustrate the specific details of each step.

[0046] In step S1, the camera 11 can capture images of the operating environment at a certain frame rate to obtain at least one video frame. Among them, the video frame captured in real time is the current video frame, and the video frame captured before the current video frame is called a historical video frame. For example, assuming that the video frame captured in real time is the i+1th video frame, then the i+1th video frame is the current video frame, and the 1st video frame to the i-th video frame are all historical video frames, and i is a positive integer. The detection result may be a successful detection or a failed detection. For example, when the pizza Z in the current video frame is not blocked or the blocked ratio is less than a preset ratio, the detection result is a successful detection; and when the blocked ratio of the pizza Z in the current video frame is greater than or equal to the preset ratio, the detection result is a failed detection.

[0047] In step S2, if the detection fails, the first position of the pizza Z in the current video frame can be predicted based on the first tracking result of the pizza Z. The first tracking result is obtained by tracking the pizza Z in at least one historical video frame collected from the operating environment. The first tracking result may include the position of the pizza Z in each historical video frame in at least one historical video frame. The position here can be represented by the coordinates of a certain point on the pizza Z (for example, the center point), or by the detection frame of the pizza Z.

[0048] Assume that the first tracking result is recorded as {P1, P2, ..., PN}, where Pi corresponds to the i-th video frame, is the position of pizza Z in the i-th video frame, and each position in the first tracking result belongs to the same pizza Z. The following takes the i-th video frame as an example to illustrate the method of determining the first position of pizza Z in the current video frame.

[0049] Since the position of pizza Z in the i-th video frame and its historical video frames satisfy the constraints of physical space (i.e., the position of pizza Z will not change suddenly), in this embodiment, the position of pizza Z in the i-th video frame can be predicted based on the position of pizza Z in the historical video frames. Specifically, the target historical video frame can be determined from at least one historical video frame, and the first position of pizza Z in the i-th video frame can be predicted based on the position of pizza Z in the target historical video frame. The number of target historical video frames can be greater than or equal to 1. For example, the previous video frame of the i-th video frame (i.e., the i-1-th video frame) can be determined as the target historical video frame, or multiple video frames before the i-th video frame (for example, the j-th video frame to the i-1-th video frame, j<i-1, and j is a positive integer) can be determined as target historical video frames.

[0050] When the number of target historical video frames is 1, the position of pizza Z in the target historical video frame can be determined as the first position of pizza Z in the i-th video frame. When the number of target historical video frames is greater than 1, the positions of the same pizza Z in multiple target historical video frames can be weighted, and the weighted position can be determined as the first position of pizza Z in the i-th video frame.

[0051] In some embodiments, the chef U in the i-th video frame can also be detected to obtain the position of the chef U in the i-th video frame, and the offset between the position of the chef U in the i-th video frame and the position of the chef U in the target historical video frame can be determined. Based on the offset and the position of the pizza Z in the target historical video frame, the first position of the pizza Z in the i-th video frame can be predicted. The chef U here is the chef who makes the pizza Z. When the chef U makes the pizza Z, the distance between the chef U and the pizza Z is usually small. Therefore, the first position of the pizza Z in the i-th video frame can be predicted based on the position of the chef U in the i-th video frame. Since the volume of the chef U is larger than the volume of the pizza Z, the probability of the chef U being lost in the video frame is relatively small, so the first position of the pizza Z can be accurately predicted with the help of the position of the chef U.

[0052] Specifically, if the offset is less than a preset offset threshold, the first position of pizza Z in the current video frame can be determined based on the position of pizza Z in the target historical video frame. If the offset is greater than or equal to the preset offset threshold, the first position of pizza Z in the current video frame can be determined based on the sum of the position of pizza Z in the target historical video frame and the offset.

[0053] In some embodiments, the first position includes a horizontal position and a vertical position. In order to improve the accuracy of the first position, the offset may include a first offset in the horizontal direction and a second offset in the vertical direction. If the first offset is less than a preset offset threshold, the horizontal position of pizza Z in the current video frame may be determined based on the horizontal position of pizza Z in the target historical video frame, otherwise, the horizontal position of pizza Z in the current video frame may be determined based on the sum of the horizontal position of pizza Z in the target historical video frame and the first offset. If the second offset is less than a preset offset threshold, the vertical position of pizza Z in the current video frame may be determined based on the vertical position of pizza Z in the target historical video frame, otherwise, the vertical position of pizza Z in the current video frame may be determined based on the sum of the vertical position of pizza Z in the target historical video frame and the second offset.

[0054] When the number of target video frames is equal to 1, if the offset is less than the preset offset threshold, the position of pizza Z in the target historical video frame can be determined as the first position of pizza Z in the current video frame. If the offset is greater than or equal to the preset offset threshold, the sum of the position of pizza Z in the target historical video frame and the offset can be determined as the first position of pizza Z in the current video frame. When the number of target video frames is greater than 1, the positions of pizza Z in multiple frames of target historical video frames can be weighted to obtain a weighted position. If the offset is less than the preset offset threshold, the weighted position can be determined as the first position of pizza Z in the current video frame. If the offset is greater than or equal to the preset offset threshold, the sum of the weighted position and the offset can be determined as the first position of pizza Z in the current video frame.

[0055] Since the distance between chef U and pizza Z is small during the process of chef U making pizza Z, the above embodiment predicts the offset of the position of pizza Z in the current video frame and the target historical video frame based on the offset of the position of chef U in the current video frame and the target historical video frame, and can accurately determine the offset of the position of pizza Z in different video frames, thereby accurately predicting the first position of pizza Z in the current video frame.

[0056] In the above embodiment, the position of the chef U in the i-th video frame can be obtained by detecting the i-th video frame, or can be obtained based on the second tracking result of the chef U. The second tracking result is obtained by tracking the chef U in at least one historical video frame collected from the operating environment, and the at least one historical video frame used to obtain the first tracking result and the at least one historical video frame used to obtain the second tracking result can be the same video frame. The second tracking result may include the position of the chef U in each historical video frame in the at least one historical video frame. Assume that the second tracking result is recorded as {Q1, Q2, ..., QN}, where Qi corresponds to the i-th video frame, is the position of the chef U in the i-th video frame, and each position in the second tracking result belongs to the same chef U.

[0057] For any historical video frame, the distance between the position of pizza Z in the historical video frame and the position of chef U in the historical video frame is less than the preset distance threshold. For example, in the first video frame, the distance between the position P1 of pizza Z and the position Q1 of chef U is less than the preset distance threshold; in the second video frame, the distance between the position P2 of pizza Z and the position Q2 of chef U is less than the preset distance threshold. Since the first tracking result represents the position sequence of the same pizza Z in each historical video frame, and the second tracking result represents the position sequence of the same chef U in each historical video frame, the position of pizza Z in each video frame can be associated with the position of the same chef U in the corresponding video frame through the preset distance threshold. When pizza Z is not successfully detected from the i-th video frame, but chef U can be detected, since the position associated with the predicted first position and the position in the first tracking result belongs to the same chef U, the predicted first position and the position in the first tracking result can be determined as the position of the same pizza Z and added to the first tracking result of the same pizza Z, so that the first tracking result of the occluded pizza Z is kept from being interrupted and lost through the position of the same chef U.

[0058] In some embodiments, when the first video frame is captured, the position of pizza Z and the position of chef U in the first video frame can be detected, and the distance between pizza Z and chef U in the first video frame can be determined based on the position of pizza Z and the position of chef U in the first video frame. If the distance is less than a preset distance threshold, pizza Z detected from the first video frame is associated with chef U. Thereafter, each time a video frame is detected, the position of chef U detected from the video frame is added to the second tracking result of chef U detected from the first video frame. Furthermore, if pizza Z is detected from the video frame, the position of pizza Z detected from the video frame is added to the first tracking result of pizza Z detected from the first video frame. If pizza Z is not detected from the video frame, the first position of pizza Z in the video frame is predicted based on the first tracking result of pizza Z. If the distance between the predicted first position and the position of chef U in the video frame is less than the preset distance threshold, the predicted first position is added to the first tracking result of pizza Z detected from the first video frame. If pizza Z is detected again in a certain video frame after the video frame in which pizza Z is not detected, the position of the re-detected pizza Z can still be added to the first tracking result of pizza Z. In this way, even if pizza Z has been blocked, a continuous and complete position sequence of pizza Z can be obtained, and the re-detected pizza Z will not be misjudged as a new pizza.

[0059] In some embodiments, a tracking ID may be assigned to the detected pizza Z, and the tracking ID is used to uniquely identify the pizza Z. Therefore, different pizzas Z may be assigned different tracking IDs. Since each position in the first tracking result represents the position of the same pizza Z, the pizza Z corresponding to each position in the first tracking result may be assigned the same tracking ID. By assigning a tracking ID, the positions of the same pizza Z in each video frame may be associated. Therefore, when the first position is added to the first tracking result, the tracking ID assigned to the pizza Z in at least one historical video frame may be continued to the current video frame to add the first position to the first tracking result based on the tracking ID.

[0060] For example, assuming that a pizza Z is detected for the first time in the first video frame, a tracking ID (assuming it is ID1) can be assigned to the pizza Z, and the position of the first detected pizza Z is added to the position sequence of the pizza Z with the tracking ID ID1 (i.e., the first tracking result). In the second video frame, if the same pizza Z is detected, the tracking ID assigned to the pizza Z in the second video frame is determined to be ID1, and the position of the pizza Z in the second video frame is added to the position sequence of the pizza Z with the tracking ID ID1. And so on. Assuming that the current video frame is the i-th video frame, and the detection of pizza Z in the current video frame fails, the first position of pizza Z in the current video frame is predicted. If the distance between the first position and the second position of chef U in the current video frame is less than the preset distance threshold, the tracking ID assigned to pizza Z in the previous video frame is continued to the i-th video frame, that is, the tracking ID assigned to pizza Z in the i-th video frame is also ID1, so that the first position can also be added to the position sequence of pizza Z with the tracking ID ID1.

[0061] Similarly, a tracking ID may be assigned to the detected chef U, and the tracking ID is used to uniquely identify the chef U. Different chefs U may be assigned different tracking IDs. The chef U corresponding to each position in the second tracking result will be assigned the same tracking ID. By assigning tracking IDs, the positions of the same chef U in each video frame may be associated. When a chef U is detected for the first time, a tracking ID may be assigned to the chef U, and the position of the chef U detected for the first time may be added to the position sequence of the chef U with the tracking ID. In subsequent video frames, if the same chef U is detected, the same tracking ID may continue to be assigned to the chef U in subsequent video frames, and the position of the chef U in subsequent video frames may be added to the position sequence of the chef U with the same tracking ID.

[0062] In step S3, if the detection is successful, the quality detection can be performed on the pixel area where the pizza Z is located in the current video frame to obtain the quality detection result of the current video frame. The quality detection result can be represented by a quality score, which can be a natural number from 0 to 100 or 0 to 5 or other score ranges. Alternatively, the quality detection result can also be represented by characters, for example, using a character set such as {"excellent", "good", "medium", "poor"} or {"SSS", "SS", "S"} to represent the quality detection results from good to bad in sequence.

[0063] The quality detection result of the current video frame is used to characterize the quality of the pixel area where pizza Z is located in the current video frame. The first quality detection result of the pixel area where pizza Z is located can be obtained based on the clarity and completeness of pizza Z in the pixel area where pizza Z is located. Generally, the higher the clarity and completeness of pizza Z in the pixel area of ​​the current video frame, the higher the quality of the pixel area where pizza Z is located in the current video frame.

[0064] Alternatively, a second quality detection result of the pixel area where pizza Z is located may also be obtained by using a pre-trained quality detection model. The quality detection model may be trained using a sample image including pizza Z and a label corresponding to the sample image, wherein the label corresponding to the sample image is used to characterize the quality of the sample image, for example, the label corresponding to the sample image may be a quality score of the sample image.

[0065] The quality score of the current video frame may also be obtained based on the first quality detection result and the second quality detection result. For example, the first quality score and the second quality score are weighted averaged to obtain the quality score of the current video frame.

[0066] Based on the quality detection result, it can be determined whether to use the current video frame as a target video frame for quality monitoring of pizza Z. Figure 3 , the quality detection result of the video frame in the cache can be compared with the quality detection result of the current video frame. If the quality detection result of the current video frame is better than the quality detection result of the video frame in the cache, the current video frame replaces the video frame in the cache. The video frame in the cache is any historical video frame containing pizza Z. For example, the video frame in the cache can be a historical video frame with the best quality detection result, or a historical video frame detected most recently.

[0067] Taking the video frame in the cache as the historical video frame with the best quality detection result as an example, each time a quality detection is performed on a frame of video, it can be determined whether the video frame is the video frame with the best quality detection result among the detected video frames. If so, the video frame replaces the video frame in the cache. For example, the video frame in the cache is initially empty. After the quality detection is performed on the first frame of video, the first frame of video can be replaced in the cache, so that the first frame of video is currently cached in the cache. After the quality detection is performed on the second frame of video, it can be determined whether the quality detection result of the second frame of video is better than the quality detection result of the first frame of video. If so, the second frame of video is replaced in the cache, so that the second frame of video is currently cached in the cache. If not, the first frame of video is still cached in the cache, and so on. Assuming that when the quality detection is performed on the current video frame, the video frame in the cache is the jth frame of video, and the current video frame is the ith frame of video (j is less than i), it can be determined whether the quality detection result of the ith frame of video is better than the quality detection result of the jth frame of video. If yes, the i-th video frame is replaced in the cache, so that the i-th video frame is currently cached in the cache. If no, the j-th video frame is still cached. The video frame finally retained in the cache can be used to monitor the quality of pizza Z.

[0068] Furthermore, if the quality detection result of the current video frame is better than the preset quality detection result, the current video frame can replace the video frame in the cache, and the quality monitoring of subsequent video frames containing pizza Z is no longer performed. Taking the quality detection result represented by a quality score as an example, if the quality score is greater than a preset score threshold (for example, 80 points), the current video frame can directly replace the video frame in the cache, and the quality detection of subsequent video frames containing pizza Z is no longer performed. In this way, after determining that the quality detection result of the current video frame is better than the preset quality detection result, the current video frame can be directly used as the target video frame to monitor the quality of pizza Z, and there is no need to perform quality detection on subsequent video frames, thereby reducing resource consumption in the quality monitoring process and improving the efficiency of quality monitoring.

[0069] If the quality detection result of the current video frame is better than the quality detection result of the historical video frame, but not better than the preset quality detection result, the current video frame can replace the video frame in the cache, and the subsequent video frame including pizza Z can be used as the current video frame, and the step of comparing the quality detection result of the video frame in the cache with the quality detection result of the current video frame can be returned until a subsequent video frame with a quality detection result better than the preset quality detection result is obtained. If a video frame with a quality detection result better than the preset quality detection result is not obtained, the video frame with the best quality detection result among the detected video frames can be used as the target video frame to monitor the quality of pizza Z.

[0070] In step S3, the tracking ID assigned to the pizza Z in at least one historical video frame may be continued to the current video frame, so that the position of the pizza Z detected in the current video frame is added to the first tracking result based on the tracking ID. In this way, the first tracking result of the pizza Z can be kept uninterrupted. After the detection of the pizza Z in the current video frame is completed, the first tracking result including the position of the pizza Z in the current video frame may be further used to detect the pizza Z in the subsequent video frames.

[0071] If the preset conditions are met, the first tracking result may be deleted and the video frames in the cache may be output to the service database. The preset conditions include at least one of the following:

[0072] Condition 1: The duration of Pizza Z detection failure reaches the preset duration threshold;

[0073] Condition 2: The duration of pizza Z being blocked by other pizzas reaches a second preset duration threshold;

[0074] Condition 3: The distance between the position of pizza Z in the current video frame and the position of pizza Z in the video frame before the current video frame is greater than a preset distance threshold;

[0075] Condition 4: Chef U's position is beyond the preset operation area.

[0076] For condition 1, assuming that the current video frame is the i-th video frame, and the duration of pizza Z detection failure reaches the preset duration threshold until the i-th video frame, it indicates that pizza Z may have been taken away. Therefore, the first tracking result can be directly deleted to reduce cache space usage.

[0077] For condition two, suppose that pizza Z1 is currently being detected, but because chef U stacks pizza Z2 on pizza Z1, pizza Z1 is obscured by pizza Z2. If the above situation exists for a long time, the first tracking result of pizza Z1 can be deleted. When performing quality detection on pizza Z1, it is necessary to obtain the target video frame of pizza Z1. If pizza Z1 can be detected from the video frame, but pizza Z1 is obscured by pizza Z2 for a long time, the pixel area where pizza Z2 is located in the video frame may be determined as the pixel area where pizza Z1 is located, resulting in the quality monitoring result of pizza Z2 being misjudged as the quality monitoring result of pizza Z1. Therefore, when condition two is met, the first tracking result of pizza Z1 can be deleted to reduce the above-mentioned misjudgment.

[0078] For condition three, if this condition is met, it means that the position of pizza Z does not meet the physical constraints, indicating that pizza Z in the current video frame may not be the same pizza as pizza Z in the previous video frame. Therefore, when condition three is met, the first tracking result of pizza Z can be deleted to reduce the confusion between two different pizzas.

[0079] For condition 4, the operation area can be the aforementioned operation table D, or other areas in the operation environment for operating the operation object. For example, when the operation area is the operation table D, chef U is generally around the operation table D when making pizza Z. When pizza Z is finished, chef U will leave the operation table D and take away pizza Z. Therefore, when the position of chef U exceeds the operation table D, it can be considered that chef U has taken pizza Z away, so the first tracking result of pizza Z can be deleted.

[0080] When any of the above conditions is met, the first tracking result of Pizza Z can be considered to be in an expired state, and the first tracking result of Pizza Z is deleted. A flag can be generated for the first tracking result to record the expiration status of the first tracking result. For example, the flag can be a 1-bit binary number, "0" indicates not expired, and "1" indicates expired. Of course, other forms of data can also be used to record the expiration status of the first tracking result.

[0081] When deleting the expired first tracking result, it can be determined whether the target video frame of pizza Z to which the first tracking result belongs has been taken out from the cache. If so, the expired first tracking result is directly deleted; if not, the target video frame is taken out from the cache and the expired first tracking result is deleted.

[0082] In some embodiments, the pizza Z in the current video frame can also be classified to obtain the category of the pizza Z in the current video frame. Different categories correspond to different operation stages of pizza Z. For example, based on whether pizza Z is sliced, pizza Z can be divided into two categories: before slicing and after slicing; or, based on whether pizza Z is topped with stuffing, pizza Z can be divided into two categories: before and after the stuffing is topped. Other categories can also be divided based on actual needs, which will not be described here. Similarly, when the operation object is other objects, corresponding categories can also be divided according to actual needs, which will not be listed here one by one.

[0083] After obtaining the category of pizza Z in the current video frame, it can be determined based on the quality detection result whether to use the current video frame for quality monitoring of pizza Z of the category. Assuming that the category of pizza Z includes a pre-slicing category and a post-slicing category, if the category of pizza Z in the current video frame is the pre-slicing category, it can be determined based on the quality detection result of the current video frame whether to use the current video frame for quality monitoring of pizza Z of the pre-slicing category.

[0084] In some embodiments, the quality detection result of the video frame in the cache can be obtained, wherein the pizza Z in the video frame in the cache and the pizza Z in the current video frame are the same pizza Z and belong to the same category. Then, the quality detection result of the video frame in the cache can be compared with the quality detection result of the current video frame. If the quality detection result of the current video frame is better than the quality detection result of the video frame in the cache, the current video frame replaces the video frame in the cache. The video frame in the cache is any historical video frame containing pizza Z, and the video frame in the cache is used to monitor the quality of pizza Z of this category.

[0085] For example, if the category of pizza Z in the current video frame is the pre-slice category, then the video frame of pizza Z in the cache including the pre-slice category can be obtained, and its quality detection result can be compared with the quality detection result of the current video frame. If the quality detection result of pizza Z in the current video frame is better than the quality detection result of the video frame in the cache, the current video frame replaces the video frame in the cache, so as to monitor the quality of pizza Z in the pre-slice category through the video frame in the cache after replacement.

[0086] In some embodiments, the operating environment includes multiple operating tables D, and different operating tables D are used to place pizza Z in different operating links. The current video frame includes the current video frame corresponding to each operating table D. For example, assuming that the number of operating tables D is 2, respectively recorded as operating table D1 and operating table D2, cameras 11 can be set above operating table D1 and operating table D2, respectively. The current video frame captured by the camera 11 set above the operating table D1 is the current video frame corresponding to the operating table D1, and the current video frame captured by the camera 11 set above the operating table D2 is the current video frame corresponding to the operating table D2.

[0087] When classifying pizza Z in the current video frame, pizza Z in the current video frame corresponding to at least one of the multiple operating stations D may be classified. Still taking operating stations D including operating stations D1 and operating stations D2 as an example, only pizza Z in the current video frame corresponding to operating station D1 may be classified, while pizza Z in the current video frame corresponding to operating station D2 may not be classified. Alternatively, pizza Z in both the current video frame corresponding to operating station D1 and the current video frame corresponding to operating station D2 may be classified.

[0088] Figure 4 The overall process of video processing in one embodiment is shown. The specific process is as follows:

[0089] First, the pizza Z and chef U are detected in the entire video frame captured by the camera 11, and the detection frames of pizza Z and chef U and the rawness and doneness category of pizza Z are extracted. Among them, the rawness and doneness category of pizza Z is used to characterize whether pizza Z is a raw pizza Z or a cooked pizza Z. Raw pizza Z is pizza Z on a raw product workbench D, that is, pizza Z before baking, and cooked pizza Z is pizza Z on a cooked product workbench D, that is, pizza Z after baking. The rawness and doneness category of pizza Z can be determined based on the camera 11 from which the video frame originates. Specifically, the pizza Z in the video frame captured by the camera 11 used to capture the image of the raw product workbench D can be determined as a raw pizza Z, and the pizza Z in the video frame captured by the camera 11 used to capture the image of the cooked product workbench D can be determined as a cooked pizza Z.

[0090] The above detection process can be implemented by a detection model (such as SSD, YOLO, etc.). The detection model can be obtained based on sample images including pizza Z and chef U and corresponding label information, wherein the label information includes location information of pizza Z, location information of chef U, and categories of pizza Z and chef U.

[0091] Then, the raw pizza Z and the cooked pizza Z are tracked to obtain the corresponding position sequence of pizza Z (i.e., the first tracking result), and the detection frame of pizza Z is associated with the detection frame of chef U, so as to predict the first position of the occluded pizza Z in the video frame based on the tracking result of chef U, thereby reducing the tracking loss of pizza Z when pizza Z is occluded. If there are multiple pizzas Z in the video frame, the detection frame area corresponding to the same pizza Z can be retrieved based on the first tracking result of pizza Z, so as to ensure that each position in the position sequence is the position of the same pizza Z.

[0092] The position sequence can also be analyzed for expiration, that is, to determine whether the position sequence is in an expired state. For cooked pizza Z, if the position sequence has not expired, the video frames of pizza Z to which the position sequence belongs can be classified to obtain video frames including sliced ​​pizza Z and unsliced ​​pizza Z. The video frames of each category of pizza Z can be analyzed for clarity and completeness as well as occlusion judgment to obtain the quality score of the video frame. The currently detected video frame with the highest quality score replaces the existing video frame in the cache. If the position sequence has expired, the video frame of pizza Z to which the position sequence belongs is taken out of the cache, and the expired position sequence is deleted. For raw pizza Z, no classification is required. If the position sequence has not expired, the video frame of pizza Z to which the position sequence belongs is quality checked, and the cache is updated according to the quality check result.

[0093] The above classification process can be implemented by a classification model (such as ResNet, EfficientNet, etc.). The classification model can be obtained based on a sample image including pizza Z and corresponding label information, wherein the label information includes location information and category information of pizza Z.

[0094] For the raw pizza Z and the two types of cooked pizza Z, it can be determined whether the video frames in the cache meet the quality requirements such as clarity and completeness. If yes, the image fetching operation is performed, the corresponding video frame is fetched from the cache as the target video frame and the quality of the pizza Z is tested. If it does not meet the requirements, the target video frame is continued to be acquired.

[0095] The solution of the above embodiment does not need to manually shoot the pizza Z image, which reduces the labor cost. When the pizza Z is blocked, the stable output of the position sequence of the pizza Z can be maintained. In addition, the pizza Z of each operation link and operation stage can be selected (the video frame with the highest quality is selected).

[0096] The present disclosure also provides a video processing device. Figure 5 , the device comprises:

[0097] A detection module 21, used to detect an operation object in a current video frame collected from an operation environment;

[0098] an adding module 22, configured to predict a first position of the operating object in the current video frame based on the first tracking result of the operating object if the detection fails, and add the first position to the first tracking result when the distance between the first position and a second position of the operator of the operating object in the current video frame is less than a preset distance threshold; the first tracking result is obtained by tracking the operating object in at least one historical video frame collected from the operating environment;

[0099] The determination module 23 is used to perform quality detection on the pixel area where the operation object is located in the current video frame if the detection is successful, and determine whether to use the current video frame for quality monitoring of the operation object based on the quality detection result.

[0100] An embodiment of the present disclosure further provides a computer device, which at least includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any of the above embodiments is implemented.

[0101] Figure 6A more specific hardware structure diagram of a computing device provided by an embodiment of the present disclosure is shown, and the device may include: a processor 31, a memory 32, an input / output interface 33, a communication interface 34, and a bus 35. The processor 31, the memory 32, the input / output interface 33, and the communication interface 34 are connected to each other in communication within the device through the bus 35.

[0102] The processor 31 can be implemented by a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided by the embodiments of the present disclosure. The processor 31 can also include a graphics card, which can be a Nvidia titan X graphics card or a 10120Ti graphics card, etc.

[0103] The memory 32 can be implemented in the form of a read-only memory (ROM), a random access memory (RAM), a static storage device, a dynamic storage device, etc. The memory 32 can store an operating system and other application programs. When the technical solution provided by the embodiment of the present disclosure is implemented by software or firmware, the relevant program code is stored in the memory 32 and called and executed by the processor 31.

[0104] The input / output interface 33 is used to connect the input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0105] The communication interface 34 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0106] The bus 35 comprises a pathway for transmitting information between the various components of the device (eg, the processor 31 , the memory 32 , the input / output interface 33 , and the communication interface 34 ).

[0107] It should be noted that, although the above device only shows a processor 31, a memory 32, an input / output interface 33, a communication interface 34, and a bus 35, in a specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0108] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0109] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0110] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the embodiments of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure or some parts of the embodiments.

[0111] The systems, devices, modules or units described in the above embodiments may be implemented by computer devices or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0112] Each embodiment in the present disclosure is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment of the present disclosure. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0113] The above is only a specific implementation of the embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present disclosure. These improvements and modifications should also be regarded as the protection scope of the embodiment of the present disclosure.

Claims

1. A video processing method, the method comprising: Detecting an operation object in a current video frame collected from an operation environment; If the detection fails, predicting a first position of the operation object in the current video frame based on the first tracking result of the operation object, and adding the first position to the first tracking result when a distance between the first position and a second position of the operator of the operation object in the current video frame is less than a preset distance threshold; The first tracking result is obtained by tracking the operation object in at least one historical video frame collected from the operation environment; If the detection is successful, a quality detection is performed on the pixel area where the operation object is located in the current video frame, and based on the quality detection result, it is determined whether the current video frame is used for quality monitoring of the operation object.

2. The method according to claim 1, wherein the first tracking result includes the position of the operation object in each historical video frame of the at least one historical video frame; and the predicting the first position of the operation object in the current video frame based on the first tracking result of the operation object comprises: Acquire a target historical video frame from the at least one historical video frame; A first position of the operation object in the current video frame is predicted based on the position of the operation object in the target historical video frame.

3. The method according to claim 2, wherein predicting the first position of the operation object in the current video frame based on the position of the operation object in the target historical video frame comprises: Detecting the operator in the current video frame to obtain a position of the operator in the current video frame; Determining an offset between a position of the operator in the current video frame and a position of the operator in the target historical video frame; Based on the offset and the position of the operation object in the target historical video frame, a first position of the operation object in the current video frame is predicted.

4. The method according to claim 3, wherein the predicting the first position of the operation object in the current video frame based on the offset and the position of the operation object in the target historical video frame comprises: If the offset is less than a preset offset threshold, determining a first position of the operation object in the current video frame based on the position of the operation object in the target historical video frame; If the offset is greater than or equal to a preset offset threshold, a first position of the operation object in the current video frame is determined based on the sum of the position of the operation object in the target historical video frame and the offset.

5. The method according to claim 1, wherein the first tracking result includes the position of the operation object in each historical video frame of the at least one historical video frame, and the operation object corresponding to each position in the first tracking result is assigned the same tracking ID; If the detection fails, adding the first position to the first tracking result includes: continuing the tracking ID assigned to the operation object in the at least one historical video frame to the current video frame, so as to add the first position to the first tracking result based on the tracking ID; If the detection is successful, the method further comprises: The tracking ID assigned to the operation object in the at least one historical video frame is continued to the current video frame, so as to add the position of the operation object detected from the current video frame to the first tracking result based on the tracking ID.

6. The method according to claim 1, wherein the quality detection of the pixel area where the operation object is located in the current video frame comprises: Based on the clarity and completeness of the operation object in the pixel area, obtaining a first quality detection result of the pixel area; Acquire a second quality detection result of the pixel area based on a pre-trained quality detection model; The quality detection result of the current video frame is obtained based on the first quality detection result and the second quality detection result.

7. The method according to claim 1, wherein determining whether to use the current video frame for quality monitoring of the operation object based on the quality detection result comprises: Comparing a quality detection result of a video frame in a cache with a quality detection result of the current video frame, wherein the video frame in the cache is any historical video frame containing the operation object; If the quality detection result of the current video frame is better than the quality detection result of the video frame in the cache, the current video frame replaces the video frame in the cache; wherein the video frame in the cache is used to monitor the quality of the operation object.

8. The method according to claim 7, wherein replacing the video frame in the cache with the current video frame comprises: If the quality detection result of the current video frame is better than the preset quality detection result, the current video frame replaces the video frame in the cache, and no quality detection is performed on subsequent video frames containing the operation object; If the quality detection result of the current video frame is better than the quality detection result of the historical video frame, but not better than the preset quality detection result, the current video frame replaces the video frame in the cache.

9. The method according to any one of claims 7 or 8, further comprising: If the preset condition is met, deleting the first tracking result, and outputting the video frame in the cache to the business database; The preset condition includes at least one of the following: The duration of the failure of the operation object detection reaches a first preset duration threshold; The duration for which the operation object is blocked by other operation objects reaches a second preset duration threshold; The distance between the position of the operation object in the current video frame and the position of the operation object in the previous video frame of the current video frame is greater than a preset distance threshold; The position of the operator exceeds the preset operating area.

10. A video processing device, comprising: A detection module, used for detecting an operation object in a current video frame collected from an operation environment; an adding module, configured to predict a first position of the operation object in the current video frame based on the first tracking result of the operation object if the detection fails, and add the first position to the first tracking result when the distance between the first position and a second position of the operator of the operation object in the current video frame is less than a preset distance threshold; The first tracking result is obtained by tracking the operation object in at least one historical video frame collected from the operation environment; The determination module is used to perform quality detection on the pixel area where the operation object is located in the current video frame if the detection is successful, and determine whether to use the current video frame for quality monitoring of the operation object based on the quality detection result.

11. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 9 when executing the program.

Images