Target snapshot filtering method and system and storage medium

Abstract

The invention relates to a target snapshot filtering method and system and a storage medium, and the method comprises the steps: obtaining a snapshot object in a to-be-detected video frame; calculating a discrimination statistical index of the snapshot object; detecting a small target object in each snapshot object based on the discrimination statistical index, and obtaining a statistical result of the small target object; determining a target filtering mode based on the statistical result; in the target filtering mode, judging whether the snapshot object is filtered or not; if yes, filtering the snapshot object; otherwise, generating a target snapshot result. According to the invention, the problem of low video snapshot accuracy is solved.

Description

Technical Field

[0001] The present application relates to the field of snapshot technology, and in particular to a target snapshot filtering method, system and storage medium. Background Art

[0002] In order to improve the effectiveness of structured data of targets in complex scenes and reduce storage consumption, the structured data of targets is usually collected by capturing. However, in some feature scenes such as large viewing angles, high density or low illumination, the human body is partially blocked or blurred in low illumination. In this case, the features of the human body and small targets such as small animals cannot be distinguished, causing the detection algorithm to misdetect small targets such as small animals as humans, ultimately resulting in a large number of miscaptured small targets, which in turn affects the accuracy of video capture.

[0003] Currently, no effective solution has been proposed for the problem of low accuracy of video capture in related technologies. Summary of the invention

[0004] The embodiments of the present application provide a target capture filtering method, system and storage medium to at least solve the problem of low accuracy of video capture in the related art.

[0005] In a first aspect, an embodiment of the present application provides a target capture filtering method, the method comprising:

[0006] Obtain the captured object in the video frame to be tested;

[0007] Calculating the discriminant statistical index of the captured object; determining the small target object in each captured object based on the discriminant statistical index, and obtaining the statistical results of the small target object;

[0008] Based on the statistical results, determining a target filtering mode;

[0009] In the target filtering mode, it is determined whether to filter the captured object; if so, the captured object is filtered; otherwise, a target capture result is generated.

[0010] In some embodiments, obtaining the captured object in the video frame to be tested includes:

[0011] Acquire a video stream, and detect a target frame in a deleted state in the video stream;

[0012] Based on the target frame, a number of the to-be-tested video frames within the target life cycle in the video stream are determined, and the captured object detected by the target in each of the to-be-tested video frames is obtained.

[0013] In some embodiments, obtaining the statistical results of the small target object includes:

[0014] Obtain the current snapshot queue, and obtain the queue length and current storage quantity of the snapshot queue;

[0015] Based on the queue length and the current storage quantity, each of the captured objects is stored in the current captured queue to obtain a new captured queue;

[0016] Count the small target objects in the new snapshot queue and obtain the statistical results.

[0017] In some embodiments, determining a target filtering mode based on the statistical results includes:

[0018] If the statistical result indicates that the proportion of small target objects is greater than the first mode switching threshold, the target filtering mode is determined to be the first filtering mode; the proportion of small target objects is the proportion of small target objects in each of the captured objects;

[0019] If the statistical result indicates that the proportion of small target objects is less than the second mode switching threshold, the target filtering mode is determined to be the second filtering mode; wherein the first mode switching threshold is greater than the second mode switching threshold.

[0020] In some embodiments, in the target filtering mode, determining whether to filter the captured object; if so, filtering the captured object; otherwise, generating a target capture result, including:

[0021] querying the target filtering mode;

[0022] When it is found that the target filtering mode is in the first filtering mode, judging whether the captured object is a small target object based on the discrimination statistical index; if so, filtering the captured object, otherwise, generating the target capture result;

[0023] When it is found that the target filtering mode is in the second filtering mode, the target snapshot result is generated.

[0024] In some embodiments, generating the target snapshot result includes:

[0025] Obtaining target quality information of each of the video frames to be tested, and obtaining historical quality information;

[0026] The video frames whose target quality information is higher than the historical quality information in each of the video frames to be tested are determined as output video frames, and the target capture results are generated based on the output video frames.

[0027] In some embodiments, the discrimination statistical indicators include the number of small target discriminations and the number of effective discriminations; and determining the small target objects in each of the captured objects based on the discrimination statistical indicators includes:

[0028] Determine the proportion of the number of times each of the captured objects is identified according to the number of times the small target is identified and the number of times the effective identification is identified;

[0029] Based on the proportion of the number of determinations, the small target object is determined from the captured objects.

[0030] In some embodiments, the calculating of the discriminant statistical index of the captured object includes:

[0031] Obtaining a prediction value and a confidence level of the captured object;

[0032] Detecting a discrimination result and a prediction effective result for the captured object according to the prediction value and the confidence level;

[0033] According to the discrimination result, the number of small target discriminations is counted, and according to the predicted effective result, the number of effective discriminations is counted.

[0034] In a second aspect, an embodiment of the present application provides a target capture filtering system, including:

[0035] A captured object acquisition unit, used to acquire the captured object in the video frame to be tested;

[0036] A small target object statistics unit is used to calculate the discrimination statistical index of the captured object; based on the discrimination statistical index, determine the small target object in each captured object, and obtain the statistical results of the small target object;

[0037] A mode switching unit, used for determining a target filtering mode based on the statistical results;

[0038] The snapshot filtering unit is used to determine whether to filter the snapshot object in the target filtering mode; if so, filter the snapshot object; otherwise, generate a target snapshot result.

[0039] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the target capture filtering method as described in the first aspect above.

[0040] Compared with the related art, the target capture filtering method, system and storage medium provided in the embodiments of the present application obtain the capture object in the video frame to be tested; calculate the discrimination statistical index of the capture object; based on the discrimination statistical index, detect the small target object in each capture object, and obtain the statistical results of the small target object; based on the statistical results, determine the target filtering mode; in the target filtering mode, determine whether to filter the capture object; if so, filter the capture object; otherwise, generate the target capture result. Based on this, a scene-adaptive filtering mode switching strategy is implemented; and the image picking process and the filtering mode strategy are interactively perceived to perform corresponding filtering operations, so that even in special scenes such as large viewing angles, high density or low illumination, small target objects can be filtered quickly and accurately, thereby effectively improving the accuracy of video capture.

[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 It is a hardware structure block diagram of a terminal of a target capture filtering method according to an embodiment of the present application;

[0044] Figure 2 is a flow chart of a target capture filtering method according to an embodiment of the present application;

[0045] Figure 3 is a flow chart of another target capture filtering method according to an embodiment of the present application;

[0046] Figure 4 is a flow chart of another target capture filtering method according to an embodiment of the present application;

[0047] Figure 5 It is a structural block diagram of a target capture and filtering system according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0049] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0050] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0051] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 1 is a hardware structure block diagram of a terminal according to a target capture filtering method of an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0052] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the target capture filtering method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0053] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0054] In actual business such as traffic management, urban management, and regional monitoring, video structuring is required. As the name implies, video structuring is to perform structured analysis on video scenes and extract attribute information and feature information, i.e., structured information, including human bodies, motor vehicles, non-motor vehicles, license plates, and facial targets. In order to ensure the effectiveness of video structuring and reduce storage consumption, small targets such as small animals that may be captured in video scenes need to be filtered to avoid misidentification of such small targets in subsequent steps and output for attribute recognition processing.

[0055] Based on this, this embodiment provides a target capture filtering method. Figure 2 is a flow chart of a target capture filtering method according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:

[0056] Step S210, obtaining the captured object in the video frame to be tested.

[0057] The above-mentioned video frames to be tested may be multiple video frames in a video stream data containing a certain video scene obtained from an image database, or may be multiple video frames in a video stream data obtained by shooting a video scene using a camera device. In this step, each frame of the video frame to be tested may be processed using methods such as a target detection algorithm to obtain position information of multiple captured objects, and an identification (ID) may be assigned to each captured object.

[0058] Step S220, calculating the discriminant statistical index of the captured object; based on the discriminant statistical index, determining the small target object in each captured object, and obtaining the statistical result of the small target object.

[0059] Among them, the discriminant statistical index is a statistic used to measure whether the captured object meets certain specific conditions or attributes. These indicators may include but are not limited to: the number of occurrences (the total number of times the captured object appears in the image or video), the recognition accuracy (the proportion of the captured object correctly identified), the confidence (the credibility of the captured object being identified as a specific category), and the category distribution (the proportion of the captured object belonging to different categories). For example, for each captured object, according to the output results of the detection algorithm of each frame of the video to be tested, the total number of times it appears in each frame of the video to be tested, the corresponding prediction results, and the confidence level are counted to obtain the discriminant statistical index. In addition, in the present embodiment, the above-mentioned discriminant statistical indicators can be stored in the preset trajectory target cache table according to the target dimension; when each frame of the target is updated, the corresponding discriminant statistical indicators can be updated in the cache table.

[0060] Next, by comprehensively analyzing the multi-dimensional discriminant statistical indicators, the captured objects belonging to small targets such as small animals can be accurately identified in the captured objects; if the number of frames in which a captured object is judged as a small target is large or the confidence is high according to the discriminant statistical indicators, it can be preliminarily considered that the captured object is a small target object. Small target objects specifically refer to small-sized moving objects appearing in the video scene; the small size can be a size smaller than the preset size value. Usually, the small target object can be a small animal (such as a mouse, etc.), small debris (such as falling leaves, paper, plastic bags, etc.) or other objects. In addition, although pedestrians and vehicles are important objects in video structured analysis, when they are far away from the camera, resulting in only a small part of the picture, the accuracy of their recognition and analysis may be affected; in this case, according to specific needs, objects such as pedestrians and vehicles in the distance can also be considered as small target objects for filtering. In this step, the recognition of small target objects is mainly to count the number of small target objects in each captured object, so that different filtering modes can be switched adaptively according to the scene later.

[0061] Step S230: determining a target filtering mode based on the statistical results.

[0062] In specific implementation, based on the statistical results of the above-mentioned small target objects, a scene discrimination result for the above-mentioned video frame to be tested can be generated; the scene discrimination result refers to the number discrimination result of the captured objects that are initially detected as small targets such as small animals in the video scene in the video frame to be tested. In other words, according to the statistical results of the number of small target objects determined in each captured object, and compared with the preset mode switching threshold, it can be determined whether the number of captured objects that are initially detected as small targets such as small animals in the video scene contained in the video frame to be tested is large, and then the corresponding target filtering mode is determined.

[0063] For example, if based on the statistical results, it is detected that the number of small target objects is greater than a preset quantity threshold, or the ratio between small target objects and all captured objects is greater than a preset ratio threshold, it is considered that the number of objects judged to be small targets in the current video scene is large. In order to ensure the accuracy of the final capture output, it is necessary to determine the filter mode 1 with a higher filter threshold and switch to the currently determined filter mode 1, so that the captured objects can be filtered under this filter mode in subsequent steps. The above-mentioned quantity threshold or ratio threshold can be used as a mode switching threshold. On the contrary, it can be considered that the proportion of small targets in the current video scene is not large, and the filter mode 2 with a relatively lower filter threshold can be determined, and the captured objects can be filtered under this filter mode 2. Afterwards, the current filter mode is switched to the target filter mode; of course, if the current filter mode happens to be the target filter mode, there is no need to switch.

[0064] It should be understood that, if permitted by the embodiment, the above determination of the target filtering mode can be further subdivided. For example, more ratio thresholds of gears can be set, and the comparison results of the ratio between the small target object and all captured objects and the ratio thresholds of different gears can be detected according to the above statistical results, and the matched target filtering mode with different filtering thresholds can be determined accordingly.

[0065] It can be seen that through the above step S230, based on the statistical results of small target objects obtained by previous calculations, the preliminary statistics of the number of small target objects in the current video scene can be analyzed to determine the corresponding target filtering mode, so that the filtering mode can be adapted to the video scene, thereby realizing a scene-adaptive designated filtering strategy, and the filtering mode can be automatically switched based on the filtering strategy.

[0066] Step S240, in the target filtering mode, determine whether to filter the captured object; if so, filter the captured object; otherwise, generate the target capture result.

[0067] In the actual processing of video structuring, usually a frame of image with the highest target score appearing in the video screen is selected, and specific operations such as attribute recognition are performed on this target on this frame; this process is usually called image selection. In this step, when selecting images, in the target filtering mode determined and switched as above, it is determined whether each captured object is a small animal based on the discriminant statistical indicators. If so, the captured objects determined as small targets are filtered, that is, the captured object is not output at this time; otherwise, it means that the captured object does not belong to the mistakenly captured targets such as small animals, and there is no need to filter the captured object at this time, and a frame of video containing the captured object is output for subsequent attribute recognition, that is, the above-mentioned target capture result is generated. Through the above steps, the image selection process can interact with the filtering strategy to perceive the current filtering mode and perform corresponding filtering operations.

[0068] The following is a brief description of the process of judging whether to filter the captured object in the target filtering mode. Based on the above analysis, it can be seen that different filtering modes are configured with different filtering thresholds; and when there are multiple filtering modes, one of the filtering modes may not be configured with a filtering threshold. Take the filtering mode as an example, which includes three different modes; filtering mode 1, filtering mode 2 and filtering mode 3 are respectively configured with corresponding filtering thresholds N1, filtering thresholds N2 and filtering thresholds N3; wherein N1 is greater than N2, and N2 is greater than N3. Under filtering mode 1, the number of video frames detected based on the discriminant statistical index indicating that the captured object is judged to be a small target is compared with the filtering threshold N1. If the number of video frames detected is greater than or equal to the filtering threshold N1, it can be considered that the captured object is a small target and needs to be filtered; otherwise, no filtering is required, and the video frame containing the captured object can be output, that is, the target capture result is generated. Similarly, under filtering mode 2 or filtering mode 3, the number of video frames detected based on the discriminant statistical index is compared with the filtering threshold N2 or the filtering threshold N3 to determine whether the captured object needs to be filtered.

[0069] For another example, the filtering mode includes the above-mentioned filtering mode 1, and also includes filtering mode 4 without configuring a filtering threshold. For filtering mode 4 without configuring a filtering threshold, in this filtering mode, it is generally believed that small targets account for a small proportion in the video scene, so the probability of erroneous output is not high, and the target video frame containing the captured object can be directly output.

[0070] In the above-mentioned target capture filtering method, according to the statistical indicators of each captured object, a preliminary judgment is made as to whether it is a small target such as a small animal, and according to the statistical results of the preliminary judgment, it is judged whether the number of small target objects in the video scene is large, and the target filtering mode is adjusted according to the judgment result, thereby realizing a scene-adaptive filtering mode switching strategy; and the image picking process and the filtering mode strategy are interactively perceived to perform corresponding filtering operations, so that even in special scenes such as large viewing angle, high density or low illumination, small target objects can be filtered quickly and accurately. Therefore, the target capture filtering does not need to rely on the results of the detection and tracking algorithm, avoiding the problem that the detection algorithm is prone to false detection or missed detection in special scenes, resulting in low accuracy of video capture, and effectively improving the accuracy of video capture.

[0071] In some embodiments, the above-mentioned obtaining the captured object in the video frame to be tested may also include the following steps:

[0072] A video stream is obtained, and a target frame in a deleted state is detected in the video stream; based on the target frame, a number of video frames to be tested within the target life cycle in the video stream are determined, and a captured object detected by the target in each video frame to be tested is obtained.

[0073] The video stream may be video stream data containing a certain video scene stored in a database or cached; or, in order to ensure the real-time performance of target capture filtering, the video stream may be video stream data obtained in real time when a camera device shoots a video scene. By performing target detection on each frame of the video stream parsed, the state of each frame may be determined.

[0074] Specifically, in the first frame of the video frame sequence, all potential targets are identified by the target detection algorithm, and a unique identifier (ID) is assigned to each target. In each subsequent frame, the tracking algorithm module is used to track these targets. The tracking algorithm attempts to match the target in the current frame with the target in the previous frame based on the target's appearance features, motion trajectory and other information. For the successfully matched target, its state is updated to the confirmed state (confirmed). If the target fails to find a match in the current frame, but it has successfully matched in the past frame, its state is set to a tentative state (tentative), indicating that the target is subject to further observation. If the target fails to find a match in multiple consecutive frames and it has exceeded the preset life cycle threshold (i.e., it has not been associated with the target in the past frame for a long time), its state is updated to a deleted state (deleted). If there are targets in the list with a deleted state (deleted), it is considered that these targets will disappear after this frame, that is, they are the captured targets of the current frame. Then, through the above-mentioned target current frame determined to be in the deleted state, the life cycle of the target appearing in the video screen can be determined.

[0075] It should be noted that the target life cycle refers to the entire life cycle of the target from its appearance to its disappearance. According to the detection results of the target frame, the entire time window in the video stream within the target life cycle can be divided. Select the video frames to be tested in the time window, and these frames will be used for subsequent captured object detection; among them, all frames in the time window can be selected, or some frames can be selected according to the sampling rate. Afterwards, for each captured object identified from the selected video frames to be tested, the statistical results of the small target objects are calculated based on the statistical discrimination statistical indicators, thereby realizing scene-adaptive filtering mode switching.

[0076] Through the above embodiments, statistical indicators of captured objects detected in video frames within the target life cycle are recorded, as well as statistical records of small target object discrimination results. This allows the system to switch filtering modes more efficiently based on the monitoring and analysis of the target life cycle, reducing unnecessary calculations and resource consumption, thereby helping to improve the accuracy and efficiency of the filtering mode switching strategy.

[0077] In some embodiments, the above-mentioned discrimination statistical indicators include the number of small target discriminations and the number of effective discriminations; the above-mentioned determination of the small target objects in each captured object based on the discrimination statistical indicators also includes the following steps:

[0078] According to the number of small target identifications and the number of effective identifications, the ratio of identification times of each captured object is determined; based on the ratio of identification times, the small target object is determined from the captured objects.

[0079] Among them, the number of small target identifications refers to the number of frames in which the captured object is identified as a small target object in each frame of the video to be tested; the number of valid identifications refers to the number of frames in which the target detection output results for the captured object are determined to be valid in each frame of the video to be tested. By calculating the ratio of the number of small target identifications to the number of valid identifications, that is, calculating the ratio of the number of times the captured object is identified as a small target object to the number of valid identifications, it is helpful to accurately detect whether the captured object is a small target object, and thus determine the object that is determined to be a small target object among all captured objects according to the above method.

[0080] Furthermore, the above-mentioned calculation of the discriminant statistical index of the captured object may also include the following steps:

[0081] Obtain the predicted value and confidence of the captured object; detect the discrimination result and the predicted effective result for the captured object based on the predicted value and confidence; count the number of small target discriminations based on the discrimination result, and count the number of effective discriminations based on the predicted effective result.

[0082] The above-mentioned prediction value refers to the prediction value of the small animal of each captured object obtained for each frame of the video to be tested; the above-mentioned confidence level refers to the confidence level of the detection result of each captured object obtained for each frame of the video to be tested.

[0083] Specifically, if the predicted value is detected to be 1, and the confidence is greater than or equal to the preset small target confidence threshold T1, then it is considered that for this frame of the video to be tested, the detected captured object is determined to be a small target, and the prediction is valid. If the predicted value is detected to be greater than 1, then it is considered that for this frame of the video to be tested, the detected captured object is not a small target such as a small animal, but a target of other categories, but the detection algorithm is valid for the prediction of this frame. If the predicted value is detected to be 1, and the statistical confidence is less than the small target confidence threshold T1; or, if the predicted value is detected to be 0, then it is considered that for this frame of the video to be tested, the detected captured object category is not determined to be a small animal, and the prediction is invalid.

[0084] Based on the above analysis, according to the statistical results of whether the captured object is judged as a small target in different situations, and whether the prediction is valid, the corresponding small target discrimination times and valid discrimination times are accumulated. Among them, for each frame of the video to be tested, when the result of the captured object being judged as a small target is detected, the number of small target discrimination times is accumulated by 1 (the initial value is 0); otherwise, it is not accumulated, and the number of small target discrimination times remains unchanged. When the prediction result of the frame prediction is valid is detected, the number of valid discrimination times is accumulated by 1 (the initial value is 0); otherwise, it is not accumulated, and the number of valid discrimination times remains unchanged.

[0085] Through the above embodiment, the discrimination statistical indicators of two dimensions, namely the number of small target discriminations and the number of effective discriminations, are set. Whether the captured object is a small target is judged by multi-dimensional statistical indicators, thereby effectively eliminating some miscapture situations and further improving the accuracy of video capture.

[0086] In some embodiments, the above-mentioned obtaining of statistical results of small target objects may further include the following steps:

[0087] Get the current snapshot queue, and get the queue length and current storage quantity of the snapshot queue; based on the queue length and current storage quantity, store each snapshot object into the current snapshot queue to get a new snapshot queue; count the small target objects in the new snapshot queue and get the statistical results.

[0088] In order to reflect the change in the proportion of small target objects in the current video screen over time, in this embodiment, when a certain frame target is about to be deleted, the captured object can be stored in the capture queue. Since the storage capacity of the capture queue is limited, it is not possible to add capture objects indefinitely, and the capture object that was first added to the queue is likely to no longer be in the video screen, and should not be included in the proportion statistics; therefore, the queue length N of the preset capture queue can be determined first, and the capture object can be added to the capture queue accordingly by comparing the queue length N with the number of capture objects stored in the capture queue, that is, the current storage number.

[0089] Specifically, traverse each snapshot object, and for the currently traversed snapshot object, check whether the current storage quantity has reached the queue length before storing; if so, remove the snapshot object that was added earliest in the current snapshot queue, and then add the currently traversed snapshot object to the queue; otherwise, directly store the current snapshot object in the snapshot queue. Repeat the above steps for other traversed snapshot objects until all snapshot objects have been stored in the queue, or the storage quantity in the snapshot queue has reached N, or it has not reached N but the overall statistical time has reached t1 (considering sparse target scenarios), and a new snapshot queue is obtained. Afterwards, for the snapshot queue, the proportion of small target objects in the queue is counted, and the filtering mode is switched.

[0090] Through the above embodiment, the captured objects are stored in the capture queue based on the queue length and the current storage quantity, which can maintain the effectiveness of the captured objects in the queue as much as possible, reflecting the change in the proportion of small target objects in the current video screen over time, thereby effectively improving the accuracy and real-time performance of filtering detection.

[0091] In some embodiments, the above-mentioned determination of the target filtering mode based on the statistical results may further include the following steps:

[0092] If the statistical results indicate that the proportion of small target objects is greater than the first mode switching threshold, the target filtering mode is determined to be the first filtering mode; the proportion of small target objects is the proportion of small target objects in each captured object; if the statistical results indicate that the proportion of small target objects is less than the second mode switching threshold, the target filtering mode is determined to be the second filtering mode; wherein the first mode switching threshold is greater than the second mode switching threshold.

[0093] Specifically, if it is detected that the proportion of small target objects in all captured objects is greater than the first mode switching threshold (or called the strict threshold), it means that the proportion of small target objects in the current video scene is not low enough, and the target filtering mode can be determined to be the first filtering mode (also regarded as the strict mode). At this time, if the current filtering mode is the second filtering mode, it is switched to the first filtering mode; if the current filtering mode is already the first filtering mode, no change is required. If it is detected that the proportion of small target objects is less than the second mode switching threshold (or called the ordinary threshold), it means that the proportion of small target objects in the current video scene is not high enough, and the target filtering mode can be determined to be the second filtering mode (also regarded as the ordinary mode). At this time, if the current filtering mode is the first filtering mode, it is switched to the second filtering mode; if the current filtering mode is already the second filtering mode, no change is required. It should also be understood that if the statistical results indicate that the proportion of small target objects is less than or equal to the first mode switching threshold, and greater than or equal to the second mode switching threshold, the current filtering mode can be kept unchanged.

[0094] Through the above embodiments, the filtering mode can be dynamically adjusted according to the real-time proportion of small target objects in the video screen; this adaptive capability enables the system to respond to different monitoring scenarios and requirements more flexibly. In addition, by setting the first mode switching threshold and the second mode switching threshold, the system can automatically switch to a more appropriate filtering mode when the proportion of small target objects is too high or too low, thereby ensuring that the system always maintains the optimal working state through this switching mechanism.

[0095] In some embodiments, in the target filtering mode, determining whether to filter the captured object; if so, filtering the captured object; otherwise, generating the target capture result may further include the following steps:

[0096] Query the target filtering mode; when it is found that the target filtering mode is in the first filtering mode, determine whether the captured object is a small target object based on the discrimination statistical indicators; if so, filter the captured object, otherwise, generate the target capture result; when it is found that the target filtering mode is in the second filtering mode, generate the target capture result.

[0097] In specific implementation, before outputting the target, first obtain the filter mode saved based on the above-mentioned filter mode switching strategy through query means. If the current target filter mode is found to be the first filter mode, it is necessary to detect whether the current captured object belongs to a small target object through the above-mentioned discriminant statistical indicators. If it is a small target, the output of the captured object is restricted; otherwise, the video frame containing the captured object is output so that the target contained therein can be subjected to structured processing such as attribute recognition in subsequent steps. If the current target filter mode is found to be the second filter mode, it means that the proportion of small animals in the current video scene is not large, and the probability of target error output is not high. Therefore, in this filter mode, it can be determined that there is no need to filter the captured object, and the video frame containing the captured object can be directly output, thereby effectively improving the efficiency of video capture processing.

[0098] In some embodiments, the above-mentioned generation of the target snapshot result may also include the following steps:

[0099] Obtain target quality information of each video frame to be tested, and obtain historical quality information; determine the video frame whose target quality information is higher than the historical quality information among the video frames to be tested as the output video frame, and generate the target capture result based on the output video frame.

[0100] Specifically, in the image picking process, for each frame of video to be tested that is sent to the image picking algorithm, the target quality score of the detected captured object is calculated; the frame of image with a target quality score higher than the historical quality score, and the target information of the captured object contained in the frame of image are recorded. If it is determined through the above steps that the captured object is not to be filtered, the video frame picked out above, or the frame of video with the highest target quality score, is used as the target capture result for subsequent structured processing and display. Through the above embodiments, the image picking process is closely linked to the filtering strategy, thereby effectively avoiding the situation in the related technology where the selection of captured frames depends solely on the target quality score, which leads to the problem of easy miscapture.

[0101] The following is a description of the specific embodiments. Figure 3 , provides a flowchart for adaptively switching filtering modes through a program-defined policy module, the process comprising the following steps:

[0102] Step S301, start the process; create a policy module and initialize the current filtering mode (generally initialized to 0-normal mode).

[0103] Step S302, when each video frame comes in, traverse the set of included targets (i.e. the captured objects mentioned above).

[0104] Step S303, obtain the small animal prediction value is_animal and its confidence is_animal_conf (output of the target scoring algorithm) of each target. At the same time, define two statistical indicators: the number of times the target is judged as a small animal (animal_count) and the number of times the target is effectively judged (valid_count). The specific assignment rules of the above indicators are:

[0105] If is_animal is 1, and is_animal_conf is greater than the small animal confidence threshold T1, then animal_count is increased by 1, and valid_count is increased by 1, which means that the target in this frame is considered to be a small animal and the prediction is valid;

[0106] If is_animal is greater than 1, animal_count remains unchanged and valid_count increases by 1, which means that the target is not an animal in this frame, but the prediction is valid;

[0107] If is_animal is 1, and is_animal_conf is less than the animal confidence threshold T1, or is_animal is 0, then animal_count and valid_count remain unchanged, that is, the target in this frame is considered not an animal, and the prediction is invalid;

[0108] For the above information, the trajectory target cache table can be used to store it according to the target dimension. When the target is updated in each frame, its corresponding statistical indicators will be updated.

[0109] Step S304, after the track target cache table is updated, it is necessary to traverse all targets again. If the target is in the Delete state in the current frame, it is considered that the target will disappear after this frame. Generally, such a target is called a captured target. For the captured target, the ratio of the number of times it is judged as a small animal in its life cycle to the number of valid judgments is calculated, that is, animal_count / valid_count. If the ratio is greater than the small animal ratio threshold T2, the current target is considered to be a small animal, otherwise it is considered not to be a small animal.

[0110] Step S305, storing the above target in the snapshot queue. It is determined whether the current storage quantity of the snapshot queue has reached the queue length; if so, the earliest target in the queue is removed and added to the current target; otherwise, the target is directly added to the current target.

[0111] Step S306, determining whether the proportion of small animals in the capture queue is greater than the strict threshold. If so, switching the current filtering mode to the strict mode, and continuing to execute the subsequent step S308.

[0112] Step S307, if the judgment result of the above step S306 is no, then continue to judge whether the proportion of small animals in the queue is less than the normal threshold. If so, switch the filtering mode to the normal mode and continue to execute the subsequent step S308; otherwise, directly execute step S308.

[0113] Step S308, the subsequent video frames are cycled back and forth until all the video frames to be tested are detected, or the preset detection time is reached; and the process ends.

[0114] On the other hand, see Figure 4 , provides a flowchart for capturing and filtering images through a program-defined image selection module, which includes the following steps:

[0115] Step S401, start the process; when each frame of data enters, in addition to recording this frame of image and target information when the target quality score is higher than the historical score, it is also necessary to record the statistical indicators of the small animals of the target according to similar rules in the above step S302.

[0116] Step S402, determine whether the target meets the capture condition. If the capture condition is not met, return to the above step S401.

[0117] Step S403, when the target is judged to meet the capture condition (generally in the deletion state) through the above step S402, before outputting the target, the filtering mode saved by the policy module will be obtained by query means, and it will be judged whether the current filtering mode is a strict mode. If the judgment result of the above step S403 is no, that is, the current filtering mode is a normal mode, no filtering processing will be performed, and the subsequent step S405 will be directly executed.

[0118] Step S404, if the judgment result of the above step S403 is yes, that is, the current filtering mode is the strict mode, it is necessary to judge whether the current target belongs to the small animal type according to the similar rules in the above step S303. If it is a small animal, the output of the target is restricted, that is, the target is not output; otherwise, no processing is performed, and the subsequent step S405 is directly executed.

[0119] Step S405, output the target.

[0120] Step S406, the subsequent frames are cycled back and forth until all the video frames to be tested are detected, or the preset picture selection time is reached; the process ends.

[0121] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0122] This embodiment also provides a target capture filtering system, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the terms "part", "module", "unit", "sub-unit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0123] Figure 5 is a structural block diagram of a target capture filtering system according to an embodiment of the present application, such as Figure 5 As shown, the system includes:

[0124] The captured object acquisition unit 51 is used to acquire the captured object in the video frame to be tested;

[0125] A small target object statistics unit 52 is used to calculate the discriminant statistical index of the captured object; based on the discriminant statistical index, determine the small target object in each captured object, and obtain the statistical results of the small target object;

[0126] A mode switching unit 53, used to determine a target filtering mode based on the statistical results;

[0127] The capture filtering unit 54 is used to determine whether to filter the capture object in the target filtering mode; if so, filter the capture object; otherwise, generate the target capture result.

[0128] In some of the embodiments, the captured object acquisition unit 51 is also used to acquire the video stream and detect the target frame in the video stream that is in a deleted state; the captured object acquisition unit 51 is also used to determine a number of video frames to be tested within the target life cycle in the video stream based on the target frame, and acquire the captured object detected by the target from each video frame to be tested.

[0129] In some of the embodiments, the small target object statistics unit 52 is also used to obtain the current snapshot queue, and obtain the queue length and current storage quantity of the snapshot queue; the small target object statistics unit 52 is also used to store each snapshot object into the current snapshot queue based on the queue length and the current storage quantity to obtain a new snapshot queue; the small target object statistics unit 52 is also used to count the small target objects in the new snapshot queue and obtain the statistical results.

[0130] In some of the embodiments, the mode switching unit 53 is also used to determine that the target filtering mode is the first filtering mode if the statistical result indicates that the proportion of small target objects is greater than the first mode switching threshold; the proportion of small target objects is the proportion of small target objects in each captured object; the mode switching unit 53 is also used to determine that the target filtering mode is the second filtering mode if the statistical result indicates that the proportion of small target objects is less than the second mode switching threshold; wherein the first mode switching threshold is greater than the second mode switching threshold.

[0131] In some embodiments, the snapshot filter unit 54 is also used to query the target filtering mode; the snapshot filter unit 54 is also used to determine whether the captured object is a small target object based on the discrimination statistical indicators when the target filtering mode is queried to be in the first filtering mode; if so, the captured object is filtered, otherwise, the target snapshot result is generated; the snapshot filter unit 54 is also used to generate the target snapshot result when the target filtering mode is queried to be in the second filtering mode.

[0132] In some of the embodiments, the above-mentioned capture filter unit 54 is also used to obtain target quality information of each video frame to be tested and to obtain historical quality information; the capture filter unit 54 is also used to determine the video frames in each video frame to be tested whose target quality information is higher than the historical quality information as output video frames, and to generate target capture results based on the output video frames.

[0133] In some of the embodiments, the above-mentioned discrimination statistical indicators include the number of small target discriminations and the number of effective discriminations; the above-mentioned small target object statistics unit 52 is also used to determine the proportion of discrimination times of each captured object based on the number of small target discriminations and the number of effective discriminations; the small target object statistics unit 52 is also used to determine the small target objects from the captured objects based on the proportion of discrimination times.

[0134] In some of the embodiments, the small target object statistics unit 52 is also used to obtain the predicted value and confidence of the captured object; the small target object statistics unit is also used to detect the discrimination result and the predicted effective result for the captured object based on the predicted value and confidence; the small target object statistics unit 52 is also used to count the number of small target discriminations based on the discrimination results, and to count the number of effective discriminations based on the predicted effective results.

[0135] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0136] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0137] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0138] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0139] S1, obtaining the captured object in the video frame to be tested.

[0140] S2, calculating the discriminant statistical index of the captured object; based on the discriminant statistical index, detecting the small target object in each captured object, and obtaining the statistical result of the small target object.

[0141] S3, determining a target filtering mode based on the statistical results.

[0142] S4, in the target filtering mode, determine whether to filter the captured object; if so, filter the captured object; otherwise, generate the target capture result.

[0143] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0144] In addition, in combination with the target capture filtering method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any target capture filtering method in the above embodiment is implemented.

[0145] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target capture filtering method, characterized in that: The method comprises: Obtain the captured object in the video frame to be tested; Calculating the discriminant statistical index of the captured object; based on the discriminant statistical index, detecting the small target object in each captured object, and obtaining the statistical result of the small target object; Based on the statistical results, determining a target filtering mode; In the target filtering mode, it is determined whether to filter the captured object; if so, the captured object is filtered; otherwise, a target capture result is generated.

2. The target capture filtering method according to claim 1, characterized in that: The step of obtaining the captured object in the video frame to be tested includes: Acquire a video stream, and detect a target frame in a deleted state in the video stream; Based on the target frame, a number of the to-be-tested video frames within the target life cycle in the video stream are determined, and the captured object detected by the target in each of the to-be-tested video frames is obtained.

3. The target capture filtering method according to claim 1, characterized in that: The obtaining of the statistical result of the small target object includes: Obtain the current snapshot queue, and obtain the queue length and current storage quantity of the snapshot queue; Based on the queue length and the current storage quantity, each of the captured objects is stored in the current captured queue to obtain a new captured queue; Count the small target objects in the new snapshot queue and obtain the statistical results.

4. The target capture filtering method according to claim 1, characterized in that: Determining a target filtering mode based on the statistical results includes: If the statistical result indicates that the proportion of small target objects is greater than the first mode switching threshold, the target filtering mode is determined to be the first filtering mode; the proportion of small target objects is the proportion of small target objects in each of the captured objects; If the statistical result indicates that the proportion of small target objects is less than the second mode switching threshold, the target filtering mode is determined to be the second filtering mode; wherein the first mode switching threshold is greater than the second mode switching threshold.

5. The target capture filtering method according to claim 4, characterized in that: In the target filtering mode, determining whether to filter the captured object; If yes, filtering the captured object; Otherwise, the target snapshot results are generated, including: querying the target filtering mode; When it is found that the target filtering mode is in the first filtering mode, judging whether the captured object is a small target object based on the discrimination statistical index; if so, filtering the captured object, otherwise, generating the target capture result; When it is found that the target filtering mode is in the second filtering mode, the target snapshot result is generated.

6. The target capture filtering method according to claim 1, characterized in that: The generating target capture result includes: Obtaining target quality information of each of the video frames to be tested, and obtaining historical quality information; The video frames whose target quality information is higher than the historical quality information in each of the video frames to be tested are determined as output video frames, and the target capture results are generated based on the output video frames.

7. The target capture filtering method according to any one of claims 1 to 6, characterized in that: The discrimination statistical indicators include the number of small target discriminations and the number of effective discriminations; The step of determining the small target object in each of the captured objects based on the discriminant statistical indicator includes: Determine the proportion of the number of times each of the captured objects is identified according to the number of times the small target is identified and the number of times the effective identification is identified; Based on the proportion of the number of determinations, the small target object is determined from the captured objects.

8. The target capture filtering method according to claim 7, characterized in that: The calculating of the statistical index for distinguishing the captured object comprises: Obtaining a prediction value and a confidence level of the captured object; Detecting a discrimination result and a prediction effective result for the captured object according to the prediction value and the confidence level; According to the discrimination result, the number of small target discriminations is counted, and according to the predicted effective result, the number of effective discriminations is counted.

9. A target capture filtering system, characterized in that: include: A captured object acquisition unit, used to acquire the captured object in the video frame to be tested; A small target object statistics unit, used to calculate the discriminant statistical index of the captured object; Based on the discriminant statistical index, determine the small target object in each of the captured objects, and obtain the statistical results of the small target object; A mode switching unit, used for determining a target filtering mode based on the statistical results; A capture filtering unit, used for determining whether to filter the capture object in the target filtering mode; If yes, filtering the captured object; Otherwise, the target capture result is generated.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the target capture filtering method according to any one of claims 1 to 8 when running.

Images