Image screening method, image screening device, and image recognition system

Abstract

The application provides an image screening method, an image screening device and an image recognition system. The method comprises: acquiring a to-be-screened video; screening a plurality of image frames in the to-be-screened video through image recognition to obtain a first target object image frame, the first target object image frame being an image frame of the to-be-screened video in which a target object exists; screening a first image frame and a second image frame to obtain a second target object image frame, the second target object image frame being the first image frame or the second image frame of the to-be-screened video in which the target object exists, the first image frame being one or more image frames of the to-be-screened video in front of the first target object image frame, and the second image frame being one or more image frames of the to-be-screened video behind the first target object image frame. The method solves the problem of poor screening effect of manual image screening in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image screening method, an image screening device, a computer readable storage medium, a processor and an image recognition system. BACKGROUND

[0002] At present, it is difficult to efficiently find out the target to be supplemented when some data collected is intended to be used for training. A large amount of manual annotation is required to manually extract the target, and most pictures do not include the desired category, which is time-consuming and inefficient. Some methods are needed to automatically find out the pictures that may exist the object of interest in the massive collected data.

[0003] The above information disclosed in the background section is only for the purpose of enhancing the understanding of the background of the technology described herein, therefore, the background section can include some information that is not known to those skilled in the art as prior art in the country. SUMMARY

[0004] The main purpose of the present application is to provide an image screening method, an image screening device, a computer readable storage medium, a processor and an image recognition system to solve the problem of poor screening effect of manual image screening in the prior art.

[0005] According to one aspect of the embodiment of the present application, an image screening method is provided, comprising: acquiring a to-be-screened video; screening a plurality of image frames in the to-be-screened video through image recognition to obtain a first target object image frame, the first target object image frame being an image frame of the to-be-screened video in which a target object exists; screening a first image frame and a second image frame to obtain a second target object image frame, the second target object image frame being the first image frame or the second image frame of the to-be-screened video in which the target object exists, the first image frame being one or more image frames of the to-be-screened video before the first target object image frame, and the second image frame being one or more image frames of the to-be-screened video after the first target object image frame.

[0006] Optionally, the to-be-screened video is screened through image recognition to obtain the first target object image frame, comprising: extracting a feature of a to-be-recognized object in a to-be-screened image frame of the to-be-screened video, the feature comprising a shape feature and a color feature; and in a case where the feature of the to-be-recognized object is the same as a feature of the target object, determining that the to-be-screened image frame is the first target object image frame.

[0007] Optionally, the first image frame and the second image frame are screened to obtain a second target object image frame, including: labeling the target object of the first target object image frame to obtain a labeled first target object image, the label being a bounding box of the target object; starting from the labeled first target object image, tracking the target object in the first image frame and the second image frame through a target tracking algorithm to obtain a labeled first image frame and a labeled second image frame; and determining the labeled first image frame and the labeled second image frame as the second target object image frame.

[0008] Optionally, after the first image frame and the second image frame are screened to obtain a second target object image frame, the method further includes: deleting the first target object image frame and the second target object image frame to obtain a remaining image frame; obtaining point clouds of unknown objects in the remaining image frame to obtain a plurality of to-be-identified object point clouds; and in a case where the to-be-identified object point cloud satisfying an obstacle condition exists, determining the remaining image frame as a third target object image frame, the third target object image frame being an image frame of the to-be-screened video in which unknown objects exist, and the obstacle condition including that a point cloud volume is greater than a predetermined volume, a minimum height of a point cloud from the ground is less than a predetermined height, and point cloud data corresponding to a previous frame and a next frame of the remaining image frame both exist corresponding to-be-identified object point clouds.

[0009] Optionally, obtaining point clouds of unknown objects to obtain a plurality of to-be-identified object point clouds includes: obtaining point cloud data corresponding to the remaining image frame to obtain to-be-analyzed point cloud data, the to-be-analyzed point cloud data including point cloud data of all detected objects; clustering the to-be-analyzed point cloud data to obtain a plurality of object point clouds; and deleting point clouds corresponding to objects recognized by the remaining image frame from the plurality of object point clouds to obtain a plurality of to-be-identified object point clouds.

[0010] Optionally, before the remaining image frame is determined as a third target object image frame in a case where the to-be-identified object point cloud satisfying an obstacle condition exists, the method further includes: obtaining color information of the plurality of to-be-identified object point clouds, the color information being a color of an object corresponding to the to-be-identified object point cloud in the remaining image frame; and deleting the to-be-identified object point cloud with a predetermined color.

[0011] According to another aspect of the embodiments of the present application, there is also provided an image screening device, comprising: an acquisition unit configured to acquire a video to be screened; an identification unit configured to screen a plurality of image frames in the video to be screened through image recognition to obtain a first target object image frame, the first target object image frame being an image frame of the video to be screened in which a target object exists; and a screening unit configured to screen a first image frame and a second image frame to obtain a second target object image frame, the second target object image frame being the first image frame or the second image frame of the video to be screened in which the target object exists, the first image frame being one or more image frames of the video to be screened in front of the first target object image frame, and the second image frame being one or more image frames of the video to be screened behind the first target object image frame.

[0012] According to still another aspect of the embodiments of the present application, there is also provided a computer readable storage medium, the computer readable storage medium including a stored program, wherein the program performs any of the methods.

[0013] According to yet another aspect of the embodiments of the present application, there is also provided a processor configured to execute a program, wherein the program performs any of the methods when executed.

[0014] According to still another aspect of the embodiments of the present application, there is also provided an image recognition system, comprising: an image recognition model, one or more processors, a memory, a display device and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing any of the methods, the first target object image frame and the second target object image frame both being training samples of the image recognition model.

[0015] In the embodiment of the present application, the image screening method is characterized in that: first, a video to be screened is obtained; then, a plurality of image frames in the video to be screened are screened through image recognition to obtain a first target object image frame, the first target object image frame being an image frame of the video to be screened in which a target object exists; finally, a first image frame and a second image frame are screened to obtain a second target object image frame, the second target object image frame being the first image frame or the second image frame of the video to be screened in which the target object exists, the first image frame being one or more image frames of the video to be screened before the first target object image frame, and the second image frame being one or more image frames of the video to be screened after the first target object image frame. The image screening method screens the image frame in which the target object exists through image recognition, and screens the image frame before and after the image frame in which the target object exists according to the continuity of object movement to obtain the image frame in which the target object may exist, thereby avoiding missing the image frame in which the target object exists and reducing the proportion of the image frame in which the target object does not exist in the screened image frame. Compared with manual image screening, the screened image has a large number of images in which the target object does not exist, greatly improving the screening effect and solving the problem of poor screening effect of manual image screening in the prior art. Moreover, the method greatly improves the efficiency compared with manual screening and saves labor cost. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application and are incorporated herein for illustrative purposes. The schematic embodiments of the present application and the description thereof are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A flow chart of an image screening method according to an embodiment of the present application is shown;

[0018] Figure 2 A schematic diagram of an image screening device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0020] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0021] It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements can also be present. In addition, when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements can be present.

[0022] As mentioned in the background, the screening effect of manual screening of images in the prior art is poor, in order to solve the above problems, in a typical embodiment of the present application, an image screening method, an image screening device, a computer readable storage medium, a processor and an image recognition system are provided.

[0023] According to the embodiments of the present application, an image screening method is provided.

[0024] Figure 1 is a flowchart of the image screening method according to the embodiments of the present application. As shown in Figure 1 the method comprises the following steps:

[0025] Step S101, obtaining a to-be-screened video;

[0026] It should be noted that in the field of obstacle recognition of automatic driving, the above-mentioned to-be-screened video is a video of driving record taken by an automatic driving vehicle, which is used as a sample for obstacle recognition.

[0027] Step S102, screening a plurality of image frames in the above-mentioned to-be-screened video through image recognition to obtain a first target object image frame, the first target object image frame being an image frame of the above-mentioned to-be-screened video in which a target object exists;

[0028] Optionally, the present application does not limit the specific process of screening a plurality of image frames in the above-mentioned to-be-screened video through image recognition to obtain a first target object image frame, any feasible way belongs to the protection scope of the present application.

[0029] In order to improve the accuracy of image recognition, for example, in an optional embodiment, the above-mentioned step S102 comprises:

[0030] Step S1021, extracting features of the to-be-recognized object in the to-be-screened image frame of the to-be-screened video, the features including shape features and color features;

[0031] Step S1022, in a case where the features of the to-be-recognized object are the same as the features of the target object, determining that the to-be-screened image frame is the first target object image frame.

[0032] In the above embodiment, the features of the to-be-recognized object in the to-be-screened image frame are recognized by image recognition, and it is determined whether the to-be-recognized object is the target object, so as to screen the image frame of the to-be-screened video in which the target object exists, i.e., the first target object image frame.

[0033] Step S103, screening the first image frame and the second image frame to obtain a second target object image frame, the second target object image frame being the first image frame or the second image frame of the to-be-screened video in which the target object exists, the first image frame being the image frame before the first target object image frame or the image frame of the to-be-screened video, and the second image frame being the image frame after the first target object image frame or the image frame of the to-be-screened video.

[0034] Optionally, the present application does not limit the specific process of screening the first image frame and the second image frame to obtain the second target object image frame, and any feasible manner belongs to the protection scope of the present application.

[0035] In order to avoid missing the image frame of the to-be-screened video in which the target object exists, for example, in an optional embodiment, the step S103 includes:

[0036] Step S1031, labeling the target object in the first target object image frame to obtain a labeled first target object image, the label being a bounding box of the target object.

[0037] Step S1032, starting from the labeled first target object image, tracking the target object in the first image frame and the second image frame by using a target tracking algorithm to obtain a labeled first image frame and a labeled second image frame.

[0038] Step S1033, determining the labeled first image frame and the labeled second image frame as the second target object image frame.

[0039] In the above embodiment, the target object in the first target object image frame is labeled, the bounding box of the target object is marked, and the target object is tracked to mark the bounding box of the target object in the first image frame and the second image frame, and the first image frame and the second image frame in which the target object exists are found, so that the second target object image frame is obtained. Since the object moving track is continuous, the tracking algorithm determines whether there is an image frame in which the target object exists before and after the first target object image, so as to avoid omission.

[0040] The above image screening method is not limited to screening the image of the target object, but can also screen the unknown object of interest, so as to enrich the training library of the image recognition model. In an optional embodiment, after the step S103, the method further comprises:

[0041] Step 201, deleting the first target object image frame and the second target object image frame to obtain a remaining image frame;

[0042] Step 202, obtaining the point cloud of the unknown object in the remaining image frame to obtain a plurality of to-be-recognized object point clouds;

[0043] Step 203, in the case that the to-be-recognized object point cloud meets the obstacle condition, determining the remaining image frame as a third target object image frame, the third target object image frame being an image frame of the to-be-screened video in which the unknown object exists, and the obstacle condition including that the point cloud volume is greater than a predetermined volume, the minimum height of the point cloud from the ground is less than a predetermined height, and the point cloud data corresponding to the previous frame and the next frame of the remaining image frame both exist corresponding to-be-recognized object point clouds.

[0044] In the above embodiment, in the image frame of the video data, not only the target object of interest exists, but also unknown objects that cannot be recognized. Of course, these unknown objects are not necessarily obstacles. The point cloud of the unknown object is screened through the obstacle condition, the point cloud volume is greater than a predetermined volume to ensure that the unknown object really exists, the minimum height of the point cloud from the ground is less than a predetermined height to determine that the unknown object will form an obstacle, and the point cloud data corresponding to the previous frame and the next frame of the remaining image frame both exist corresponding to-be-recognized object point clouds to determine that the motion of the unknown object is continuous, further ensuring that the unknown object really exists, so as to screen the unknown object which can be an obstacle, and add the image frame in which the unknown obstacle exists to the image recognition training set, so that the image recognition model can recognize more obstacles.

[0045] In order to find the point cloud of the unknown object, in an optional embodiment, the step 202 comprises:

[0046] In step 2021, the point cloud data corresponding to the remaining image frames is obtained to obtain the to-be-analyzed point cloud data, and the to-be-analyzed point cloud data includes the point cloud data of all the detected objects.

[0047] In step 2022, the to-be-analyzed point cloud data is clustered to obtain a plurality of object point clouds.

[0048] In step 2023, the point cloud corresponding to the object recognized by the remaining image frames is deleted from the plurality of object point clouds to obtain a plurality of to-be-identified object point clouds.

[0049] In the above embodiment, the point cloud data is clustered to form a plurality of object point clouds, and the point cloud of the recognized object is deleted to obtain the point cloud of the unknown object, i.e., the to-be-identified object point cloud.

[0050] In order to improve the efficiency of screening, in an optional embodiment, before step 203, the method further includes:

[0051] In step 204, the color information of the plurality of to-be-identified object point clouds is obtained, and the color information is the color of the object corresponding to the to-be-identified object point cloud in the remaining image frames.

[0052] In step 205, the to-be-identified object point cloud with the color information being a predetermined color is deleted.

[0053] In the above embodiment, the predetermined color is artificially set, for example, the color of the obstacle does not exist, so as to delete the to-be-identified object point cloud with the predetermined color and reduce the workload of subsequent screening, thereby improving the efficiency.

[0054] In the image screening method, first, a video to be screened is obtained; then, a plurality of image frames in the video to be screened are screened through image recognition to obtain a first target object image frame, the first target object image frame being an image frame of the video to be screened in which a target object exists; finally, a first image frame and a second image frame are screened to obtain a second target object image frame, the second target object image frame being the first image frame or the second image frame of the video to be screened in which the target object exists, the first image frame being one or more image frames of the video to be screened before the first target object image frame, and the second image frame being one or more image frames of the video to be screened after the first target object image frame. The image screening method screens the image frame in which the target object exists through image recognition, and screens the image frame before and after the image frame in which the target object exists according to the continuity of object movement to obtain the image frame in which the target object may exist, thereby avoiding missing the image frame in which the target object exists, reducing the proportion of the image frame in which the target object does not exist in the screened image frame, and greatly improving the screening effect compared with manually screening the image. A large number of images do not contain the target object in the screened image, which greatly improves the screening effect and solves the problem of poor screening effect of manual image screening in the prior art. Moreover, the method greatly improves the efficiency compared with manual screening and saves labor costs.

[0055] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0056] The embodiment of the present application also provides an image screening device. It should be noted that the image screening device of the embodiment of the present application can be used to execute the image screening method provided by the embodiment of the present application. The image screening device provided by the embodiment of the present application is introduced as follows.

[0057] Figure 2 is a schematic diagram of the image screening device according to the embodiment of the present application. As shown in Figure 2 , the device comprises:

[0058] The acquisition unit 10 is configured to obtain a video to be screened.

[0059] It should be noted that in the field of obstacle recognition of automatic driving, the video to be screened is a video of driving record taken by an automatic driving vehicle as a sample for obstacle recognition.

[0060] The recognition unit 20 is configured to screen a plurality of image frames in the to-be-screened video by image recognition to obtain a first target object image frame, the first target object image frame being an image frame of the to-be-screened video in which the target object exists.

[0061] Optionally, the present application does not limit the specific process of screening a plurality of image frames in the to-be-screened video by image recognition to obtain a first target object image frame, and any feasible manner belongs to the protection scope of the present application.

[0062] In order to improve the accuracy of image recognition, for example, in an optional embodiment, the recognition unit comprises:

[0063] The first processing module is configured to extract a feature of the to-be-recognized object in the to-be-screened image frame of the to-be-screened video, the feature comprising a shape feature and a color feature;

[0064] The first determination module is configured to determine that the to-be-screened image frame is the first target object image frame in a case where the feature of the to-be-recognized object is the same as the feature of the target object.

[0065] In the above embodiment, the feature of the to-be-recognized object in the to-be-screened image frame is recognized by image recognition to determine whether the to-be-recognized object is the target object, thereby screening the image frame of the to-be-screened video in which the target object exists, i.e., the first target object image frame.

[0066] The screening unit 30 is configured to screen the first image frame and the second image frame to obtain a second target object image frame, the second target object image frame being the first image frame or the second image frame of the to-be-screened video in which the target object exists, the first image frame being one or more image frames of the to-be-screened video in front of the first target object image frame, and the second image frame being one or more image frames of the to-be-screened video behind the first target object image frame.

[0067] Optionally, the present application does not limit the specific process of screening the first image frame and the second image frame to obtain a second target object image frame, and any feasible manner belongs to the protection scope of the present application.

[0068] In order to avoid missing the image frame of the to-be-screened video in which the target object exists, for example, in an optional embodiment, the screening unit comprises:

[0069] The second processing module is configured to label the target object in the first target object image frame to obtain a labeled first target object image, the label being a bounding box of the target object.

[0070] a tracking module configured to track the target object in the first image frame and the second image frame by a target tracking algorithm, starting from the first target object image with labels, to obtain the first image frame with labels and the second image frame with labels;

[0071] a second determining module configured to determine the first image frame with labels and the second image frame with labels as the second target object image frame.

[0072] In the above embodiment, the target object in the first target object image frame is labeled to mark the bounding box of the target object, and then the target object is tracked to mark the bounding box of the target object in the first image frame and the second image frame, and the first image frame and the second image frame in which the target object exists are found, i.e. the second target object image frame is obtained. Since the object moving track is continuous, the tracking algorithm determines whether there are image frames with target objects before and after the first target object image, thereby avoiding omission.

[0073] The above image screening method is not limited to screening the image of the target object, but can also screen the unknown object of interest, so as to enrich the training library of the image recognition model. In an optional embodiment, the device further comprises:

[0074] a first deleting module configured to delete the first target object image frame and the second target object image frame to obtain a remaining image frame;

[0075] a first obtaining module configured to obtain the point cloud of the unknown object in the remaining image frame to obtain a plurality of to-be-recognized object point clouds;

[0076] a third determining module configured to determine the remaining image frame as a third target object image frame in the case that the to-be-recognized object point cloud satisfying the obstacle condition exists, the third target object image frame being an image frame of the to-be-screened video in which the unknown object exists, and the obstacle condition including that the point cloud volume is greater than a predetermined volume, the minimum height of the point cloud from the ground is less than a predetermined height, and the point cloud data corresponding to the previous frame and the next frame of the remaining image frame both exist corresponding to-be-recognized object point clouds.

[0077] In the above embodiment, in the image frames of the video data, not only the target objects of interest exist, but also unknown objects that cannot be identified. Of course, these unknown objects are not necessarily obstacles. The point cloud of the unknown object is filtered by the obstacle condition. The volume of the point cloud is greater than a predetermined volume to ensure that the unknown object actually exists. The minimum height of the point cloud from the ground is less than a predetermined height to determine that the unknown object will form an obstacle. The point cloud data corresponding to the previous frame and the next frame of the above-mentioned remaining image frames of the above-mentioned remaining image frames corresponding to the above-mentioned to-be-identified object point cloud exists to determine that the motion of the unknown object is coherent, and further ensure that the unknown object actually exists, thereby filtering out the unknown objects that can be obstacles. The image frames with unknown obstacles are added to the image recognition training set, so that the image recognition model can recognize more obstacles.

[0078] In order to find the point cloud of the unknown object, in an optional embodiment, the first acquisition module comprises:

[0079] The acquisition submodule is configured to acquire the point cloud data corresponding to the above-mentioned remaining image frames to obtain to-be-analyzed point cloud data. The to-be-analyzed point cloud data comprises the point cloud data of all detected objects.

[0080] The processing submodule is configured to cluster the to-be-analyzed point cloud data to obtain a plurality of object point clouds.

[0081] The deletion submodule is configured to delete the point cloud corresponding to the object recognized by the above-mentioned remaining image frames from the plurality of object point clouds to obtain a plurality of to-be-identified object point clouds.

[0082] In the above embodiment, the point cloud data is clustered to form a plurality of object point clouds, and the point cloud of the recognized object is deleted, so as to obtain the point cloud of the unknown object, i.e. the to-be-identified object point cloud.

[0083] In order to improve the efficiency of the filtering, in an optional embodiment, the device further comprises:

[0084] The second acquisition module is configured to acquire color information of the plurality of to-be-identified object point clouds. The color information is the color of the object corresponding to the to-be-identified object point cloud in the above-mentioned remaining image frames.

[0085] The second deletion module is configured to delete the to-be-identified object point cloud with the predetermined color.

[0086] In the above embodiment, the predetermined color is artificially set, for example, the color in which the obstacle does not exist, so as to delete the to-be-identified object point cloud with the predetermined color, reduce the workload of subsequent filtering, and improve the efficiency.

[0087] In the image filtering device described above, the acquisition unit acquires a video to be filtered; the recognition unit filters multiple image frames in the video to be filtered through image recognition to obtain a first target object image frame, wherein the first target object image frame is an image frame of the video to be filtered containing a target object; the filtering unit filters the first image frame and the second image frame to obtain a second target object image frame, wherein the second target object image frame is either the first image frame or the second image frame of the video to be filtered containing the target object, wherein the first image frame is one or more image frames of the video to be filtered preceding the first target object image frame, and the second image frame is one or more image frames of the video to be filtered following the first target object image frame. This image filtering device uses image recognition to filter image frames containing target objects. Furthermore, based on the continuity of object movement, it filters image frames before and after the image frame containing the target object, thus obtaining image frames that may contain the target object. This avoids missing image frames containing the target object and reduces the proportion of image frames without the target object in the filtered image frame list. Compared to manual image filtering, which often results in a large number of images without the target object, this device significantly improves the filtering effect and solves the problem of poor filtering results in existing technologies. Moreover, this device greatly improves efficiency and saves labor costs compared to manual filtering.

[0088] The image filtering device described above includes a processor and a memory. The acquisition unit, recognition unit, and filtering unit are all stored as program units in the memory, and the processor executes the program units stored in the memory to achieve the corresponding functions.

[0089] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of poor image selection results in existing technologies where manual image selection is ineffective.

[0090] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0091] This invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-described method.

[0092] This invention provides a processor for running a program, wherein the program executes the method described above when it runs.

[0093] The embodiment of the present application provides an image recognition system, comprising an image recognition model, a processor, a memory, and a program stored in the memory and capable of running on the processor, the first target object image frame and the second target object image frame are training samples of the image recognition model, and the processor implements at least the following steps when executing the program:

[0094] In step S101, a video to be screened is acquired.

[0095] In step S102, a plurality of image frames in the video to be screened are screened through image recognition, to obtain a first target object image frame, and the first target object image frame is an image frame of the video to be screened in which a target object exists.

[0096] In step S103, the first image frame and the second image frame are screened, to obtain a second target object image frame, and the second target object image frame is the first image frame or the second image frame of the video to be screened in which the target object exists, the first image frame is one or more image frames of the video to be screened in front of the first target object image frame, and the second image frame is one or more image frames of the video to be screened behind the first target object image frame.

[0097] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program that is initialized with at least the following method steps:

[0098] In step S101, a video to be screened is acquired.

[0099] In step S102, a plurality of image frames in the video to be screened are screened through image recognition, to obtain a first target object image frame, and the first target object image frame is an image frame of the video to be screened in which a target object exists.

[0100] In step S103, the first image frame and the second image frame are screened, to obtain a second target object image frame, and the second target object image frame is the first image frame or the second image frame of the video to be screened in which the target object exists, the first image frame is one or more image frames of the video to be screened in front of the first target object image frame, and the second image frame is one or more image frames of the video to be screened behind the first target object image frame.

[0101] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0102] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented by other manners. Among them, the above-described device embodiments are only illustrative, for example, the division of the above-mentioned units can be a logical function division, and another division manner can be used in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection between the units or modules through some interfaces, and can be electrical or other forms.

[0103] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed to a plurality of units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0104] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0105] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable computer-readable storage medium. Based on this understanding, the technical solutions of the present application or the essential part or all or part of the technical solutions that contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a computer-readable storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the above-mentioned method of each embodiment of the present application. The aforementioned computer-readable storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0106] From the above description, it can be seen that the above-described embodiments of the present application achieve the following technical effects:

[0107] 1) In the image screening method of the present application, first, a video to be screened is obtained; then, a plurality of image frames in the above-mentioned video to be screened are screened through image recognition to obtain a first target object image frame, the above-mentioned first target object image frame is an image frame of the above-mentioned video to be screened that exists a target object; finally, the first image frame and the second image frame are screened to obtain a second target object image frame, the above-mentioned second target object image frame is the above-mentioned first image frame or the above-mentioned second image frame of the above-mentioned video to be screened that exists the above-mentioned target object, the above-mentioned first image frame is one or more image frames of the above-mentioned video to be screened before the above-mentioned first target object image frame, and the above-mentioned second image frame is one or more image frames of the above-mentioned video to be screened after the above-mentioned first target object image frame. The image screening method screens the image frame that exists the target object through image recognition, and according to the continuity of the object motion, screens the image frame before and after the image frame that exists the target object to obtain the image frame that may exist the target object, avoids the omission of the image frame that exists the target object, and reduces the proportion of the image frame that does not exist the target object in the screened image frame. Compared with the image screening by manual screening, there are a large number of images that do not exist the target object, which greatly improves the screening effect, solves the problem of poor screening effect of manual screening of images in the prior art, and greatly improves the efficiency compared with manual screening, thereby saving labor cost.

[0108] 2) In the image screening device of the present application, the acquisition unit acquires a video to be screened; the recognition unit screens a plurality of image frames in the above-mentioned video to be screened through image recognition to obtain a first target object image frame, the above-mentioned first target object image frame is an image frame of the above-mentioned video to be screened that exists a target object; the screening unit screens the first image frame and the second image frame to obtain a second target object image frame, the above-mentioned second target object image frame is the above-mentioned first image frame or the above-mentioned second image frame of the above-mentioned video to be screened that exists the above-mentioned target object, the above-mentioned first image frame is one or more image frames of the above-mentioned video to be screened before the above-mentioned first target object image frame, and the above-mentioned second image frame is one or more image frames of the above-mentioned video to be screened after the above-mentioned first target object image frame. The image screening device screens the image frame that exists the target object through image recognition, and according to the continuity of the object motion, screens the image frame before and after the image frame that exists the target object to obtain the image frame that may exist the target object, avoids the omission of the image frame that exists the target object, and reduces the proportion of the image frame that does not exist the target object in the screened image frame. Compared with the image screening by manual screening, there are a large number of images that do not exist the target object, which greatly improves the screening effect, solves the problem of poor screening effect of manual screening of images in the prior art, and greatly improves the efficiency compared with manual screening, thereby saving labor cost.

[0109] The above descriptions are only the preferred embodiments of the present application, and are not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image filtering method, characterized in that, include: Get the videos to be filtered; Multiple image frames in the video to be screened are screened by image recognition to obtain a first target object image frame, which is an image frame of the video to be screened that contains a target object. The first image frame and the second image frame are filtered to obtain the second target object image frame. The second target object image frame is either the first image frame or the second image frame of the video to be filtered that contains the target object. The first image frame is one or more image frames of the video to be filtered that precede the first target object image frame, and the second image frame is one or more image frames of the video to be filtered that follow the first target object image frame. After filtering the first and second image frames to obtain the second target object image frame, the method further includes: deleting the first and second target object image frames to obtain remaining image frames; acquiring point clouds of unknown objects in the remaining image frames to obtain multiple object point clouds to be identified; and determining the remaining image frames as third target object image frames when there are object point clouds that meet obstacle conditions. The third target object image frames are image frames of the video to be filtered that contain unknown objects. The obstacle conditions include a point cloud volume greater than a predetermined volume, a minimum height of the point cloud from the ground less than a predetermined height, and the existence of corresponding object point clouds in the point cloud data of the previous and next frames of the remaining image frames.

2. The method according to claim 1, characterized in that, The video to be filtered is used to obtain a first target object image frame, including: Extract features of the objects to be identified from the image frames to be screened in the video to be screened, the features including shape features and color features; If the features of the object to be identified are the same as the features of the target object, the image frame to be screened is determined to be the first target object image frame.

3. The method according to claim 2, characterized in that, The first image frame and the second image frame are filtered to obtain the second target object image frame, including: The target object in the first target object image frame is tagged to obtain a tagged first target object image, where the tag is the bounding box of the target object; Starting with the first image of the target object with a label, the target object is tracked in the first image frame and the second image frame using a target tracking algorithm to obtain the first image frame with a label and the second image frame with a label. The first image frame with the label and the second image frame with the label are identified as the second target object image frames.

4. The method according to claim 1, characterized in that, Obtain the point cloud of the unknown object, resulting in point clouds of multiple objects to be identified, including: Obtain the point cloud data corresponding to the remaining image frames to obtain the point cloud data to be analyzed, which includes the point cloud data of all detected objects; Clustering is performed on the point cloud data to be analyzed to obtain multiple object point clouds; Delete the point clouds corresponding to the objects identified in the remaining image frames of the multiple object point clouds to obtain multiple object point clouds to be identified.

5. The method according to claim 1, characterized in that, Before determining the remaining image frame as the third target object image frame, in the case where a point cloud of the object to be identified exists that satisfies the obstacle condition, the method further includes: Obtain color information of multiple point clouds of the objects to be identified, wherein the color information is the color of the object corresponding to the point cloud of the object to be identified in the remaining image frames; Delete the point cloud of the object to be identified whose color information is a predetermined color.

6. An image filtering device, characterized in that, include: The acquisition unit is used to acquire the videos to be filtered. The identification unit is used to filter multiple image frames in the video to be screened through image recognition to obtain a first target object image frame, wherein the first target object image frame is an image frame of the video to be screened that contains a target object; A filtering unit is used to filter a first image frame and a second image frame to obtain a second target object image frame. The second target object image frame is either the first image frame or the second image frame of the video to be filtered that contains the target object. The first image frame is one or more image frames of the video to be filtered that precede the first target object image frame, and the second image frame is one or more image frames of the video to be filtered that follow the first target object image frame. The device further includes: a first deletion module, used to delete the first target object image frame and the second target object image frame to obtain a remaining image frame; a first acquisition module, used to acquire the point cloud of the unknown object in the remaining image frame to obtain multiple object point clouds to be identified; and a third determination module, used to determine the remaining image frame as a third target object image frame when there is an object point cloud that meets the obstacle conditions, wherein the third target object image frame is an image frame of the video to be screened containing an unknown object, and the obstacle conditions include a point cloud volume greater than a predetermined volume, a minimum height of the point cloud from the ground less than a predetermined height, and the point cloud data corresponding to the previous and next frames of the remaining image frame both contain the corresponding object point cloud to be identified.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program performs the method according to any one of claims 1 to 5.

8. A processor, characterized in that, The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 5 when it runs.

9. An image recognition system, characterized in that, include: An image recognition model, one or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any one of claims 1 to 5, wherein the first target object image frame and the second target object image frame are training samples of the image recognition model.

Images