Target object detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application relate to a target object detection method and device, electronic equipment and storage medium. The method comprises: classifying video frames included in a target video to obtain a first target video frame sequence and a similar target video frame sequence; for each similar target video frame in the similar target video frame sequence, determining face information of a similar target object included in the similar target video frame; determining a second target video frame sequence from the similar target video frame sequence according to the face information of each similar target video frame, each target video frame in the second target video frame sequence including a target object; and merging the first target video frame sequence and the second target video frame sequence to obtain a target video frame sequence. Thus, when detecting a target object in a video, the detection accuracy is improved and false detection is avoided.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, apparatus, electronic device and storage medium for detecting a target object. Background Technology

[0002] News segmentation refers to breaking down a news report delivered by a news anchor into independent segments. In practice, news segments can be divided by detecting text content within the news footage. However, the same news segment often has different titles during broadcast, with differences between main and sub-titles, and the news footage also contains a lot of distracting text information, such as dialogue, advertisements, and introductions of people. Therefore, when performing news segmentation, it is also necessary to detect the anchor in the footage to divide the news segments based on the anchor.

[0003] However, existing methods for detecting news anchors often rely on a single model or technique. For example, deep learning classification models are used to categorize input frame images to determine whether they depict a news anchor. While simple, this approach suffers from numerous false positives. For instance, a speaker in a meeting scene might resemble the anchor in both background and clothing, making accurate news anchor detection impossible. Another approach, adding human detection to the classification model, fails to detect human presence in every news scene due to variations in perspective (foreground and background). Furthermore, this method still cannot prevent false positives from interviewing or meeting scenes.

[0004] In summary, existing technologies have poor detection accuracy when detecting news anchors, and there are even cases of false detections. Summary of the Invention

[0005] In view of this, in order to solve the technical problem that the detection accuracy of news anchors is poor or even false detection exists in the existing technology, the present invention provides a method, device, electronic device and storage medium for detecting target objects.

[0006] In a first aspect, embodiments of the present invention provide a method for detecting a target object, the method comprising:

[0007] The video frames included in the target video are classified to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object.

[0008] For each of the similar target video frames in the similar target video frame sequence, determine the facial information of the similar target object included in the similar target video frame;

[0009] Based on the facial information of each of the similar target video frames, a second target video frame sequence is determined from the similar target video frame sequence, wherein each target video frame in the second target video frame sequence includes a target object;

[0010] The first target video frame sequence and the second target video frame sequence are merged to obtain the target video frame sequence.

[0011] As one possible implementation, classifying the video frames included in the target video to obtain a first target video frame sequence and a sequence of similar target video frames includes:

[0012] Each video frame of the target video is sequentially input into a preset classification model to obtain the probability value of each video frame corresponding to a different type output by the classification model.

[0013] Based on the probability values ​​of different types, the target type corresponding to each video frame is determined;

[0014] When the target type is the first type, the corresponding video frame is divided into the first target video frame sequence, where the first type indicates that the video frame includes the target object;

[0015] When the target type is the second type, the corresponding video frame is divided into a sequence of video frames that resemble the target, where the second type indicates that the video frame includes a video frame that resembles the target object.

[0016] As a possible implementation, the target type further includes a third type, which indicates that the video frame does not include the target object, and does not include objects resembling the target object. Determining the target type for each video frame based on probability values ​​of different types includes:

[0017] For each video frame, the maximum probability value is determined from a plurality of probability values ​​of different types;

[0018] If the maximum probability value is greater than or equal to a preset probability value threshold, the type corresponding to the maximum probability value is determined as the target type of the video frame.

[0019] If the maximum probability value is less than the probability threshold, the third type is determined as the target type of the video frame.

[0020] As one possible implementation, the facial information includes facial location information and facial vectors, and determining the facial information of the similar target object included in the similar target video frame includes:

[0021] If the detected target-like video frame includes a face resembling a target object, determine the position information of the detected face bounding box of the target-like video frame, and determine the position information as the face position information of the target object;

[0022] The detected faces resembling the target object are encoded to obtain the face vectors of the target object.

[0023] As one possible implementation, determining the second target video frame sequence from the sequence of similar target video frames based on the facial information of each similar target video frame includes:

[0024] For each of the aforementioned similar target video frames, obtain the frame number of the similar target video frame;

[0025] The frame number, the face location information, and the face vector are used to construct a vector group corresponding to the similar target video frame, thereby obtaining a set of vector groups corresponding to the similar target video frame sequence.

[0026] Based on the set of vector groups, a second target video frame sequence is determined from the similar target video frame sequence.

[0027] As one possible implementation, determining the second target video frame sequence from the similar target video frame sequence based on the vector set includes:

[0028] Based on the set of vector groups, multiple candidate target video frame sequences are determined from the similar target video frame sequences;

[0029] A second target video frame sequence is determined from the plurality of candidate target video frame sequences.

[0030] As one possible implementation, determining multiple candidate target video frame sequences from the similar target video frame sequences based on the vector set includes:

[0031] The vector set is iteratively processed using the following steps until at most one vector set exists in the vector set, or the calculation results obtained by comparing any vector set in the vector set with any other vector set in the vector set do not meet the preset conditions:

[0032] For the set of vector groups obtained in the previous iteration, select any vector group and perform a preset operation on the vector group and each other vector group in the set of vector groups to obtain the calculation result of the vector group for each other vector group.

[0033] If the calculation result meets the preset conditions, the vector group and the other vector groups corresponding to the calculation result are removed from the vector group set, and the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation result are respectively assigned to a candidate target video frame sequence;

[0034] If the calculation result does not meet the preset condition, the other vector groups corresponding to the calculation result will be retained in the vector group set;

[0035] After the iteration, multiple candidate target video frame sequences are obtained.

[0036] As one possible implementation, performing a preset operation on the vector group and each other vector group in the set of vector groups to obtain the calculation result of the vector group for each other vector group includes:

[0037] For each other vector group in the vector group set, calculate the intersection-union ratio of the position information of the vector group with the position information of the other vector groups, and the similarity between the face vector of the vector group and the face vector of the other vector groups;

[0038] When the calculation result meets preset conditions, the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation result are respectively classified into a candidate target video frame sequence, including:

[0039] If the cross-union ratio is greater than a preset cross-union ratio threshold and the similarity is greater than a preset similarity threshold, then the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation results are respectively classified into a candidate target video frame sequence.

[0040] As one possible implementation, the candidate target video frame sequence includes the frame number of each candidate target video frame, and determining the second target video frame sequence from the plurality of candidate target video frame sequences includes:

[0041] For each candidate target video frame sequence, obtain the minimum frame number, the maximum frame number, and the number of candidate target video frames in the candidate target video frame sequence;

[0042] The frame number difference is obtained by subtracting the maximum frame number and the minimum frame number;

[0043] Determine whether the frame number difference is greater than a preset frame number threshold, and determine whether the number is greater than a preset number threshold;

[0044] If the frame number difference is greater than the frame number threshold and the number is greater than the number threshold, the candidate target video frame sequence is determined as the second target video frame sequence.

[0045] Secondly, embodiments of the present invention provide a target object detection device, the device comprising:

[0046] The classification module is used to classify the video frames included in the target video to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object.

[0047] The first determining module is used to determine the facial information of the similar target object included in each similar target video frame in the similar target video frame sequence;

[0048] The second determining module is used to determine a second target video frame sequence from the sequence of similar target video frames based on the facial information of each similar target video frame, wherein each target video frame in the second target video frame sequence includes a target object;

[0049] The merging module is used to merge the first target video frame sequence and the second target video frame sequence to obtain a target video frame sequence.

[0050] As one possible implementation, the classification module includes:

[0051] The input submodule is used to sequentially input each video frame of the target video into a preset classification model to obtain the probability value of each video frame corresponding to a different type output by the classification model.

[0052] The first determination submodule is used to determine the target type corresponding to each video frame based on the probability values ​​of different types;

[0053] The first partitioning submodule is used to partition the corresponding video frame into a first target video frame sequence when the target type is a first type, wherein the first type indicates that the video frame includes a target object;

[0054] The second segmentation submodule is used to segment the corresponding video frame into a sequence of video frames resembling the target when the target type is the second type, wherein the second type indicates that the video frame includes a target object.

[0055] As a possible implementation, the target type further includes a third type, which indicates that the video frame does not include the target object, and does not include objects resembling the target object. The first determining submodule is specifically used for:

[0056] For each video frame, the maximum probability value is determined from a plurality of probability values ​​of different types;

[0057] If the maximum probability value is greater than or equal to a preset probability value threshold, the type corresponding to the maximum probability value is determined as the target type of the video frame.

[0058] If the maximum probability value is less than the probability threshold, the third type is determined as the target type of the video frame.

[0059] As one possible implementation, the facial information includes facial location information and facial vectors, and the first determining module includes:

[0060] The second determining submodule is used to determine the position information of the face frame of the detected similar target video frame when the similar target video frame is detected to include a face of a similar target object, and to determine the position information as the face position information of the similar target object;

[0061] The encoding submodule is used to encode the detected face of the similar target object to obtain the face vector of the similar target object.

[0062] As one possible implementation, the second determining module includes:

[0063] The acquisition submodule is used to acquire the frame number of each of the similar target video frames;

[0064] A construction submodule is used to construct the frame number, the face location information, and the face vector into a vector group corresponding to the similar target video frame, thereby obtaining a set of vector groups corresponding to the similar target video frame sequence.

[0065] The third determining submodule is used to determine the second target video frame sequence from the similar target video frame sequence based on the vector set.

[0066] As one possible implementation, the third determining submodule includes:

[0067] The first determining unit is configured to determine multiple candidate target video frame sequences from the similar target video frame sequences based on the vector set;

[0068] The second determining unit is used to determine a second target video frame sequence from the plurality of candidate target video frame sequences.

[0069] As one possible implementation, the first determining unit includes:

[0070] The operation subunit is used to iteratively process the vector group set using the following steps until there is at most one vector group in the vector group set, or the calculation result obtained by any vector group in the vector group set and any other vector group in the vector group set does not meet the preset condition: for the vector group set obtained in the previous iteration, select any vector group, and perform a preset operation on the vector group and each other vector group in the vector group set to obtain the calculation result of the vector group for each other vector group;

[0071] The elimination subunit is used to remove the vector group and the other vector groups corresponding to the calculation result from the vector group set when the calculation result meets the preset conditions, and to classify the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation result into a candidate target video frame sequence.

[0072] The retention subunit is used to retain the other vector groups corresponding to the calculation result in the vector group set when the calculation result does not meet the preset condition;

[0073] The sub-units are determined to obtain multiple candidate target video frame sequences after the iteration is completed.

[0074] As one possible implementation, the computational subunit is specifically used for:

[0075] For each other vector group in the vector group set, calculate the intersection-union ratio of the position information of the vector group with the position information of the other vector groups, and the similarity between the face vector of the vector group and the face vector of the other vector groups;

[0076] When the calculation result meets preset conditions, the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation result are respectively classified into a candidate target video frame sequence, including:

[0077] If the cross-union ratio is greater than a preset cross-union ratio threshold and the similarity is greater than a preset similarity threshold, then the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation results are respectively classified into a candidate target video frame sequence.

[0078] As one possible implementation, the candidate target video frame sequence includes the frame number of each candidate target video frame, and the second determining unit is specifically used for:

[0079] For each candidate target video frame sequence, obtain the minimum frame number, the maximum frame number, and the number of candidate target video frames in the candidate target video frame sequence;

[0080] The frame number difference is obtained by subtracting the maximum frame number and the minimum frame number;

[0081] Determine whether the frame number difference is greater than a preset frame number threshold, and determine whether the number is greater than a preset number threshold;

[0082] If the frame number difference is greater than the frame number threshold and the number is greater than the number threshold, the candidate target video frame sequence is determined as the second target video frame sequence.

[0083] Thirdly, embodiments of the present invention provide an electronic device, including: a processor and a memory, wherein the processor is configured to execute a target object detection program stored in the memory to implement the target object detection method described in any one of the first aspects.

[0084] Fourthly, embodiments of the present invention provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the target object detection method described in any one aspect.

[0085] The technical solution provided by this invention classifies the videos included in the target video to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object. For each similar target video frame in the similar target video frame sequence, the facial information of the similar target object included in the similar target video frame is determined. Based on the facial information of each similar target video frame, a second target video frame sequence is determined from the similar target video frame sequence. Each target video frame in the second target video frame sequence includes a target object. The first target video frame sequence and the second target video frame sequence are merged to obtain a target video frame sequence. This technical solution first performs preliminary screening of the video frames in the target video to obtain a first target video frame sequence containing the target object and a similar target video frame sequence containing objects resembling the target object. Then, based on the facial information corresponding to each similar target video frame in the similar target video frame sequence, a second target video frame sequence containing the target object is determined from the similar target video frame sequence. This second target video frame sequence is then merged with the first target video frame sequence to obtain the final target video frame sequence containing the target object. This improves detection accuracy and avoids false detections when detecting target objects in video. Attached Figure Description

[0086] Figure 1 A flowchart illustrating an embodiment of a target object detection method provided by this invention;

[0087] Figure 2 A flowchart illustrating an embodiment of another target object detection method provided by this invention;

[0088] Figure 3 A flowchart illustrating another embodiment of a target object detection method provided by this invention;

[0089] Figure 4 A schematic diagram of the flow structure of a target object detection method provided in an embodiment of the present invention;

[0090] Figure 5 This is a schematic diagram of the structure of a face module provided in an embodiment of the present invention;

[0091] Figure 6 A block diagram illustrating an embodiment of a target object detection device provided by this invention;

[0092] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0093] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0094] The method for detecting target objects provided by the present invention will be further explained below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0095] See Figure 1 This is a flowchart illustrating an embodiment of a target object detection method provided by an embodiment of the present invention. Figure 1 As shown, the process may include the following steps:

[0096] Step 101: Classify the video frames included in the target video to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object.

[0097] The aforementioned target video refers to the video to be processed, which can be a news video or other videos. This embodiment of the invention does not impose any restrictions on this.

[0098] The aforementioned target objects refer to the objects included in the pre-defined video, such as news anchors in news videos.

[0099] The aforementioned similar target object refers to an object that resembles the target object. It may be the target object or a non-target object, and it can be further determined.

[0100] The aforementioned first target video frame sequence is a video frame sequence consisting of video frames including the target object.

[0101] The aforementioned target-like video frame sequence is a video frame sequence consisting of video frames that resemble the target object.

[0102] Optionally, the aforementioned first target video frame sequence and the aforementioned similar target video frame sequence may be an image sequence composed of images corresponding to video frames, a frame number sequence composed of frame numbers corresponding to video frames, or an image frame number sequence composed of images and frame numbers corresponding to video frames. This embodiment of the invention does not impose any restrictions on these.

[0103] In one embodiment, the executing entity of this invention can process the target video based on the target objects included in the target video. For example, if the target video is a news video, then processing the target video can involve news segmentation. Optionally, news segmentation can be performed based on the news anchor in the news video.

[0104] In one embodiment, when the executing entity of this invention processes the target video, it may first classify the video frames included in the target video according to the global information of the target video, so as to initially filter out video frames that include the target object and video frames that resemble the target object.

[0105] Specifically, the execution entity of this embodiment may include a preset classification model, which can predict the probability values ​​of different types corresponding to the input video frames. The aforementioned types may be a first type, a second type, or a third type. Optionally, the first type can be used to characterize that the target video includes a target object, the second type can be used to characterize that the target video includes a similar-looking target object, and the third type can be used to characterize that it neither includes the target object nor a similar-looking target object. This embodiment of the invention does not impose any limitations on these aspects.

[0106] Based on this, the execution entity of this embodiment of the invention can sequentially input each video frame of the target video into the above-mentioned classification model to obtain the probability value of each video frame corresponding to a different type output by the classification model. Then, based on the probability values ​​of different types, the target type corresponding to each video frame can be determined.

[0107] Optionally, if the target type is the first type, the corresponding video frame can be assigned to the first target video frame sequence, that is, the video frame including the target object can be assigned to the first target video frame sequence.

[0108] Optionally, if the target type is the second type, the corresponding video frames can be assigned to a sequence of video frames that resemble the target; that is, video frames that resemble the target video can be assigned to a sequence of target video frames.

[0109] In one embodiment, when determining the target type corresponding to each video frame based on different types of probability values, the executing entity of this invention may determine the maximum probability value from multiple different types of probability values ​​for each video frame.

[0110] Optionally, if the maximum probability value is greater than or equal to a preset probability value threshold, the type corresponding to the maximum probability value is determined as the target type of the video frame.

[0111] Conversely, if the maximum probability value is less than the aforementioned probability threshold, the third type can be identified as the target type of the video frame.

[0112] For example, suppose the target video is a news video, the target subject is a news anchor, and the video frames included in the target video can be of the following types: Type 0 to Type 6. Type 0 represents a non-anchor, Type 1 represents a single anchor in a seated position, Type 2 represents two anchors in a seated position, Type 3 represents a single anchor in a standing position, Type 4 represents a long shot / transition of an anchor in a standing position, Type 5 represents a transition of an anchor in a seated position, and Type 6 represents a single anchor in a side view.

[0113] Based on this, if the classification result is type 0, which indicates a non-host scene, then the target type of the corresponding video frame can be determined as the third type; if the classification result is type 1, type 2, type 3, and type 6, then the target type of the corresponding video frame can be determined as the second type; if the classification result is type 4 and type 5, it means that the host in the video frame is small or there is no host, but the scene is special and not easy to be falsely detected, so the target type of the corresponding video frame can be determined as the first type.

[0114] Step 102: For each similar target video frame in the above similar target video frame sequence, determine the facial information of the similar target object included in the similar target video frame.

[0115] The aforementioned video frames resembling targets refer to video frames that include objects resembling targets.

[0116] The aforementioned facial information refers to the facial information of the object resembling the target, which may include facial location information and facial vectors, etc., and the embodiments of the present invention do not limit this.

[0117] In one embodiment, the aforementioned facial information includes facial location information. Based on this, the executing entity of this embodiment can determine the location information of the detected face bounding box of the similar-looking video frame when a similar-looking video frame is detected to include a face resembling a target object, and then determine this location information as the facial location information of the similar-looking target object.

[0118] In another embodiment, the aforementioned facial information includes a face vector. Based on this, the execution entity of this embodiment can encode the detected face resembling the target object when a video frame resembling the target object is detected, thereby obtaining the face vector of the target object.

[0119] In another embodiment, the aforementioned face information may include face location information and face vector. Based on this, the execution subject of this embodiment of the invention can, when detecting that a similar target video frame includes a similar target object, determine the location information of the face bounding box of the detected similar target video frame, assign the location information to the face location information of the similar target object, and encode the face of the detected similar target object to obtain the face vector of the similar target object.

[0120] Furthermore, in order to further reduce false detections and improve the detection accuracy of similar target objects, the execution subject of this embodiment of the invention may first perform face detection on the similar target video frames. If a face similar to the target object is detected, the above-mentioned steps of determining face information are performed. If no face similar to the target object is detected, the target type of the similar target video frame can be classified into the above-mentioned third type to further confirm the similar target video frame sequence.

[0121] Step 103: Based on the facial information of each similar target video frame, determine the second target video frame sequence from the similar target video frame sequence, wherein each target video frame in the second target video frame sequence includes the target object.

[0122] The aforementioned second target video frame sequence is a video frame sequence that includes the target object, determined from a similar target video frame sequence.

[0123] Optionally, the aforementioned second target video frame sequence may be an image sequence composed of images corresponding to video frames, a frame number sequence composed of frame numbers corresponding to video frames, or an image frame number sequence composed of images and frame numbers corresponding to video frames. This embodiment of the invention does not impose any restrictions on this.

[0124] In this embodiment of the invention, the executing entity can further determine a second target video frame sequence from the sequence of similar target video frames based on the facial information corresponding to each similar target video frame, so as to further filter target video frames including the target object from the sequence of similar target video frames.

[0125] As to how the executing entity of this invention specifically determines the second target video frame sequence from the sequence of similar target video frames based on the facial information of each similar target video frame, it can be explained below. Figure 2 The process shown will be explained in detail, but will not be elaborated here.

[0126] Step 104: Merge the first target video frame sequence and the second target video frame sequence to obtain the target video frame sequence.

[0127] The aforementioned target video frame sequence is a sequence consisting of videos including the target object.

[0128] In this embodiment of the invention, as described above, each target video frame in the first target video frame sequence includes a target object, and each target video frame in the second target video frame sequence is a video frame that includes a target object, determined from a sequence of similar target video frames. Therefore, the executing entity of this embodiment can merge the first target video frame sequence and the second target video frame sequence to obtain a target video frame sequence, thereby determining a video frame sequence in the target video that includes a target object.

[0129] The technical solution provided by this invention classifies the videos included in the target video to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object. For each similar target video frame in the similar target video frame sequence, the facial information of the similar target object included in the similar target video frame is determined. Based on the facial information of each similar target video frame, a second target video frame sequence is determined from the similar target video frame sequence. Each target video frame in the second target video frame sequence includes a target object. The first target video frame sequence and the second target video frame sequence are merged to obtain a target video frame sequence. This technical solution first performs preliminary screening of the video frames in the target video to obtain a first target video frame sequence containing the target object and a similar target video frame sequence containing objects resembling the target object. Then, based on the facial information corresponding to each similar target video frame in the similar target video frame sequence, a second target video frame sequence containing the target object is determined from the similar target video frame sequence. This second target video frame sequence is then merged with the first target video frame sequence to obtain the final target video frame sequence containing the target object. This improves detection accuracy and avoids false detections when detecting target objects in video.

[0130] See Figure 2 The above is a flowchart of an embodiment of another target object detection method provided by the present invention. Figure 2 The process shown is in Figure 1 Based on the illustrated process, the text further describes how the second target video frame sequence is determined from the sequence of similar target video frames based on the facial information of each similar target video frame. For example... Figure 2 As shown, the process may include:

[0131] Step 201: For each similar target video frame, obtain the frame number of that similar target video frame.

[0132] Step 202: Construct a vector group corresponding to the target video frame by combining the frame number, face location information, and face vector, and obtain a set of vector groups corresponding to the sequence of target video frames.

[0133] The following provides a unified explanation of steps 201 and 202:

[0134] The aforementioned target-like video frames are video frames that include target-like objects, and correspondingly, the aforementioned target-like video frame sequence is a sequence composed of the aforementioned target-like video frames.

[0135] The frame number mentioned above refers to the frame number of each frame that resembles the target video within the target video.

[0136] Depend on Figure 1 As described in the description, the execution entity of this embodiment can determine the facial information of each resembling target object, which may include facial location information and facial vector. Based on this, the execution entity of this embodiment can construct the above-mentioned frame number, facial location information, and facial vector into a vector group corresponding to the resembling target video frame, thereby obtaining the vector group set corresponding to the above-mentioned resembling target video frame sequence.

[0137] For example, suppose a video frame with a similar shape to the target has a frame number of 3, a face location information of box, and a face vector of V. Then the vector group corresponding to the video frame with the similar shape to the target is (3, box, V).

[0138] Step 203: Determine the second target video frame sequence from the similar target video frame sequence based on the vector set.

[0139] The aforementioned target-like video frame sequence is a sequence consisting of video frames that resemble the target object.

[0140] The aforementioned second target video frame sequence is a sequence consisting of video frames including the target object.

[0141] In one embodiment, the executing entity of this invention may determine a second target video frame sequence from the aforementioned similar target video frame sequence based on the vector group set corresponding to the constructed formal target video frame sequence.

[0142] Specifically, based on the aforementioned set of vector groups, multiple candidate target video frame sequences can be determined from the aforementioned similar target video frame sequences, and a second target video frame sequence can be determined from the multiple aforementioned candidate video frame sequences.

[0143] Furthermore, the execution entity of this embodiment of the invention can determine multiple candidate target video frame sequences from similar target video frame sequences by iteratively processing the vector set.

[0144] For details, please refer to Figure 3 This is a flowchart illustrating another embodiment of a target object detection method provided by the present invention. Figure 3 As shown, the execution subject of this embodiment of the invention can utilize Figure 3 The steps shown iteratively process the set of vectors until a preset iteration stopping condition is met:

[0145] Step 301: For the set of vector groups obtained in the previous iteration, select any vector group and perform a preset operation on the vector group and each other vector group in the set of vector groups to obtain the calculation result of the vector group for each other vector group.

[0146] The above set of vector groups is the set of vector groups obtained in step 202. Optionally, a new set of vector groups can be obtained in each iteration.

[0147] The aforementioned preset operations may refer to intersection-union-comparison (OCU) operations, similarity operations, or a combination of OCU and similarity operations. This embodiment of the invention does not impose any restrictions on these operations.

[0148] In one embodiment, when the executing entity of this invention performs a preset operation on a vector group and each other vector group in the vector group set to obtain the calculation result of the vector group for each other vector group, it can calculate the intersection-union ratio (IUGR) of the position information of any selected vector group with the position information of the other vector group, and the similarity between the face vector of the selected vector group and the face vectors of the other vector groups, for each other vector group in the vector group set. Then, the IUGR and the similarity can be used as the above calculation results.

[0149] Step 302: Determine whether the above calculation result meets the preset conditions. If it does, proceed to step 303; if it does not, proceed to step 304.

[0150] Step 303: If the above calculation results meet the preset conditions, remove the vector group and other vector groups corresponding to the above calculation results from the vector group set, and classify the similar target video frames corresponding to the vector group and other vector groups corresponding to the calculation results into a candidate target video frame sequence.

[0151] Step 304: If the above calculation results do not meet the above preset conditions, other vector groups corresponding to the calculation results can be retained in the vector group set.

[0152] The following provides a unified explanation of steps 302 to 304:

[0153] The aforementioned preset conditions may be that the cross-union ratio is greater than a preset cross-union ratio threshold, and the aforementioned similarity is greater than a preset similarity threshold.

[0154] In one embodiment, the above calculation results can be the intersection-over-union ratio (IoU) and similarity obtained from the above calculations. Based on this, it can be determined whether the above calculation results meet preset conditions. Specifically, if the above calculation results meet preset conditions, that is, if the IoU is greater than a preset IoU threshold and the similarity is greater than a preset similarity threshold, the similar target video frames corresponding to the vector group and other vector groups corresponding to the above calculation results can be classified into a candidate target video frame sequence.

[0155] Based on this, if the above calculation results (the calculated crossover ratio and similarity) do not meet the above preset conditions, that is, if the crossover ratio is less than or equal to the preset crossover ratio threshold, and / or the similarity is less than or equal to the preset similarity threshold, the other vector groups corresponding to the above calculation results can be retained in the vector group set.

[0156] Step 305: Determine whether the iteration stopping condition is met. If it is met, proceed to step 306; otherwise, return to step 301.

[0157] Step 306: Obtain multiple candidate target video frame sequences.

[0158] The following provides a unified explanation of steps 305 and 306:

[0159] The above iteration stopping condition can be that there is at most one vector group in the vector group set, or that the above calculation result obtained by performing the above-mentioned operation on any vector group in the vector group set and any other vector group in the vector group set does not meet the preset condition.

[0160] Based on this, the execution subject of this embodiment of the invention can determine whether the set of vector groups obtained in this iteration satisfies the above-mentioned iteration stopping condition. If the set of vector groups satisfies the above-mentioned iteration stopping condition, multiple candidate target video frame sequences can be finally obtained. If the set of vector groups does not satisfy the above-mentioned iteration stopping condition, the execution can return to continue executing step 301.

[0161] The above completes the task. Figure 3 A detailed description of the process shown is provided by Figure 3 The steps shown can identify multiple candidate target video frame sequences from a sequence of similar-looking target video frames.

[0162] Subsequently, the executing entity of this embodiment of the invention can determine the second target video frame sequence from the plurality of candidate target video frame sequences.

[0163] Specifically, the aforementioned candidate target video frame sequence can be a frame number sequence consisting of the frame number corresponding to each candidate target video frame, or an image frame number sequence consisting of the image and frame number corresponding to each candidate video frame, which may include the frame number of each candidate target video frame. Based on this, the execution subject of this embodiment of the invention can obtain the minimum frame number, the maximum frame number, and the number of candidate target video frames in each candidate target video frame sequence.

[0164] Then, the difference between the maximum and minimum frame numbers can be calculated to obtain the frame number difference value. It can then be determined whether the frame number difference value is greater than a preset frame number threshold, and whether the number of differences is greater than a preset number threshold.

[0165] Optionally, if the frame number difference is greater than the frame number threshold and the number is greater than the number threshold, the candidate target video frame sequence is determined as the second target video frame sequence.

[0166] For example, assuming the target video is a news video, the above process can yield a sequence of resembling target video frames containing images of a news anchor, and a corresponding set of vectors {(idx1, box1, V1), (idx2, box2, V2)...(idxn, boxn, Vn)}. Then, the second target video frame sequence can be determined from the resembling target video frame sequence using the following steps:

[0167] (1) Obtain the first vector group (idx1, box1, V1) and compare it with the second to nth vector groups: calculate the intersection-union ratio between the boxes. Next, calculate the similarity s between the vectors, where s can be the Euclidean distance. It can also be the cosine distance.

[0168] (2) If IOU>=th2 and s>=th3, then the images of the idx1 frame and the idxi frame are considered as a set of similar candidate host results; where th2 and th3 are preset thresholds respectively.

[0169] (3) After the traversal is completed, the image of frame idx1 will yield a set of candidate host results (idx1, idxi, ...).

[0170] (4) Skip the similar host results that have been selected in the above process, take the next (idxj, boxj, Vj) and continue to traverse, and so on, until k similar candidate host results are obtained.

[0171] (5) For each similar candidate host result, obtain the minimum frame number idx_min and the maximum frame number idx_max, calculate the difference between the two. If it is greater than the threshold th4 and the number of candidate host results is greater than the threshold th5, then this group of candidate host results is considered to be the correct host result; otherwise, it is considered to be a false detection and is filtered.

[0172] In practical applications, the presenter should appear at least t52 times in the entire result and should be relatively evenly distributed throughout the time series. The earliest and latest time of appearance should span at least th4. Furthermore, the correct presenter results should not be concentrated in certain segments. Therefore, step 5 above is to filter out those falsely detected presenters.

[0173] The technical solution provided by this invention obtains the frame number of each similar-looking target video frame, constructs a vector group corresponding to the similar-looking target video frame using the frame number, face location information, and face vector, and obtains a set of vector groups corresponding to the similar-looking target video frame sequence. Based on the set of vector groups, a second target video frame sequence is determined from the similar-looking target video frame sequence. This technical solution constructs a vector group corresponding to each similar-looking target video frame using the frame number, face location information, and face vector, and determines the second target video frame sequence from the similar-looking target video frame sequence based on the vector group. By utilizing face location information and face vectors, it can more accurately determine whether the similar-looking target video frames in the similar-looking target video frame sequence are target video frames containing the target object. This further determines whether the similar-looking target object included in the similar-looking target video frame is the target object, thereby improving detection accuracy and avoiding false detections when detecting target objects in target videos.

[0174] See Figure 4 This is a schematic diagram of the flow structure of a target object detection method provided in an embodiment of the present invention. Figure 4 As shown, the process structure can include three modules: classification module C, face module F, and logic judgment module L.

[0175] The classification module C outputs two parts: category 0 (non-host) is discarded, and categories 4 and 5 are directly saved as the host detection result (result1). Categories 1, 2, 3, and 6 are used as input to the face module F. The logic judgment module L is only called and executed after all input frames have passed through the classification module C and the face module F. It outputs the filtered host detection result (result2), which is then merged with the result1 output from the classification module C to form the final host detection result (result). In practice, to reduce the detection frequency and save resources, only one frame per second is sampled as input.

[0176] In one embodiment, image I is input into classification module C, which outputs seven category probabilities. The image with the highest probability that satisfies a threshold greater than th1 is the classification result of image I. If the highest probability value is less than th1, it is directly classified as category 0, where th1 is a preset threshold. These seven categories are: 0 Non-host, 1 Single host sitting posture, 2 Two hosts sitting posture, 3 Single host standing posture, 4 Host standing posture distant view / host standing posture transition, 5 Host sitting posture transition, 6 Single host side posture. If the classification result is 0, it indicates a non-host image and is discarded directly. If the classification result is 1, 2, 3, or 6, image I is input to subsequent modules for further filtering and judgment. If the classification result is 4 or 5, as mentioned above, these images have a small or no host, but the image is special and not easily misdetected, so they are directly retained as the host detection result result1.

[0177] In one embodiment, the face module may include two parts: face detection D and face encoding E. For example... Figure 5 The diagram shown is a structural schematic of a face module provided in an embodiment of the present invention. Figure 5 As shown, the image I that module C determines needs further filtering is used as the input of this module. The specific process is as follows:

[0178] (1) Image I is processed by face detection D to detect faces in the input image.

[0179] (2) If no face is detected, image I is determined to be non-host and discarded directly.

[0180] (3) If a face is detected, the face region is input into the face code E to obtain a vector V with dimensions (1, n);

[0181] (4) The frame number idx, the position of each detected face box box box and the vector V are combined to form a vector group (idx, box, V), which is used as the input of the subsequent logic judgment module.

[0182] In one embodiment, the logic judgment module L is as follows: After all input images I have passed through the above steps, all images classified as categories 4 and 5 will be directly retained in result1. All images classified as categories 1, 2, 3, and 6, after passing through module F, will have some filtered out because no faces were detected, and the rest will obtain n positions and vector groups (idx, box, V). This module will filter these n (idx, box, V) as follows:

[0183] (1) Obtain the first vector group (idx1, box1, V1) and compare it with the second to nth vector groups: calculate the intersection-union ratio between the boxes. Next, calculate the similarity s between the vectors, where s can be the Euclidean distance. It can also be the cosine distance.

[0184] (2) If IOU>=th2 and s>=th3, then the images of the idx1 frame and the idxi frame are considered as a set of similar candidate host results; where th2 and th3 are preset thresholds respectively.

[0185] (3) After the traversal is completed, the image of frame idx1 will yield a set of candidate host results (idx1, idxi, ...).

[0186] (4) Skip the similar host results that have been selected in the above process, take the next (idxj, boxj, Vj) and continue to traverse, and so on, until k similar candidate host results are obtained.

[0187] (5) For each similar candidate host result, obtain the minimum frame number idx_min and the maximum frame number idx_max, calculate the difference between the two. If it is greater than the threshold th4 and the number of candidate host results is greater than the threshold th5, then this group of candidate host results is considered to be the correct host result; otherwise, it is considered to be a false detection and is filtered.

[0188] In practical applications, the host should appear at least th5 times in the entire result and should be relatively evenly distributed throughout the time series. The earliest and latest appearance times should span at least th4. Furthermore, the correct host results should not be concentrated in certain segments. Therefore, step 5 above is to filter out those falsely detected hosts.

[0189] This invention proposes a comprehensive judgment and filtering scheme to address the news anchor detection needs in actual projects. It integrates global information, position information, and facial feature information of the anchor in the screen, which greatly improves the accuracy of anchor detection under the single-model technology solution and meets the accuracy requirements of online production.

[0190] See Figure 6 This is a block diagram of an embodiment of a target object detection device provided by an embodiment of the present invention. Figure 6 As shown, the device includes:

[0191] The classification module 61 is used to classify the video frames included in the target video to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object.

[0192] The first determining module 62 is used to determine the facial information of the similar target object included in each similar target video frame in the similar target video frame sequence;

[0193] The second determining module 63 is used to determine a second target video frame sequence from the sequence of similar target video frames based on the face information of each similar target video frame, wherein each target video frame in the second target video frame sequence includes a target object;

[0194] The merging module 64 is used to merge the first target video frame sequence and the second target video frame sequence to obtain a target video frame sequence.

[0195] As one possible implementation, the classification module 61 includes (not shown in the figure):

[0196] The input submodule is used to sequentially input each video frame of the target video into a preset classification model to obtain the probability value of each video frame corresponding to a different type output by the classification model.

[0197] The first determination submodule is used to determine the target type corresponding to each video frame based on the probability values ​​of different types;

[0198] The first partitioning submodule is used to partition the corresponding video frame into a first target video frame sequence when the target type is a first type, wherein the first type indicates that the video frame includes a target object;

[0199] The second segmentation submodule is used to segment the corresponding video frame into a sequence of video frames resembling the target when the target type is the second type, wherein the second type indicates that the video frame includes a target object.

[0200] As a possible implementation, the target type further includes a third type, which indicates that the video frame does not include the target object, and does not include objects resembling the target object. The first determining submodule is specifically used for:

[0201] For each video frame, the maximum probability value is determined from a plurality of probability values ​​of different types;

[0202] If the maximum probability value is greater than or equal to a preset probability value threshold, the type corresponding to the maximum probability value is determined as the target type of the video frame.

[0203] If the maximum probability value is less than the probability threshold, the third type is determined as the target type of the video frame.

[0204] As one possible implementation, the face information includes face location information and face vector, and the first determining module 62 includes:

[0205] The second determining submodule is used to determine the position information of the face frame of the detected similar target video frame when the similar target video frame is detected to include a face of a similar target object, and to determine the position information as the face position information of the similar target object;

[0206] The encoding submodule is used to encode the detected face of the similar target object to obtain the face vector of the similar target object.

[0207] As one possible implementation, the second determining module 63 includes:

[0208] The acquisition submodule is used to acquire the frame number of each of the similar target video frames;

[0209] A construction submodule is used to construct the frame number, the face location information, and the face vector into a vector group corresponding to the similar target video frame, thereby obtaining a set of vector groups corresponding to the similar target video frame sequence.

[0210] The third determining submodule is used to determine the second target video frame sequence from the similar target video frame sequence based on the vector set.

[0211] As one possible implementation, the third determining submodule includes:

[0212] The first determining unit is configured to determine multiple candidate target video frame sequences from the similar target video frame sequences based on the vector set;

[0213] The second determining unit is used to determine a second target video frame sequence from the plurality of candidate target video frame sequences.

[0214] As one possible implementation, the first determining unit includes:

[0215] The operation subunit is used to iteratively process the vector group set using the following steps until there is at most one vector group in the vector group set, or the calculation result obtained by any vector group in the vector group set and any other vector group in the vector group set does not meet the preset condition: for the vector group set obtained in the previous iteration, select any vector group, and perform a preset operation on the vector group and each other vector group in the vector group set to obtain the calculation result of the vector group for each other vector group;

[0216] The elimination subunit is used to remove the vector group and the other vector groups corresponding to the calculation result from the vector group set when the calculation result meets the preset conditions, and to classify the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation result into a candidate target video frame sequence.

[0217] The retention subunit is used to retain the other vector groups corresponding to the calculation result in the vector group set when the calculation result does not meet the preset condition;

[0218] The sub-units are determined to obtain multiple candidate target video frame sequences after the iteration is completed.

[0219] As one possible implementation, the computational subunit is specifically used for:

[0220] For each other vector group in the vector group set, calculate the intersection-union ratio of the position information of the vector group with the position information of the other vector groups, and the similarity between the face vector of the vector group and the face vector of the other vector groups;

[0221] When the calculation result meets preset conditions, the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation result are respectively classified into a candidate target video frame sequence, including:

[0222] If the cross-union ratio is greater than a preset cross-union ratio threshold and the similarity is greater than a preset similarity threshold, then the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation results are respectively classified into a candidate target video frame sequence.

[0223] As one possible implementation, the candidate target video frame sequence includes the frame number of each candidate target video frame, and the second determining unit is specifically used for:

[0224] For each candidate target video frame sequence, obtain the minimum frame number, the maximum frame number, and the number of candidate target video frames in the candidate target video frame sequence;

[0225] The frame number difference is obtained by subtracting the maximum frame number and the minimum frame number;

[0226] Determine whether the frame number difference is greater than a preset frame number threshold, and determine whether the number is greater than a preset number threshold;

[0227] If the frame number difference is greater than the frame number threshold and the number is greater than the number threshold, the candidate target video frame sequence is determined as the second target video frame sequence.

[0228] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 7 The illustrated electronic device 700 includes at least one processor 701, a memory 702, at least one network interface 704, and a user interface 703. The various components in the electronic device 700 are coupled together via a bus system 705. It is understood that the bus system 705 is used to implement communication between these components. In addition to a data bus, the bus system 705 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 7 The general labeled all buses as Bus System 705.

[0229] The user interface 703 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0230] It is understood that the memory 702 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 702 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0231] In some implementations, memory 702 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 7021 and application program 7022.

[0232] The operating system 7021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 7022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 7022.

[0233] In this embodiment of the invention, by calling the program or instructions stored in the memory 702, specifically the program or instructions stored in the application program 7022, the processor 701 executes the method steps provided in each method embodiment, including, for example:

[0234] The video frames included in the target video are classified to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object.

[0235] For each of the similar target video frames in the similar target video frame sequence, determine the facial information of the similar target object included in the similar target video frame;

[0236] Based on the facial information of each of the similar target video frames, a second target video frame sequence is determined from the similar target video frame sequence, wherein each target video frame in the second target video frame sequence includes a target object;

[0237] The first target video frame sequence and the second target video frame sequence are merged to obtain the target video frame sequence.

[0238] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in software form. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 702. Processor 701 reads the information in memory 702 and, in conjunction with its hardware, completes the steps of the above method.

[0239] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0240] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0241] The electronic device provided in this embodiment may be as follows: Figure 7 The electronic device shown can perform the following: Figure 1-3 All steps of the target object detection method are implemented to achieve... Figure 1-3 For details on the technical effectiveness of the target object detection method, please refer to [link / reference]. Figures 1-3 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0242] This invention also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory may also include combinations of the above types of memory.

[0243] When one or more programs in the storage medium can be executed by one or more processors to implement the target object detection method executed on the electronic device side.

[0244] The processor is used to execute a target object detection method stored in the memory to implement the following steps of the target object detection method executed on the electronic device side:

[0245] The video frames included in the target video are classified to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object.

[0246] For each of the similar target video frames in the similar target video frame sequence, determine the facial information of the similar target object included in the similar target video frame;

[0247] Based on the facial information of each of the similar target video frames, a second target video frame sequence is determined from the similar target video frame sequence, wherein each target video frame in the second target video frame sequence includes a target object;

[0248] The first target video frame sequence and the second target video frame sequence are merged to obtain the target video frame sequence.

[0249] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0250] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0251] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of detecting a target object, characterized by, The method includes: The video frames included in the target video are classified to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object. For each of the similar target video frames in the similar target video frame sequence, determine the facial information of the similar target object included in the similar target video frame; Based on the facial information of each of the similar target video frames, a second target video frame sequence is determined from the similar target video frame sequence, wherein each target video frame in the second target video frame sequence includes a target object; The first target video frame sequence and the second target video frame sequence are merged to obtain the target video frame sequence; The step of classifying the video frames included in the target video to obtain a first target video frame sequence and a sequence of similar target video frames includes: Each video frame of the target video is sequentially input into a preset classification model to obtain the probability value of each video frame corresponding to a different type output by the classification model. Based on the probability values ​​of different types, the target type corresponding to each video frame is determined; When the target type is the first type, the corresponding video frame is divided into the first target video frame sequence, where the first type indicates that the video frame includes the target object; When the target type is the second type, the corresponding video frame is divided into a sequence of video frames that resemble the target, where the second type indicates that the video frame includes a video frame that resembles the target object.

2. The method of claim 1, wherein, The target type also includes a third type, which indicates that the video frame does not include the target object, and does not include objects resembling the target object. Determining the target type for each video frame based on the probability values ​​of different types includes: For each video frame, the maximum probability value is determined from a plurality of probability values ​​of different types; If the maximum probability value is greater than or equal to a preset probability value threshold, the type corresponding to the maximum probability value is determined as the target type of the video frame. If the maximum probability value is less than the probability threshold, the third type is determined as the target type of the video frame.

3. The method of claim 1, wherein, The facial information includes facial location information and facial vectors. Determining the facial information of the similar-looking target object included in the similar-looking target video frame includes: If the detected target-like video frame includes a face resembling a target object, determine the position information of the detected face bounding box of the target-like video frame, and determine the position information as the face position information of the target object; The detected faces resembling the target object are encoded to obtain the face vectors of the target object.

4. The method of claim 3, wherein, The step of determining a second target video frame sequence from the sequence of similar target video frames based on the facial information of each similar target video frame includes: For each of the aforementioned similar target video frames, obtain the frame number of the similar target video frame; The frame number, the face location information, and the face vector are used to construct a vector group corresponding to the similar target video frame, thereby obtaining a set of vector groups corresponding to the similar target video frame sequence. Based on the set of vector groups, a second target video frame sequence is determined from the similar target video frame sequence.

5. The method of claim 4, wherein, The step of determining the second target video frame sequence from the similar target video frame sequence based on the vector set includes: Based on the set of vector groups, multiple candidate target video frame sequences are determined from the similar target video frame sequences; A second target video frame sequence is determined from the plurality of candidate target video frame sequences.

6. The method of claim 5, wherein, The step of determining multiple candidate target video frame sequences from the similar target video frame sequences based on the vector set includes: The vector set is iteratively processed using the following steps until at most one vector set exists in the vector set, or the calculation results obtained by comparing any vector set in the vector set with any other vector set in the vector set do not meet the preset conditions: For the set of vector groups obtained in the previous iteration, select any vector group and perform a preset operation on the vector group and each other vector group in the set of vector groups to obtain the calculation result of the vector group for each other vector group. If the calculation result meets the preset conditions, the vector group and the other vector groups corresponding to the calculation result are removed from the vector group set, and the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation result are respectively assigned to a candidate target video frame sequence; If the calculation result does not meet the preset condition, the other vector groups corresponding to the calculation result will be retained in the vector group set; After the iteration, multiple candidate target video frame sequences are obtained.

7. The method according to claim 6, characterized in that, The step of performing a preset operation on the vector group and each other vector group in the set of vector groups to obtain the calculation result of the vector group for each other vector group includes: For each other vector group in the vector group set, calculate the intersection-union ratio of the position information of the vector group with the position information of the other vector groups, and the similarity between the face vector of the vector group and the face vector of the other vector groups; When the calculation result meets preset conditions, the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation result are respectively classified into a candidate target video frame sequence, including: If the cross-union ratio is greater than a preset cross-union ratio threshold and the similarity is greater than a preset similarity threshold, then the similar target video frames corresponding to the vector group and the other vector groups corresponding to the calculation results are respectively classified into a candidate target video frame sequence.

8. The method according to claim 5, characterized in that, The candidate target video frame sequence includes the frame number of each candidate target video frame, and determining the second target video frame sequence from the plurality of candidate target video frame sequences includes: For each candidate target video frame sequence, obtain the minimum frame number, the maximum frame number, and the number of candidate target video frames in the candidate target video frame sequence; The frame number difference is obtained by subtracting the maximum frame number and the minimum frame number; Determine whether the frame number difference is greater than a preset frame number threshold, and determine whether the number is greater than a preset number threshold; If the frame number difference is greater than the frame number threshold and the number is greater than the number threshold, the candidate target video frame sequence is determined as the second target video frame sequence.

9. A target object detection device, characterized in that, The device includes: The classification module is used to classify the video frames included in the target video to obtain a first target video frame sequence and a similar target video frame sequence. Each target video frame in the first target video frame sequence includes a target object, and each similar target video frame in the similar target video frame sequence includes a similar target object. The first determining module is used to determine the facial information of the similar target object included in each similar target video frame in the similar target video frame sequence; The second determining module is used to determine a second target video frame sequence from the sequence of similar target video frames based on the facial information of each similar target video frame, wherein each target video frame in the second target video frame sequence includes a target object; The merging module is used to merge the first target video frame sequence and the second target video frame sequence to obtain a target video frame sequence; The step of classifying the video frames included in the target video to obtain a first target video frame sequence and a sequence of similar target video frames includes: Each video frame of the target video is sequentially input into a preset classification model to obtain the probability value of each video frame corresponding to a different type output by the classification model. Based on the probability values ​​of different types, the target type corresponding to each video frame is determined; When the target type is the first type, the corresponding video frame is divided into the first target video frame sequence, where the first type indicates that the video frame includes the target object; When the target type is the second type, the corresponding video frame is divided into a sequence of video frames that resemble the target, where the second type indicates that the video frame includes a video frame that resembles the target object.

10. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a target object detection program stored in the memory to implement the target object detection method according to any one of claims 1 to 8.

11. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the target object detection method according to any one of claims 1 to 8.

Images