Method and device for determining snapshot target

Abstract

The invention discloses a snapshot target determination method and device, and the method comprises the steps: determining a target image in a target video, the target image comprises a first target object, and the first target object is an object with the highest definition in a first object set obtained through the recognition of the target video; according to a target association relationship, unrecognized objects in the target image are determined, a target object set is obtained, and the target association relationship records the association relationship among the objects appearing in the target video; and determining the first target object and the object in the target object set as a snapshot target of the target image. Through application of the method and the device, the problem of target snapshot omission during target snapshot in related technologies is solved, and the effect of avoiding target snapshot omission is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the computer field, in particular, to a determination method and device for a snapshot target. BACKGROUND

[0002] An original video image is a kind of unstructured data, as an imaging technology, it can only assist in viewing work, and its content cannot be directly read and recognized by a computer. Video structuring is a video content information extraction technology, including intelligent algorithm analysis such as detection and tracking of video content, target snapshot, attribute recognition, etc. The video content is organized into text information that can be understood by a computer and a person, and is further converted into content required by actual application, becoming structured data that can be searched through certain conditions.

[0003] In the process of target snapshot analysis of a video image, the existing snapshot module relies on a preposed detection and quality scoring module. Due to the complexity and diversity of real scenes, there are often some low-quality targets in the video image, that is, targets with low quality scores, such as occluded / overlapping targets, edge targets, blurred / low-illumination targets, etc., which leads to the fact that the snapshot module cannot effectively capture low-quality targets, resulting in missed snapshot targets, and further affecting the actual target snapshot effect.

[0004] At present, there is no effective solution to the above problems. SUMMARY

[0005] Embodiments of the present application provide a determination method and device for a snapshot target, to at least solve the technical problem of missed snapshot targets when a target is snapped in the related art.

[0006] According to an aspect of an embodiment of the present application, a determination method for a snapshot target is provided, including: determining a target image in a target video, wherein the target image includes a first target object, and the first target object is an object with the highest clarity in a first object set obtained by identifying the target video; determining an object in the target image that is not recognized according to a target association relationship, to obtain a target object set, wherein the target association relationship records an association relationship between objects appearing in the target video; and determining the first target object and the objects in the target object set as snapshot targets of the target image.

[0007] In an example embodiment, the target image in the target video is determined, including: identifying a first object in the target video to obtain a first object set; determining an image containing any first object in the first object set in the target video as a first image to obtain a first image set; determining a first object with the highest clarity in the first image set as the first target object; and determining an image in which the first target object is located as the target image.

[0008] In an exemplary embodiment, determining unidentified objects in a target image based on target association relationships to obtain a target object set includes: determining the types of objects in the target object set to obtain target types; and determining the target object set based on target types and target association relationships.

[0009] In an exemplary embodiment, determining the target object set based on the target type and target association includes: determining objects in the target association that have an association with the first target object and whose type is the target type, to obtain a second object set; and determining the target object set based on the identifiers of each object in the second object set.

[0010] In an exemplary embodiment, determining the target object set based on the identifiers of each object in the second object set includes: determining the identifiers of each object in the second object set to obtain N first identifiers, where N is an integer greater than 0; determining objects in the target image whose type is the target type and whose identifier is a first identifier to obtain a third object set; and determining the target object set through the third object set.

[0011] In one exemplary embodiment, determining the target object set through the third object set includes: determining objects in the third object set as objects in the target object set; or, in the case that the first target object includes multiple sub-objects, determining sub-objects of type target type as objects in the fourth object set; and matching each object in the fourth object set with each object in the third object set to obtain the target object set.

[0012] In an exemplary embodiment, matching each object in the fourth object set with each object in the third object set to obtain a target object set includes: determining the overlap between the i-th object in the fourth object set and each object in the third object set to obtain multiple first overlaps, where i is an integer greater than 0; determining the object in the third object set corresponding to the first overlap with a first preset overlap threshold as an object in the target object set; or, if multiple first overlaps are all less than the first preset overlap threshold, determining the i-th object as an object in the fifth object set; and determining the target object set from the fifth object set.

[0013] In one exemplary embodiment, determining the target object set from the fifth object set includes: obtaining objects of the target type from the target image to obtain a sixth object set; and matching each object in the fifth object set with each object in the sixth object set to obtain the target object set.

[0014] In an exemplary embodiment, matching each object in the fifth object set with each object in the sixth object set to obtain a target object set includes: determining the overlap between the j-th object in the fifth object set and each object in the sixth object set to obtain multiple second overlaps, where j is an integer greater than 0; determining the identifier of the object corresponding to the second overlap in the sixth object set that is greater than or equal to a second preset overlap threshold as a third identifier; and determining the target object set through the third identifier.

[0015] In an exemplary embodiment, determining the target object set by a third identifier includes: determining the identifiers of objects in the target association relationship that are associated with the first target object, to obtain a target identifier set; and matching the third identifier with the target identifier set to obtain the target object set.

[0016] In an exemplary embodiment, matching the third identifier with the target identifier set to obtain a target object set includes: if the third identifier does not exist in the target identifier set, determining the j-th object as an object in the target object set; or, if the third identifier exists in the target identifier set, determining objects associated with the first target object in the target association relationships to obtain a seventh object set; if there are no objects of the target type among the objects identified by the third identifier in the seventh object set, determining the j-th object as the target object set.

[0017] According to another aspect of the embodiments of this application, a device for determining a capture target is also provided, comprising: a first determining module, configured to determine a target image in a target video, wherein the target image includes a first target object, the first target object being the object with the highest clarity in a first object set obtained by recognizing the target video; a second determining module, configured to determine unrecognized objects in the target image based on target association relationships, thereby obtaining a target object set, wherein the target association relationships record the association relationships between various objects appearing in the target video; and a third determining module, configured to determine the first target object and the objects in the target object set as capture targets of the target image.

[0018] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed by a processor.

[0019] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in any of the method embodiments described above.

[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to perform the steps of any of the above method embodiments through the computer program.

[0021] This application identifies a first target object and a target image containing the first target object in a target video, and then determines unidentified objects in the target image based on target association relationships to obtain a target object set, thereby identifying the first target object and the objects in the target object set as the capture targets of the target image.

[0022] By analyzing the target video's object relationships, the system infers potentially missed objects, identifies and supplements these objects, and ultimately determines the target image for capture. This effectively avoids situations where certain objects (such as low-quality targets) may not be fully identified in the target image due to occlusion, lighting conditions, or limitations of detection algorithms in video analysis. This reduces information loss caused by missed targets, improving capture accuracy and recall. Therefore, it solves the problem of missed targets in related technologies, achieving the effect of avoiding missed captures. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of a video structuring scheme based on snapshot according to an embodiment of this application;

[0024] Figure 2 This is a schematic diagram of a video structuring optimization scheme based on snapshot according to an embodiment of this application;

[0025] Figure 3 This is a hardware structure block diagram of a mobile terminal for a target determination method according to an embodiment of this application;

[0026] Figure 4 This is a flowchart of a method for determining a target to be captured according to an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the relationship according to the embodiments of this application;

[0028] Figure 6This is a flowchart of the false alarm database construction process according to an embodiment of this application;

[0029] Figure 7 This is a flowchart of a false alarm filtering process according to an embodiment of this application;

[0030] Figure 8 This is a flowchart illustrating the establishment of target association relationships according to embodiments of this application;

[0031] Figure 9 This is a flowchart illustrating the determination of supplementary targets for the target to be captured, according to an embodiment of this application.

[0032] Figure 10 This is a structural block diagram of a target detection device according to an embodiment of this application. Detailed Implementation

[0033] Raw video images are unstructured data, and as an imaging technique, they can only be used for auxiliary viewing; their content cannot be directly read and recognized by computers. Video structuring is a video content information extraction technology that uses intelligent algorithms such as detection and tracking, target capture, and attribute recognition to analyze video images, organizing them into text information that can be understood by computers and humans. This information is then further transformed into content needed for practical applications, making video images structured data that can be searched based on certain criteria, thereby greatly improving the usability and application value of video data.

[0034] However, existing video structuring solutions primarily rely on deep learning-based detection and quality scoring modules. For example, Figure 1 This is a schematic diagram of a video structuring scheme based on snapshot according to an embodiment of this application, such as... Figure 1 As shown, the structure of the video structuring scheme based on snapshot mainly includes the following functional modules:

[0035] The detection module is mainly used to process video frames using deep learning technology, perform target detection, detect the target location information in the picture, and generate an independent target ID for each target using a tracking algorithm. The same target is kept in the same target ID in consecutive frames so that subsequent modules can always use the target information of the same target when processing consecutive video frames.

[0036] Quality scoring module: It is mainly used to learn the image quality of the target area in the current video frame and various target-specific factors, such as pose (direction) and occlusion, by using a global learning method. This allows subsequent modules to use the quality score as one of the criteria for using the target.

[0037] Capture module: It is mainly used to output the frame with the highest quality score within the target's life cycle. By adopting a dynamic update method, it uses the target quality score as the judgment basis and only records the frame with the highest quality score when the target appears to the current frame. If a higher score appears later, the frame corresponding to the higher score is updated.

[0038] As described above, in the video structured processing workflow, the target capture module typically relies on the outputs of the detection module and the quality scoring module as the basis for judgment, capturing only the image frames with the highest quality scores. While this mechanism can guarantee high capture quality in ideal scenarios, in complex real-world scenarios, it may overlook low-quality targets in the image, such as occluded / overlapping targets, edge targets, blurred / low-light targets, and non-realistic targets such as shadows, photographs, and reflective targets. This inevitably leads to the missed capture of low-quality targets, affecting the target capture effect.

[0039] Therefore, this application adds an associated target submodule and a false alarm filtering submodule to the existing scheme structure, specifically as follows: Figure 2 As shown, by leveraging the relationships between various objects in the video, missed targets can be optimized, low-quality targets that may be missed can be identified, and supplementary targets can be generated, thus solving the problem of missed low-quality targets in existing technologies.

[0040] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0041] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0042] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 3 This is a hardware structure block diagram of a mobile terminal for determining the target to be captured according to an embodiment of this application. For example... Figure 3 As shown, a mobile terminal may include one or more ( Figure 3 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 3 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown.

[0043] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the target determination method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0044] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0045] This embodiment provides a method for determining the target to be captured on the aforementioned mobile terminal or network architecture. Figure 4 This is a flowchart of a method for determining a target to be captured according to an embodiment of this application, such as...Figure 4 As shown, the process includes the following steps:

[0046] Step S402: Determine a target image in the target video, wherein the target image includes a first target object, which is the object with the highest clarity in the first set of objects obtained by recognizing the target video;

[0047] The target video mentioned above can be a video segment that the system needs to analyze. This video segment can originate from security monitoring cameras, dashcams, drone footage, etc. The target video consists of multiple video images (i.e., video frames), and each video image can contain multiple objects. The objects contained in different video images can be the same or different.

[0048] The aforementioned first target object can be the object considered optimal among multiple detection objects in the target video based on a certain evaluation criterion (such as clarity). For example, when identifying a vehicle A in the target video, if vehicle A appears in 5 video images in the target video, there are 5 detection objects A1-A5 in the target video. If detection object A3 is the object with the highest score (such as the clearest) among these 5 detection objects, then detection object A3 is taken as the identification result of vehicle A, that is, the first target object.

[0049] The aforementioned target image can be a video image containing the first target object in the target video. For example, in the recognition of vehicle A, the detected object A3 is the recognition result of vehicle A, that is, the first target object output. At this time, the video image where A3 is located will be output as the target image.

[0050] When capturing a target based on a target video, the first target object and the target image where the first target object is located are obtained from the capture module based on the target video.

[0051] As an optional implementation, determining a target image in a target video includes: identifying a first object in the target video to obtain a first object set; determining a first image from an image in the target video that contains any first object from the first object set to obtain a first image set; determining the first object with the highest clarity in the first image set as a first target object; and determining the image containing the first target object as a target image.

[0052] The first object mentioned above can be an object that needs to be identified or captured, such as a person or vehicle; the set of the first objects mentioned above can be a set of all the first objects that have appeared in the target video; the set of the first images mentioned above can be a set of images of each first object in the target video; the first objects in the target video are identified, and video images containing the first objects in the target video are determined to obtain the first image set. The object with the highest clarity in the first image set is determined as the first target object, and the video image where the first target object is located is determined as the target image.

[0053] For example, if the target video is surveillance footage from a city street, the system attempts to capture clear images of faces. When analyzing each frame of the video, the system identifies multiple possible facial targets (i.e., the first set of objects), ranks these faces according to their clarity, and ultimately selects the image containing the face with the highest clarity (i.e., the first target object) as the target image. This ensures that subsequent face recognition and analysis tasks are based on the clearest image, thereby improving the accuracy and reliability of the recognition.

[0054] Step S404: Determine the unidentified objects in the target image based on the target association relationship to obtain the target object set, wherein the target association relationship records the association relationship between each object appearing in the target video;

[0055] The aforementioned target association can be the temporal and spatial relationship between objects appearing in the target video, that is, the relationship between each object in the target video and other objects within the lifecycle of the video frame (appearance -> disappearance). The objects in the target association can be objects such as human bodies, vehicles, and non-motorized vehicles, or detailed objects such as human heads, head and shoulders, vehicle bodies, and riders (heads, head and shoulders). This target association includes intra-class association and inter-class association. Intra-class association refers to the target association between the same object (same ID) but different types, for example... Figure 5 As shown on the left, the association between the human body and its face or head within the same human body is illustrated. Inter-class association refers to the target association between different objects (with different IDs) of any type, for example... Figure 5 As shown on the right, the relationship between the people and faces in the front and rear seats of a non-motorized vehicle is illustrated.

[0056] The aforementioned target object set can be a set of objects that the system missed due to occlusion or overlap. This set includes objects that may be missing from the target image or not correctly detected by the system.

[0057] Based on the target video record, the target video records the target association relationships of each object appearing in the target video within its lifecycle. By identifying the target association relationships, objects in the target image that have not been recognized by the system are determined, thus obtaining a set of target objects.

[0058] Step S406: Determine the first target object and the objects in the target object set as the capture targets of the target image.

[0059] Through the above steps, when performing capture detection on a target video, the first target object and the target image containing the first target object are identified in the target video. Then, based on the target association relationship, unrecognized objects in the target image are identified, resulting in a target object set. The first target object and the objects in the target object set are then identified as the capture targets of the target image. Due to limitations in video analysis such as occlusion, lighting conditions, and detection algorithms, some objects (such as low-quality targets) may not be fully recognized in the target image. Therefore, analyzing the target association relationship in the target video helps the system infer potentially missed objects, thereby finding and supplementing these objects, ultimately determining the capture targets of the target image. This effectively avoids information loss caused by missed targets, improves the accuracy and recall rate of capture, and solves the problem of missed capture targets in related technologies, avoiding the situation of missed capture targets and improving the accuracy and recall rate of capture.

[0060] The entities that perform the above steps can be base stations, terminals, etc., but are not limited to these.

[0061] As an optional implementation, determining unidentified objects in the target image based on target association relationships to obtain a target object set includes: determining the types of objects in the target object set to obtain target types; and determining the target object set based on target types and target association relationships.

[0062] The target type mentioned above can be the type of the target of interest captured and identified, that is, the type of object in the set of target objects to be identified; determine the target type of the object in the set of target objects, and then determine the set of target objects based on the target type and the target association relationship, such as determining the set of target objects by filtering the target association relationship through the target type.

[0063] Analyzing the correlation between targets can help identify objects that were not correctly identified in a particular frame. This helps the system detect objects that were missed due to environmental factors or algorithm limitations, thereby improving the coverage and completeness of target detection, as well as the accuracy of identification.

[0064] Optionally, the target object set is determined based on the target type and target association, including: identifying objects in the target association that have an association with the first target object and whose type is the target type, to obtain a second object set; and determining the target object set based on the identifiers of each object in the second object set.

[0065] The objects associated with the first target object in the above-mentioned target association relationship can be objects included in the full-process association relationship of the first target object. This full-process association relationship can be an association relationship established by the system based on all objects associated with the first target object throughout its entire lifecycle in the target video. Through the full-process association relationship, the distribution of all associated objects of the first target object within its lifecycle can be determined.

[0066] The system filters objects within the entire association relationship of a first target object based on the target type, obtaining objects of the target type from the entire association relationship of the first target object, thus obtaining a second object set. Then, based on the objects in the second object set, the target object set is determined. By clearly defining the target type, objects in the potential target object set can be accurately identified, ensuring high relevance among the objects in the target object set, reducing errors and interference from irrelevant data, and improving the accuracy of the target object set.

[0067] As an optional implementation, determining the target object set based on the identifiers of each object in the second object set includes: determining the identifiers of each object in the second object set to obtain N first identifiers, where N is an integer greater than 0; determining objects in the target image that are of the target type and are identified by the first identifiers to obtain a third object set; and determining the target object set through the third object set.

[0068] The identifiers of each object in the second object set are matched with the identifiers of each object in the target object set. If a match is successful, it means that there is an object in the target image with the same identifier as the object in the second object set (i.e., there is an object in the target image with the first identifier). At this time, the object with the target type and the first identifier is identified in the target image to obtain the third object set. The target object set is then determined through the third object set, thereby further narrowing the selection range, reducing errors and interference from irrelevant data, and improving the efficiency of screening and recognition.

[0069] Optionally, determining the target object set through the third object set includes: determining objects in the third object set as objects in the target object set; or, in the case that the first target object includes multiple sub-objects, determining sub-objects of type target type as objects in the fourth object set; and matching each object in the fourth object set with each object in the third object set to obtain the target object set.

[0070] After determining the third object set, since the objects in the third object set are both objects in the target image that are related to the first target object and also conform to the type of interest (i.e., the target type) of the capture detection, the objects in the third object set can be directly determined as objects in the target object set, that is, supplementary targets of the first target object.

[0071] In addition, to ensure the accuracy of the system's target capture, the objects in the third object set can be further filtered. This includes using attribute recognition algorithms or secondary recognition algorithms to detect and identify the captured objects, obtaining the attribute description of the first target object, including the category of the first target object, the sub-objects included in the first target object, and the categories of the sub-objects. For example, if the first target object is identified as a vehicle, the sub-objects included in the vehicle could be the vehicle's license plate, the people inside the vehicle, and faces or heads. If the first target object is identified as a head, since the head itself is a sub-object, the sub-objects included in the head are itself. Sub-objects of the type of the target type among the sub-objects included in the first target object are identified as objects in the fourth object set. The objects in the fourth object set are then matched with the objects in the third object set, thereby achieving further filtering of the objects in the third object set, improving the accuracy of matching, reducing the probability of false matches, and improving the overall reliability and accuracy of the system.

[0072] As an optional implementation, each object in the fourth object set is matched with each object in the third object set to obtain a target object set, including: determining the overlap between the i-th object in the fourth object set and each object in the third object set to obtain multiple first overlaps, where i is an integer greater than 0; determining the object in the third object set corresponding to the first overlap that is greater than or equal to a first preset overlap threshold as an object in the target object set; or, if multiple first overlaps are all less than the first preset overlap threshold, determining the i-th object as an object in the fifth object set; and determining the target object set from the fifth object set.

[0073] The i-th object mentioned above can be any object in the fourth object set. Calculate the first overlap between the i-th object and each object in the third object set. If the first overlap is greater than or equal to the first preset overlap threshold, it means that the objects in the third object set are associated with the i-th object, that is, the objects in the third object set may be associated with the first target object. In this case, the objects in the third object set are determined as objects in the target object set. If the first overlap is less than the first preset overlap threshold, it means that the objects in the third object set are not associated with the i-th object. In this case, the i-th object is taken as an object in the fifth object set, and the objects in the fifth object set are further analyzed to determine the target object set.

[0074] Alternatively, the first overlap can be determined using the following formula to ensure that smaller target boxes are prioritized:

[0075]

[0076] In the formula, IoUMin is the first overlap, S1 is the area of ​​the target box of the i-th object, S2 is the area of ​​the target box of any object in the third object set, and MIN is the minimum value operation.

[0077] Optionally, determining the target object set from the fifth object set includes: obtaining objects of the target type from the target image to obtain a sixth object set; and matching each object in the fifth object set with each object in the sixth object set to obtain the target object set.

[0078] The sixth object set mentioned above can be a collection of all target-type objects in the target image. By directly matching with the target of interest, objects in the fifth object set can be effectively filtered, avoiding resource waste and potential confusion.

[0079] As an optional implementation, each object in the fifth object set is matched with each object in the sixth object set to obtain a target object set, including: determining the overlap between the j-th object in the fifth object set and each object in the sixth object set to obtain multiple second overlaps, where j is an integer greater than 0; identifying the identifier of the object corresponding to the second overlap in the sixth object set that is greater than or equal to a second preset overlap threshold as a third identifier; and determining the target object set through the third identifier.

[0080] The j-th object mentioned above can be any object in the fifth object set. The second overlap between the j-th object and each object in the sixth object set is calculated, thereby matching the objects in the fifth object set with all target-type objects in the target image to identify objects in the target object set present in the fifth object set and determine the target object set. The specific calculation method for the second overlap is the same as that for the first overlap.

[0081] As an optional implementation, determining the target object set through a third identifier includes: determining the identifiers of objects in the target association relationship that are associated with the first target object, to obtain a target identifier set; and matching the third identifier with the target identifier set to obtain the target object set.

[0082] The aforementioned target identifier set can be a set of identifiers of objects included in the full name association relationship of the first target object; by matching the third identifier with the target identifier set, the target type objects in the target image are analyzed based on the full name association relationship of the first target object. By matching with the associated objects of the identified high-quality targets, the probability of being correctly identified can be increased, thereby reducing the false negative rate.

[0083] As an optional implementation, matching the third identifier with the target identifier set to obtain a target object set includes: if the third identifier does not exist in the target identifier set, determining the j-th object as an object in the target object set; or, if the third identifier exists in the target identifier set, determining objects associated with the first target object in the target association relationship to obtain a seventh object set; if there are no objects of the target type among the objects identified by the third identifier in the seventh object set, determining the j-th object as the target object set.

[0084] As an optional implementation, determining the first target object and the objects in the target object set as the capture targets of the target image includes: performing false alarm filtering on the first target object and the objects in the target object set to obtain the capture targets.

[0085] In this embodiment of the application, when performing false alarm filtering on the first target object and the objects in the target object set, it is necessary to establish a false alarm base library as a prior knowledge base. This false alarm base library can be generated and not fully understood before false alarm filtering, or it can be dynamically expanded after false alarm filtering. Figure 6 This is a flowchart of the false alarm database construction process according to an embodiment of this application, such as... Figure 6 As shown, the specific process is as follows:

[0086] S601, Obtain false detection targets, wherein the false detection targets can be false detection targets captured during historical time periods, or the wrong targets can be manually selected.

[0087] S602 performs attribute identification on false detection targets, including feature extraction and false detection identification.

[0088] S603, perform a false alarm database creation operation for the false alarm target, including: determining whether the current false alarm target exists in the false alarm database; if it exists, obtaining the database ID corresponding to the current false alarm target and updating the attribute recognition result of the current false alarm target to the corresponding database ID; if it does not exist, adding a database ID to the false alarm database and adding the attribute recognition result of the current false alarm target to the newly added database ID; wherein, the storage format of the false alarm database can be a database or memory, which is not limited here.

[0089] Optionally, performing false alarm filtering on the first target object and the objects in the target object set to obtain the capture target includes: performing false alarm filtering on the first target object and the objects in the target object set based on a false alarm database to obtain the capture target.

[0090] In the embodiments of this application, Figure 7 This is a flowchart of the false alarm filtering process according to an embodiment of this application, such as... Figure 7 As shown, the specific false alarm filtering process is as follows:

[0091] S701, acquire attribute recognition (including feature extraction, false alarm identification, etc.) information of the captured target (i.e. the first target object) and the supplementary target (i.e. the object in the target object set), and record it as the target to be compared. It can be selected whether to filter false alarms for the supplementary target according to actual needs.

[0092] S702, compares the target to be compared with the targets in the false alarm database, including:

[0093] (1) Traverse multiple targets in the false alarm database, and compare each target in the false alarm database with the target to be compared in turn to obtain the comparison similarity. Since the attribute recognition information contains multiple types of information, obtain the similarity Sx for each type of information comparison, and assign weights to them according to actual needs. Finally, calculate the comprehensive similarity as the final comparison similarity. The specific calculation method is as follows:

[0094] Normalize the weights:

[0095]

[0096] In the formula, w x ′ The weights after normalization, w x The values ​​are the weights before normalization, and M is the total number of weights.

[0097] Calculate the overall similarity:

[0098]

[0099] In the formula, S 综合 To assess overall similarity, w x ′ S is the x-th weight value after normalization. x Let x be the similarity score, and ε be the error value. Generally, ε = (le-5) to prevent division by zero.

[0100] (2) Sort all the obtained similarity results in descending order, and take the first similarity result as the comparison result with the comparison target.

[0101] S703: Based on the preset similarity threshold, determine whether the similarity of the first-ranked target is greater than the preset similarity threshold. If so, filter the target to be compared; otherwise, retain the target to be compared.

[0102] As an optional implementation, before determining the unidentified objects in the target image based on the target association relationship and obtaining the target object set, the method further includes: establishing the target association relationship based on the target video.

[0103] Figure 8 This is a flowchart illustrating the establishment of target association relationships according to embodiments of this application, such as... Figure 8 As shown, the specific process includes:

[0104] S801, based on the existence of objects in the actual scene corresponding to the target video, establish a prior knowledge base of the association relationships between the types of objects. A specific example is as follows:

[0105] Human body: face, human body, non-motorized vehicle, vehicle;

[0106] Human face: human body, non-motorized vehicle, vehicle;

[0107] Non-motorized vehicles: human body, face, license plate;

[0108] Vehicles: human body, human face, license plate;

[0109] S802, traverse each video image in the target video, and analyze the relationship between each object and other objects in each video image. Taking a certain video image as an example, the specific steps are as follows:

[0110] (1) Based on the prior knowledge base, filter the types of each object. If the relationship between the types of two objects is recorded in the prior knowledge base, then the two objects are considered to be unrelated; otherwise, continue execution.

[0111] (2) The positional relationship between two objects can be determined by calculating the degree of overlap. The specific formula is as follows:

[0112]

[0113] In the formula, IoU represents the overlap between two objects, and S 对象1 S is the area of ​​the target bounding box of object 1. 对象2 This is the area of ​​the target bounding box of object 2.

[0114] (3) If the overlap is greater than the preset threshold, the two objects are considered to be related; otherwise, they are considered not related.

[0115] S803: Based on the ID (identifier) ​​of the captured target (i.e., the first target object), perform ID matching from the association results of the current video image. If the ID of the captured target does not exist, the update ends; if it does exist, obtain the corresponding association result and convert it into association sequence 1.

[0116] S804, determine whether the target ID is in the mapping table (i.e., target association). If not, add a new key-value pair, with the target ID as the key and the associated sequence 2 as the value. The content of associated sequence 2 is the same as that of associated sequence 1. If it exists, update the target ID to the value corresponding to the key, including: based on the target ID, retrieve the corresponding associated sequence 3 from the mapping table, traverse each element of associated sequence 1, and match it sequentially with all elements of associated sequence 3. If a match exists, do not update associated sequence 3; if no match exists, update the element in associated sequence 3.

[0117] The associated sequence contains information such as ID, type, subtype, target status, and target location. If the ID, type, and subtype are all equal, it indicates a match; if any one of them is not equal, it indicates a mismatch.

[0118] S805, sort the above associated sequences 2 or 3 in ascending order of priority: ID > type > subtype. For the same field, sort the sequence in ascending order. For example, element 1 (id:1, type:1, subtype:1), element 2 (id:1, type:2, subtype:0), element 3 (id:2, type:1, subtype:1), element 4 (id:1, type:1, subtype:2), the sorted result is element 1-element 4-element 2-element 1.

[0119] As an optional implementation, when determining objects associated with the first target object from the target association relationship (i.e., obtaining the full association relationship of the first target object), firstly, it is determined whether the ID of the first target object is in the mapping table. If it does not exist, it is not output; if it exists, the corresponding association sequence 4 is obtained based on the ID. Secondly, each element of the association sequence 4 is filtered, and then the filtered association sequence 4 is output. The filtering conditions are as follows:

[0120] a) Whether it conforms to the prior knowledge base of association relationships.

[0121] b) Whether it conforms to the human / non-type jump anomaly.

[0122] c) Whether it exists in the single-frame association result of the preferred frame.

[0123] d) Whether it is completely consistent with the target being captured.

[0124] As an optional implementation, Figure 9 This is a flowchart illustrating the determination of supplementary targets for the target to be captured according to an embodiment of this application, such as... Figure 9 As shown, the specific process is as follows:

[0125] S901, obtain the full association information of the capture target (i.e. the first target object), filter and retain the targets of business interest (i.e. objects of type target), and obtain target 1 (i.e. the second object set).

[0126] S902: Based on the ID of target 1, perform ID matching in the trajectory information of the current capture frame (i.e., target image). If there is no match, the process ends; if there is a match, the process continues.

[0127] S903, parse the trajectory details of the matching ID, and filter them according to the predetermined priority, such as head > face > head and shoulders, retaining the highest priority target (i.e., the object of the target type). If it exists, it is recorded as target 2 (i.e., the third object set). If it does not exist, the process ends.

[0128] S904, determine whether there is an attribute recognition result. If there is, continue execution. If not, take target 2 as a supplementary target (i.e., an object in the target object set) and output it.

[0129] For non-target objects such as humans and machines, partial secondary recognition algorithms can be used to detect and identify their component information, such as faces, heads, and license plates. However, since attribute algorithms cannot perceive other target information, the component information they detect and identify may be redundant, thus requiring an additional deduplication process. Combining the overall correlation of the captured targets with the target 2 information from section 2.1, the accuracy of deduplication is increased. The specific logic is as follows:

[0130] S905, analyze the recognition results of multiple attributes corresponding to the captured target, and determine whether there is a target type sub-object among the multiple sub-objects included in the first target object. If none of them exist, the process ends; if they exist, they are recorded as target 3.x (i.e., the fourth object set), such as target 3.1, target 3.2, and so on.

[0131] S906, target 3.x is matched with target 2 in the coordinate frame (i.e., the first overlap is calculated). In order to ensure the principle of smaller target boxes having priority, the threshold calculation method is optimized by using the IoUMin method.

[0132] S907, if its matching threshold is greater than the set threshold (i.e., the first preset overlap threshold), then the current target 3.x is discarded, and target 2 is used as the valid target. If its matching threshold is less than the set threshold, then the current target 3.x is retained. After traversal, all valid targets are recorded as target 4.x (i.e., the fifth object set), such as multiple targets as target 4.1, target 4.2, and so on.

[0133] S908, the target 4.x is matched sequentially with all trajectory targets in the current capture frame (the trajectory targets are selected according to the priority of head > face > head and shoulders, i.e., the sixth object set) (i.e., the second overlap is calculated). The threshold calculation method during matching is IoUMin method.

[0134] S909, if the matching threshold is greater than the set threshold (i.e., the second preset overlap threshold), then the ID of the corresponding trajectory target is recorded as ID1.x (i.e., the third identifier), such as ID1.1, ID1.2, and so on for multiple targets. At this time, the correspondence between ID1.x and target 4.x can be obtained. If the matching threshold is less than the set threshold, no processing is performed.

[0135] S910, sequentially match ID1.x with the entire association relationship of the captured target (i.e., the seventh object set). If none of the IDs match, then the target 4.x corresponding to ID1.x is used as a supplementary target. If the IDs match, proceed to S911.

[0136] S911: Obtain all associated target information corresponding to the matching ID in the full association relationship (i.e., the seventh object set), and perform type detection on them sequentially. If there is an interesting target type, such as a face type target, then target 4.x corresponding to ID1.x does not need to be supplemented in the output. If there is no interesting target type, such as a face type target, then target 4.x corresponding to ID1.x is used as a supplementary target.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0138] This embodiment also provides a target identification device for capturing images. This device is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0139] Figure 10 This is a structural block diagram of the target determination device according to an embodiment of this application, such as... Figure 10 As shown, the device includes: a first determining module 1002, used to determine a target image in a target video, wherein the target image includes a first target object, which is the object with the highest clarity in a first set of objects obtained by recognizing the target video; a second determining module 1004, used to determine unrecognized objects in the target image according to target association relationships, thereby obtaining a set of target objects, wherein the target association relationships record the association relationships between various objects appearing in the target video; and a third determining module 1006, used to determine the first target object and the objects in the set of target objects as the capture targets of the target image.

[0140] In an exemplary embodiment, the apparatus is further configured to identify a first object in a target video to obtain a first object set; determine a first image from an image in the target video that contains any first object from the first object set to obtain a first image set; determine the first object with the highest clarity in the first image set as a first target object; and determine the image containing the first target object as a target image.

[0141] In an exemplary embodiment, the apparatus is further configured to determine the type of objects in the target object set to obtain a target type; and to determine the target object set based on the target type and target association.

[0142] In an exemplary embodiment, the apparatus is further configured to determine objects in the target association relationship that have an association relationship with the first target object and whose type is the target type, thereby obtaining a second object set; and to determine the target object set based on the identifiers of each object in the second object set.

[0143] In an exemplary embodiment, the apparatus is further configured to determine the identifiers of each object in the second object set to obtain N first identifiers, where N is an integer greater than 0; determine objects in the target image that are of the target type and are identified by the first identifiers to obtain a third object set; and determine the target object set through the third object set.

[0144] In an exemplary embodiment, the apparatus is further configured to determine objects in a third object set as objects in a target object set; or, in the case that the first target object includes multiple sub-objects, determine sub-objects of type target type as objects in a fourth object set; and match each object in the fourth object set with each object in the third object set to obtain a target object set.

[0145] In an exemplary embodiment, the apparatus is further configured to determine the overlap between the i-th object in the fourth object set and each object in the third object set, thereby obtaining a plurality of first overlaps, wherein i is an integer greater than 0; determine the object in the third object set corresponding to the first overlap that is greater than or equal to a first preset overlap threshold as an object in the target object set; or, if all the first overlaps are less than the first preset overlap threshold, determine the i-th object as an object in the fifth object set; and determine the target object set from the fifth object set.

[0146] In an exemplary embodiment, the apparatus is further configured to obtain objects of type target from the target image to obtain a sixth object set; and to match each object in the fifth object set with each object in the sixth object set to obtain a target object set.

[0147] In an exemplary embodiment, the apparatus is further configured to determine the degree of overlap between the j-th object in the fifth object set and each object in the sixth object set, thereby obtaining a plurality of second overlap degrees, wherein j is an integer greater than 0; to determine the identifier of the object corresponding to the second overlap degree in the sixth object set that is greater than or equal to a second preset overlap degree threshold as a third identifier; and to determine the target object set through the third identifier.

[0148] In an exemplary embodiment, the apparatus is further configured to determine the identifiers of objects in the target association relationship that are associated with the first target object, thereby obtaining a target identifier set; and to match the third identifier with the target identifier set to obtain a target object set.

[0149] In an exemplary embodiment, the apparatus is further configured to determine the j-th object as an object in the target object set if there is no third identifier in the target identifier set; or, if there is a third identifier in the target identifier set, determine the object that is associated with the first target object in the target association relationship to obtain a seventh object set; if there is no target type object among the objects identified by the third identifier in the seventh object set, determine the j-th object as the target object set.

[0150] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0151] Embodiments of this application also provide a computer-readable storage medium comprising a stored program, wherein the program, when executed, performs the steps of any of the above method embodiments.

[0152] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0153] Embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to perform the steps of any of the above method embodiments via the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0154] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0155] Embodiments of this application also provide a computer program product comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in the embodiments of this application. The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0156] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0157] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining a target for image capture, characterized in that, include: Determine a target image in a target video, wherein the target image includes a first target object, which is the object with the highest clarity in a first set of objects obtained by identifying the target video; Based on the target association relationship, unidentified objects in the target image are determined to obtain a target object set, wherein the target association relationship records the association relationship between each object appearing in the target video; The first target object and the objects in the target object set are determined as the capture targets of the target image.

2. The method according to claim 1, characterized in that, Identify the target image in the target video, including: Identify the first object in the target video to obtain the first object set; The first image is determined by identifying an image in the target video that contains any of the first objects from the first object set, thus obtaining a first image set; The first object with the highest clarity in the first image set is identified as the first target object; The image containing the first target object is defined as the target image.

3. The method according to claim 1, characterized in that, Based on the target association relationship, unidentified objects in the target image are determined, resulting in a target object set, including: Determine the types of objects in the target object set to obtain the target type; The set of target objects is determined based on the target type and the target association.

4. The method according to claim 3, characterized in that, Determining the set of target objects based on the target type and the target association includes: A second set of objects is obtained by identifying objects in the target association relationships that have the association relationship with the first target object and whose type is the target type; The target object set is determined based on the identifiers of each object in the second object set.

5. The method according to claim 4, characterized in that, Determining the target object set based on the identifiers of each object in the second object set includes: Determine the identifier of each object in the second object set to obtain N first identifiers, where N is an integer greater than 0; A third object set is obtained by identifying objects in the target image that are of the target type and identified by the first identifier; The target object set is determined by the third object set.

6. The method according to claim 5, characterized in that, Determining the target object set through the third object set includes: The objects in the third object set are identified as objects in the target object set; or... In the case where the first target object includes multiple sub-objects, the sub-objects of the type of the target type are identified as objects in the fourth object set; each object in the fourth object set is matched with each object in the third object set to obtain the target object set.

7. The method according to claim 6, characterized in that, Each object in the fourth object set is matched with each object in the third object set to obtain the target object set, including: Determine the overlap between the i-th object in the fourth object set and each object in the third object set to obtain multiple first overlaps, where i is an integer greater than 0; The objects in the third object set whose first overlap is greater than or equal to a first preset overlap threshold are identified as objects in the target object set; or... If multiple instances of the first overlap are all less than the first preset overlap threshold, the i-th object is determined as an object in the fifth object set; the target object set is then determined from the fifth object set.

8. The method according to claim 7, characterized in that, Determining the target object set from the fifth object set includes: Obtain objects of type the target type from the target image to obtain a sixth object set; Each object in the fifth object set is matched with each object in the sixth object set to obtain the target object set.

9. The method according to claim 8, characterized in that, Each object in the fifth object set is matched with each object in the sixth object set to obtain the target object set, including: Determine the overlap between the j-th object in the fifth object set and each object in the sixth object set to obtain multiple second overlaps, where j is an integer greater than 0; The identifier of the object in the sixth object set whose second overlap degree is greater than or equal to the second preset overlap degree threshold is determined as the third identifier; The target object set is determined by the third identifier.

10. The method according to claim 9, characterized in that, Determining the target object set through the third identifier includes: Identify the identifiers of objects in the target association relationship that have the association relationship with the first target object, and obtain a target identifier set; The third identifier is matched with the target identifier set to obtain the target object set.

11. The method according to claim 10, characterized in that, The third identifier is matched with the target identifier set to obtain the target object set, including: If the third identifier does not exist in the target identifier set, the j-th object is identified as an object in the target object set; or, If the third identifier exists in the target identifier set, an object with the association relationship with the first target object is determined in the target association relationship to obtain a seventh object set; if no object of the target type exists among the objects identified by the third identifier in the seventh object set, the j-th object is determined as the target object set.

12. A device for determining a target for image capture, characterized in that, include: The first determining module is used to determine a target image in a target video, wherein the target image includes a first target object, which is the object with the highest clarity in a first set of objects obtained by recognizing the target video; The second determining module is used to determine unidentified objects in the target image based on the target association relationship, and obtain a target object set, wherein the target association relationship records the association relationship between each object appearing in the target video; The third determining module is used to determine the first target object and the objects in the target object set as the capture targets of the target image.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

Images