Target event confirmation method and device, storage medium and electronic device

Abstract

The invention discloses a target event confirmation method and device, a storage medium and an electronic device.The method comprises the steps that first image data collected by intelligent monitoring equipment is recognized through a target detection algorithm, and second image data, including a target event, in an image indicated by the target detection algorithm is obtained; performing feature extraction on the second image data through a feature extraction model to obtain a first image feature; comparing the first image feature with an image feature in a preset forward experience library, and determining that the second image data contains the target event under the condition that an obtained first comparison result is successful comparison, the image features in the forward experience library are image features which are verified by a large model and contain a target event. According to the invention, the problem that the monitoring scheme of intelligent misinformation of the monitoring camera is difficult to have accuracy and real-time performance at the same time in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent monitoring technology, and more specifically, to a method and apparatus for confirming target events, a storage medium, and an electronic device. Background Technology

[0002] In the current surveillance industry, intelligent functions of surveillance equipment are playing an increasingly important role. From simple behavioral intrusion detection in the past, to the widespread adoption of intelligent technologies such as video structuring, traffic electronic police, and facial recognition, these technologies have played a crucial and positive role in smart city brains, real-time traffic monitoring and dispatching, and public security criminal investigation deployment. Furthermore, in important occasions such as the Asian Games security, the accuracy requirements for intelligent monitoring are even higher; if intelligent functions malfunction, it can lead to serious problems such as untimely capture of key targets or invalid alarms. Therefore, ideally, surveillance equipment should achieve 100% real-time detection and alarm accuracy, but this is difficult to achieve in practice, and real-time detection and accuracy are often difficult to simultaneously possess.

[0003] Regarding the issue that monitoring schemes for intelligent false alarms from surveillance cameras in related technologies cannot simultaneously achieve both accuracy and real-time performance, no effective solution has yet been proposed. Summary of the Invention

[0004] This invention provides a method and apparatus for confirming target events, a storage medium, and an electronic device, to at least solve the problem in related technologies that the monitoring scheme for intelligent false alarms of surveillance cameras is difficult to simultaneously possess both accuracy and real-time performance.

[0005] According to one aspect of the present invention, a method for confirming a target event is provided, comprising: identifying first image data collected by an intelligent monitoring device using a target detection algorithm to obtain second image data indicating that the target event is included in the image; extracting features from the second image data using a feature extraction model to obtain first image features; comparing the first image features with image features in a preset positive experience base, and confirming that the second image data contains the target event if the first comparison result is a successful comparison, wherein the image features in the positive experience base are image features containing the target event that have been verified by a large model.

[0006] In an exemplary embodiment, after extracting features from the second image data using a feature extraction model to obtain first image features, the method further includes: comparing the first image features with image features in a preset reverse experience library, and if the obtained second comparison result is a successful comparison, confirming that the second image data does not contain the target event, wherein the image features in the reverse experience library are verified image features that do not contain the target event.

[0007] In an exemplary embodiment, after comparing the first image feature with image features in a preset reverse experience library, the method further includes: if both the first comparison result and the second comparison result fail, classifying the second image data using the large model; if the large model indicates that the second image data contains the target event, determining to send the second image data to the target object; and if the large model indicates that the second image data does not contain the target event, determining to prohibit sending the second image data to the target object.

[0008] In an exemplary embodiment, after classifying the second image data using the large model, the method further includes: storing the first image features in the positive experience base when the large model indicates that the second image data contains the target event; and storing the first image features in the negative experience base when the large model indicates that the second image data does not contain the target event.

[0009] In an exemplary embodiment, before comparing the first image feature with image features in a preset positive experience base, the method further includes: acquiring third image data indicating the presence of a target event identified by a target detection algorithm from the original image data, wherein the original image data includes image data related to the target event and image data unrelated to the target event; classifying the third image data using a large model to obtain fourth image data and fifth image data, wherein the fourth image data is image data indicating that the large model contains the target event, and the fifth image data is image data indicating that the large model does not contain the target event; extracting features from the fourth image data and the fifth image data respectively using the feature extraction model to obtain second image features and third image features; storing the second image features in the positive experience base and storing the third image features in the reverse experience base.

[0010] In an exemplary embodiment, comparing the first image feature with image features in a preset positive experience library includes: determining a first feature set included in the first image feature; determining a second feature set in the image features of the positive experience library that corresponds to the first feature set, wherein the difference in the number of feature types between the second feature set and the first feature set is less than a target value; and determining a first comparison result by comparing the first image feature with image features in the preset positive experience library using the second feature set.

[0011] In an exemplary embodiment, determining a first comparison result by comparing the first image features with image features in a preset positive experience base using the second feature set includes: splitting the first feature set and the second feature set into multiple feature groups, wherein each feature group includes: features belonging to the same feature type in the first feature set and features belonging to the same feature type in the second feature set; determining the first comparison result as a successful comparison when the similarity between features in each feature group is greater than or equal to the similarity threshold corresponding to the same feature type; and determining the first comparison result as a failed comparison when the similarity between features in at least one feature group is less than the similarity threshold corresponding to the same feature type.

[0012] According to another aspect of the present invention, a device for confirming a target event is also provided, comprising: an identification module, configured to identify first image data collected by an intelligent monitoring device using a target detection algorithm to obtain second image data indicating that the target event is included in the image; an extraction module, configured to extract features from the second image data using a feature extraction model to obtain first image features; and a comparison module, configured to compare the first image features with image features in a preset positive experience base, and confirm that the second image data contains the target event if the first comparison result is a successful comparison, wherein the image features in the positive experience base are image features containing the target event that have been verified by a large model.

[0013] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium comprising a stored program, wherein the program, when executed, performs the above-described method for confirming the target event.

[0014] According to another aspect of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the above-described method for confirming the target event through the computer program.

[0015] According to another aspect of the present invention, a computer program product is also provided, including a computer program, wherein the computer program, when executed by a processor, implements a method for confirming the aforementioned target event.

[0016] In this embodiment of the invention, a target detection algorithm is used to identify the first image data collected by the intelligent monitoring device to obtain second image data indicating that the target event is included in the image. For the second image data detected by the target detection algorithm, a feature extraction model is used to extract the first image features of the second image data. Then, the first image features are compared with image features in a preset positive experience library. If the first comparison result is successful, it is confirmed that the second image data contains the target event. The image features in the positive experience library are image features containing the target event that have been verified by a large model. Therefore, by comparing the image features containing the target event that have been verified by a large model in the positive experience library with the second image data, it is possible to quickly and accurately confirm whether the second image data actually contains the target event. This solves the problem in related technologies where intelligent false alarm monitoring schemes for surveillance cameras cannot simultaneously possess both accuracy and real-time performance, thus enabling rapid and accurate intelligent false alarm monitoring of surveillance cameras. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computer terminal for a method of confirming a target event according to an embodiment of the present invention.

[0019] Figure 2 This is a flowchart of an optional method for confirming a target event according to an embodiment of the present invention;

[0020] Figure 3 This is a flowchart illustrating an optional method for confirming a target event according to an embodiment of the present invention;

[0021] Figure 4 This is a structural block diagram of a target event confirmation device according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] The method embodiments provided in this invention can be run on a computer terminal. Taking running on a computer terminal as an example, Figure 1 This is a hardware structure block diagram of a computer terminal for a target event confirmation method according to an embodiment of the present invention. Figure 1 As shown, a computer terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. In one exemplary embodiment, the computer terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned computer terminal. For example, the camera device may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 Equivalent functions or ratios shown Figure 1 The functions shown have more different configurations.

[0025] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the target event confirmation method in this embodiment of the invention. The processor 102 runs various functional applications and data processing by executing the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to a secure text network via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the camera equipment's communication provider. In one example, the transmission system 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet.

[0027] This embodiment provides a method for confirming a target event, including but not limited to methods applied to the aforementioned computer terminal. Figure 2 This is a flowchart of an optional method for confirming a target event according to an embodiment of the present invention, the process including the following steps:

[0028] Step S202: The first image data collected by the intelligent monitoring device is identified by the target detection algorithm to obtain second image data indicating that the target event is included in the image.

[0029] Optionally, the target detection algorithm can be a non-human / vehicle detection algorithm. This algorithm is used to perform non-human / vehicle detection and recognition on the first image data (presented as a data stream) acquired by the intelligent monitoring device. The detected target events generally refer to people and vehicles included in the image. The second image data, in most cases, includes one of the following three scenarios: people, vehicles, or both people and vehicles.

[0030] Step S204: Extract features from the second image data using a feature extraction model to obtain the first image features;

[0031] Step S206: Compare the first image feature with the image features in the preset positive experience library, and if the first comparison result is a successful comparison, confirm that the second image data contains the target event, wherein the image features in the positive experience library are image features containing the target event that have been verified by a large model.

[0032] "Large models" typically refer to models with a large number of parameters, complex structures, and extensive training datasets. Such models possess strong learning and generalization capabilities, exhibiting high accuracy and robustness when handling complex tasks. In the embodiments of this invention, the applicable large model can be a large-scale multimodal vision model (Vision-Language Model, abbreviated as VLM).

[0033] It should be noted that the number of model parameters in the object detection algorithm of this invention embodiment is much smaller than that of large models. For example, the number of parameters in the YOLO series object detection algorithms ranges from several million to tens of millions. For instance, YOLOv4 may have approximately 60 to 70 million parameters, while YOLOv6 and YOLOv7 may maintain a number of millions. In contrast, the number of parameters in a VLM large model is in the billions or more, such as the BLIP (Bootstrapping Large-scale Vision and Language Pre-training) model within the VLM large model, which has approximately 1.2 billion parameters. Object detection models are used to detect target events in images in real time and quickly. Large models have stronger feature learning and image understanding capabilities, and can extract more complex and subtle features from images, such as the texture of the target, dynamic behavior patterns, and interactions with the environment. In this invention embodiment, the image features in the positive experience base are image features containing target events that have been validated by large models.

[0034] Through the above technical solution of the embodiments of the present invention, a target detection algorithm is used to identify the first image data collected by the intelligent monitoring device to obtain second image data indicating that the target event is included in the image. For the second image data detected by the target detection algorithm, a feature extraction model is used to extract the first image features of the second image data. Then, the first image features are compared with image features in a preset positive experience library. If the first comparison result is successful, it is confirmed that the second image data contains the target event. The image features in the positive experience library are image features containing the target event that have been verified by a large model. Therefore, by comparing the image features containing the target event that have been verified by a large model in the positive experience library with the second image data, it is possible to quickly and accurately confirm whether the second image data actually contains the target event. This solves the problem in related technologies where intelligent false alarm monitoring schemes for surveillance cameras cannot simultaneously possess both accuracy and real-time performance, thus enabling rapid and accurate intelligent false alarm monitoring of surveillance cameras.

[0035] Optionally, after performing step S204, step S208 can also be performed: comparing the first image feature with the image features in a preset reverse experience library, and if the second comparison result is a successful comparison, confirming that the second image data does not contain the target event, wherein the image features in the reverse experience library are verified image features that do not contain the target event.

[0036] In other words, for the second image data that the target detection algorithm believes contains a target event, the first image features of the second image data can be compared with image features in a pre-set reverse experience library that have been verified by a large model and do not contain the target event. If the second comparison result with the image features in the reverse experience library is successful, it is confirmed that the second image data does not actually contain the target event, i.e., the target detection algorithm has issued a false alarm. It should be noted that steps S206 and S208 can be executed simultaneously or sequentially, such as executing step S206 first and then step S208, or executing step S208 first and then step S206.

[0037] In an exemplary embodiment, after comparing the first image feature with image features in a preset reverse experience library, the method further includes: if both the first comparison result and the second comparison result fail, classifying the second image data using the large model; if the large model indicates that the second image data contains the target event, determining to send the second image data to the target object; and if the large model indicates that the second image data does not contain the target event, determining to prohibit sending the second image data to the target object.

[0038] Furthermore, after classifying the second image data using the large model, the method further includes: storing the first image features in the positive experience base when the large model indicates that the second image data contains the target event; and storing the first image features in the negative experience base when the large model indicates that the second image data does not contain the target event.

[0039] Understandably, if the first image feature of the second image data fails to match both the forward and reverse experience bases, the reason for the failure is that neither the forward nor reverse experience bases contain features similar to the first image feature. In this case, the second image data is classified using a large model. If the large model indicates that the second image data contains a target event, such as a person or vehicle, then the target detection algorithm determines that the second image data detected by the algorithm is not a false alarm and sends the second image data to the target object. Otherwise, if the large model indicates that the second image data does not contain a target event, for example, the second image data actually contains tree branches, then the target detection algorithm determines that the second image data detected by the algorithm is a false alarm and prohibits sending the second image data to the target object.

[0040] Furthermore, the forward and reverse experience bases can be further supplemented based on the large model's judgment result on the second image data. Specifically, if the large model indicates that the second image data contains the target event, the first image feature of the second image data is stored in the forward experience base; if the large model indicates that the second image data does not contain the target event, the first image feature of the second image data is stored in the reverse experience base.

[0041] Through the above steps, false alarms were detected in the second image data without manual intervention, and the forward and reverse experience bases were automatically expanded. This reduces false alarms, and the expanded experience base enables faster and more accurate comparison of the second image data output by the target detection algorithm in subsequent detection processes.

[0042] In an optional embodiment, before comparing the first image features with image features in a preset positive experience base, the method further includes: acquiring third image data identified by the target detection algorithm from the original image data as containing a target event, wherein the original image data includes image data related to the target event and image data unrelated to the target event; classifying the third image data using a large model to obtain fourth image data and fifth image data, wherein the fourth image data is image data indicating that the large model contains the target event, and the fifth image data is image data indicating that the large model does not contain the target event; extracting features from the fourth image data and the fifth image data respectively using the feature extraction model to obtain second image features and third image features; storing the second image features in the positive experience base and storing the third image features in the negative experience base.

[0043] Optionally, the original image data can be pre-monitored using object detection algorithms and a large model to form preliminary positive and negative experience bases. This original image data should contain both images with target events (e.g., images containing people or vehicles) and images without target events (e.g., images containing tree branches, shadows, or telephone poles). Specifically, firstly, the object detection algorithm identifies third-generation image data indicating the presence of target events. The large model then classifies these third-generation image data into fourth-generation image data containing target events and fifth-generation image data not containing target events. Next, the feature extraction module extracts image features from the fourth-generation image data to obtain second-generation image features, and extracts image features from the fifth-generation image data to obtain third-generation image features. Finally, the second-generation image features are stored in the positive experience base, and the third-generation image features are stored in the negative experience base.

[0044] In an exemplary embodiment, comparing the first image feature with image features in a preset positive experience library includes: determining a first feature set included in the first image feature; determining a second feature set in the image features of the positive experience library that corresponds to the first feature set, wherein the difference in the number of feature types between the second feature set and the first feature set is less than a target value; and determining a first comparison result by comparing the first image feature with image features in the preset positive experience library using the second feature set.

[0045] Furthermore, determining the first comparison result by comparing the first image features with image features in a preset positive experience base using the second feature set includes: splitting the first feature set and the second feature set into multiple feature groups, wherein each feature group includes: features belonging to the same feature type in the first feature set and features belonging to the same feature type in the second feature set; determining the first comparison result as a successful comparison when the similarity between features in each feature group is greater than or equal to the similarity threshold corresponding to the same feature type; and determining the first comparison result as a failed comparison when the similarity between features in at least one feature group is less than the similarity threshold corresponding to the same feature type.

[0046] It is understood that image features in the experience base (including forward and reverse experience bases) generally refer to visual attribute features. Image features in the forward experience base include shape, color, and size features of target events (such as people or vehicles); image features in the reverse experience base include shape, color, and size features of non-target events (such as tree branches or telephone poles). In this embodiment of the invention, a similarity between features greater than or equal to the similarity threshold corresponding to the same feature type means that, when the second feature set may include multiple features, the similarity between a feature originating from the first feature set and any feature originating from the second feature set is greater than or equal to the similarity threshold. A similarity between features less than the similarity threshold corresponding to the same feature type means that, when the second feature set may include multiple features, the similarity between a feature originating from the first feature set and all features originating from the second feature set is less than the similarity threshold.

[0047] Taking the example that the target event detected by the object detection algorithm is a person, the following explains how to determine whether there are image features in the positive experience base that match the first image features of the second image data.

[0048] When the detected target event is a person, human features are extracted, such as hairstyle, shirt color, and pants color, forming a first feature set. Then, a second feature set with similar feature types to the first feature set is searched in the positive experience base. Taking human features as an example, the search will look for human features stored in the positive experience base, especially features related to hairstyle, shirt color, and pants color. The feature types in the first and second feature sets should be consistent; that is, the target value can be 1, and the difference in corresponding feature types should be 0.

[0049] The first and second feature sets are then split into multiple feature groups based on feature type. For example, a feature group containing hairstyle features, a feature group containing shirt color features, and a feature group containing pants color features are formed. Then, the features in each feature group are compared for similarity to determine if they match. For the hairstyle feature group, if the similarity to a hairstyle feature in the positive experience base is greater than or equal to a preset hairstyle similarity threshold, the hairstyle feature is considered to match. For the shirt color feature group, if the similarity to a shirt color feature in the positive experience base is greater than or equal to a preset shirt color similarity threshold, the shirt color feature is considered to match. For the pants color feature group, if the similarity to a pants color feature in the positive experience base is greater than or equal to a preset pants color similarity threshold, the pants color feature is considered to match.

[0050] If the similarity between features in each feature group reaches or exceeds the corresponding threshold (i.e., the similarity threshold corresponding to the same feature type to which each feature group belongs), then it is considered that there are image features in the positive experience base that match the first image features of the target event, and the first comparison result is determined to be a successful comparison. If the similarity between features in any feature group is less than the similarity threshold corresponding to the same feature type to which any feature group belongs, then there are no image features in the positive experience base that match the first image features of the detected target, and the first comparison result is determined to be a failed comparison.

[0051] Furthermore, the method also includes: analyzing the historical false alarm rate of the target detection algorithm on historical image data collected by the intelligent monitoring device to determine the high false alarm rate period and the low false alarm rate period of the target detection algorithm; when the current time belongs to the high false alarm rate period, increasing the similarity threshold corresponding to different feature types in the reverse experience base; when the current time belongs to the low false alarm rate period, decreasing the similarity threshold corresponding to different feature types in the forward experience base.

[0052] In other words, the false alarm rate of the target detection algorithm in historical image data collected by intelligent monitoring equipment can be analyzed periodically or in real time, and the false alarm rate at different times can be statistically analyzed to identify patterns in algorithm performance fluctuations. For example, the false alarm rate may increase significantly in the early morning or at night due to poor lighting conditions or significant environmental changes. For periods with high false alarm rates, the similarity thresholds corresponding to different feature types in the reverse experience base can be automatically increased to more strictly filter potential false alarm targets. Only when the feature matches are very close will they be considered false alarms, thus avoiding over-filtering of real targets under complex or adverse conditions. For periods with low false alarm rates, the similarity thresholds corresponding to different feature types in the forward experience base can be automatically decreased to reduce false negatives, ensuring that as many important events as possible are captured in a safe and stable environment.

[0053] To better understand the above-mentioned scheme for confirming the target event, in an optional embodiment, a scheme is also provided for explaining and illustrating the above scheme.

[0054] Optionally, intelligent monitoring false alarm handling solutions in related technologies include one of the following: A false alarm reduction solution that avoids using a large model, namely: 1) Optimizing the small model algorithm to minimize false alarms; 2) A static rule base solution, generating an experience base by manually importing images and then filtering them; 3) A front-end and back-end collaborative solution, utilizing the high performance of the back-end server to run the large model and sending the results back to the front-end; 4) Improving the large model by nesting a small model within the large model and running the small model for one frame to check if it represents a normal target.

[0055] However, these solutions have the following drawbacks: Small models are inherently limited by the structure of deep learning networks and cannot achieve the accuracy of large models. Rigid rules and manually defined filtering rules are difficult to cover complex scenarios (such as misjudging a swaying leaf as an intrusion), resulting in poor accuracy and a lack of proactive learning and knowledge extraction capabilities. The large backend model, combined with the frontend model, requires coordinated deployment, leading to frontend-backend coupling and network transmission between them, impacting real-time performance and stability. The adoption of a small model + large model structure heavily relies on the accuracy of the small model and fails to leverage the accuracy advantages of the large model.

[0056] Therefore, monitoring solutions for intelligent false alarms from surveillance cameras in related technologies often struggle to simultaneously achieve both accuracy and real-time performance.

[0057] To address the aforementioned issues, in this embodiment of the invention, the ability of a large model to detect a wide range of target types is utilized to autonomously determine whether a target is a false alarm. Then, features are extracted from the target and updated in real time to the positive or negative experience base.

[0058] Regarding the false alarm filtering strategy, since the large model consumes a significant amount of time to process on intelligent monitoring equipment, it cannot perform real-time false alarm analysis on the image data stream. Therefore, an experience base is designed between the algorithm output for pedestrians, vehicles, and other non-motorized vehicles (PMLs) and the large model. The PML output is not directly fed into the large model to determine whether it is a false alarm; instead, it is first compared with the experience base. This can handle the vast majority of cases, with the remaining small number of cases then sent to the large model for judgment.

[0059] Specifically, the process of automatically generating an experience base includes:

[0060] Step 301: The targets detected by the human, vehicle, and non-human algorithm (equivalent to the second image data in the above embodiment) are sent to the large model for analysis. The targets are then sent to the feature extraction module to extract feature information (i.e., the first image features in the above embodiment, which may include a first feature set).

[0061] Step 302: The large model outputs the result indicating whether the target is a non-false alarm or a false alarm.

[0062] Step 303: If the output result is a non-false alarm, store the feature in the positive experience base. If the output result is a false alarm, store the feature in the negative experience base.

[0063] It should be noted that the experience base in this embodiment of the invention consists of two parts: a forward experience base and a reverse experience base. The function of the experience base is that when the customer enables intelligent functions such as human detection and non-human / vehicle detection, the target output by the human detection algorithm is not directly reported to the customer, but is compared with the experience base. If the target output by the non-human / vehicle detection algorithm matches an element in the forward experience base (i.e., the second feature set in the above embodiment), it indicates that it is not a false alarm. Conversely, if it matches an element in the reverse false alarm base, it indicates that it is a false alarm.

[0064] The principle of generating the experience base in this embodiment of the invention is to extract feature data from the image using the feature extraction module, and then store the feature data into the positive feature database and the negative feature database according to whether they are non-false alarm features or false alarm features. These two feature databases are called the experience base (i.e., the positive experience base and the negative experience base).

[0065] Traditional methods for manually generating experience databases have two drawbacks. First, they require pre-prepared forward or reverse image materials. Insufficient coverage or the inclusion of poorly defined objects in the materials can affect the final comparison results. Second, collecting image materials and importing them into the device's web interface is time-consuming and labor-intensive, becoming cumbersome when dozens or even hundreds of devices are deployed on-site.

[0066] The advantage of large-scale models is their ability to accurately identify a large number of different object types. Therefore, leveraging this characteristic, the targets output by human detection intelligence are first fed into the large-scale model for secondary evaluation. If the large-scale model determines that a target is a false alarm, such as a tree branch or utility pole, the image of that tree branch is sent to the feature extraction module, and the extracted features are saved to the experience database. For example, if a tree branch is a false alarm, it is saved in the reverse experience database; if a normal person or vehicle is detected, and the feature is not saved in the forward experience database, it is saved to the forward experience database. This process requires no manual intervention and greatly reduces the difficulty of use.

[0067] Optionally, after the automatic generation of the experience base step, generate, for example, Figure 3 After the positive and negative experience bases are shown, the false alarm filtering process includes:

[0068] Step 401: The real-time data stream is detected by the human, vehicle and non-human algorithm, and the predetermined target is output.

[0069] Step 402: Send the image (i.e., the second image data) in the real-time data stream containing the target to the feature extraction module to extract target features (equivalent to the first image features or the first feature set in the above embodiment).

[0070] Step 403: Compare the target feature with the forward and reverse experience bases using the feature comparison module.

[0071] Step 404: If a match can be found in the forward database, it indicates that it is not a false alarm, and the vehicle / person not found result is reported to the user. If it is not in the forward database, it is matched with the reverse database. If a match can be found with the reverse database, it indicates that it is a false alarm, and no report is sent to the user. If it is neither in the forward nor the reverse database, the experience database is updated using the steps for automatically generating the experience database.

[0072] In other words, for smart cameras in the security industry, the limited computing power of their hardware units (CNNs) due to cost, low power consumption, and chip architecture makes it difficult to simultaneously detect targets in real time and send them to a large model for analysis. Therefore, real-time detected targets such as humans are first compared with an experience database (because the experience database processes feature data very quickly), which effectively solves the real-time problem of large models on smart cameras.

[0073] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to run the methods of the various embodiments of the present invention.

[0074] This embodiment also provides a target event confirmation device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0075] Figure 4 This is a structural block diagram of a target event confirmation device according to an embodiment of the present invention; as shown below. Figure 4 As shown, it includes:

[0076] The recognition module 42 is used to recognize the first image data collected by the intelligent monitoring device through the target detection algorithm to obtain the second image data that includes the target event in the image indicated by the target detection algorithm;

[0077] Extraction module 44 is used to extract features from the second image data using a feature extraction model to obtain first image features;

[0078] The comparison module 46 is used to compare the first image feature with the image features in the preset positive experience library, and if the first comparison result is a successful comparison, confirm that the second image data contains the target event, wherein the image features in the positive experience library are image features containing the target event that have been verified by a large model.

[0079] The device described above uses a target detection algorithm to identify first image data collected by an intelligent monitoring device, obtaining second image data indicating that the target event is included in the image. For the second image data detected by the target detection algorithm, a feature extraction model extracts first image features from the second image data. These first image features are then compared with image features in a preset positive experience library. If the first comparison result is successful, it is confirmed that the second image data contains the target event. The image features in the positive experience library are image features containing the target event that have been verified by a large model. Therefore, by comparing the image features containing the target event that have been verified by a large model in the positive experience library with the second image data, it is possible to quickly and accurately confirm whether the second image data actually contains the target event. This solves the problem in related technologies where intelligent false alarm monitoring schemes for surveillance cameras cannot simultaneously possess both accuracy and real-time performance, thus enabling rapid and accurate intelligent false alarm monitoring of surveillance cameras.

[0080] In an exemplary embodiment, the comparison module 46 is further configured to compare the first image feature with image features in a preset reverse experience library, and if the second comparison result is a successful comparison, confirm that the second image data does not contain the target event, wherein the image features in the reverse experience library are verified image features that do not contain the target event.

[0081] In one exemplary embodiment, the apparatus further includes: a classification module, configured to compare the first image features with image features in a preset reverse experience library, and, if both the first comparison result and the second comparison result fail, classify the second image data using the large model; if the large model indicates that the second image data contains the target event, determine to send the second image data to the target object; and if the large model indicates that the second image data does not contain the target event, determine to prohibit sending the second image data to the target object.

[0082] In one exemplary embodiment, the apparatus further includes: a first storage module, configured to store the first image feature in the forward experience base when the large model indicates that the second image data contains the target event; and to store the first image feature in the reverse experience base when the large model indicates that the second image data does not contain the target event.

[0083] In an exemplary embodiment, the apparatus further includes a second storage module, configured to: acquire third image data indicating the presence of a target event as identified by a target detection algorithm from the original image data before comparing the first image features with image features in a preset forward experience base; classify the third image data using a large model to obtain fourth and fifth image data, wherein the fourth image data is image data indicating the presence of the target event as indicated by the large model, and the fifth image data is image data indicating the absence of the target event as indicated by the large model; extract features from the fourth and fifth image data respectively using the feature extraction model to obtain second and third image features; store the second image features in the forward experience base and store the third image features in the reverse experience base.

[0084] In an exemplary embodiment, the comparison module 46 is further configured to: determine a first feature set including the first image feature; determine a second feature set in the image features of the positive experience library corresponding to the first feature set, wherein the difference in the number of feature types between the second feature set and the first feature set is less than a target value; and determine a first comparison result by comparing the first image feature with the image features in the preset positive experience library using the second feature set.

[0085] In an exemplary embodiment, the comparison module 46 is further configured to split the first feature set and the second feature set into multiple feature groups, wherein each feature group includes: features belonging to the same feature type in the first feature set and features belonging to the same feature type in the second feature set; if the similarity between features in each feature group is greater than or equal to the similarity threshold corresponding to the same feature type, the first comparison result is determined to be a successful comparison; if the similarity between features in at least one feature group is less than the similarity threshold corresponding to the same feature type, the first comparison result is determined to be a failed comparison.

[0086] Embodiments of the present invention also provide a storage medium comprising a stored program, wherein the program, when executed, performs any of the methods described above.

[0087] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0088] S1, the first image data collected by the intelligent monitoring device is identified by the target detection algorithm to obtain second image data indicating that the target event is included in the image;

[0089] S2, the second image data is used to extract features through a feature extraction model to obtain the first image features;

[0090] S3, compare the first image feature with the image features in the preset positive experience library, and if the first comparison result is a successful comparison, confirm that the second image data contains the target event, wherein the image features in the positive experience library are image features containing the target event that have been verified by a large model.

[0091] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0092] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0093] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0094] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0095] S1, the first image data collected by the intelligent monitoring device is identified by the target detection algorithm to obtain second image data indicating that the target event is included in the image;

[0096] S2, the second image data is used to extract features through a feature extraction model to obtain the first image features;

[0097] S3, compare the first image feature with the image features in the preset positive experience library, and if the first comparison result is a successful comparison, confirm that the second image data contains the target event, wherein the image features in the positive experience library are image features containing the target event that have been verified by a large model.

[0098] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0099] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the embodiments of the method for confirming any of the target events described above.

[0100] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the embodiments of the above-described method for confirming any of the target events.

[0101] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0102] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing systems. They can be centralized on a single computing system or distributed across a network of multiple computing systems. Optionally, they can be implemented using program code executable by a computing system, thereby storing them in a storage system for execution by the computing system. In some cases, the steps shown or described can be executed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0103] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of confirming a target event, characterized by, The method comprises: identifying first image data collected by an intelligent monitoring device through a target detection algorithm to obtain second image data including a target event indicated by the target detection algorithm; extracting features of the second image data through a feature extraction model to obtain first image features; comparing the first image features with image features in a preset positive experience library, and confirming that the second image data contains the target event if a first comparison result obtained is a comparison success, wherein the image features in the positive experience library are image features containing target events verified by a large model.

2. The method of claim 1, wherein After extracting the first image features from the second image data through the feature extraction model, the method further comprises: comparing the first image features with image features in a preset negative experience library, and confirming that the second image data does not contain the target event if a second comparison result obtained is a comparison success, wherein the image features in the negative experience library are image features not containing target events verified.

3. The method of claim 2, wherein After comparing the first image features with the image features in the preset negative experience library, the method further comprises: classifying the second image data through the large model if the first comparison result and the second comparison result are both comparison failures; determining to send the second image data to a target object if the large model indicates that the second image data contains the target event; determining to prohibit sending the second image data to the target object if the large model indicates that the second image data does not contain the target event.

4. The method of claim 3, wherein After classifying the second image data through the large model, the method further comprises: storing the first image features into the positive experience library if the large model indicates that the second image data contains the target event; storing the first image features into the negative experience library if the large model indicates that the second image data does not contain the target event.

5. The method of claim 1, wherein Before comparing the first image features with the image features in the preset positive experience library, the method further comprises: obtaining third image data containing a target event identified by a target detection algorithm from original image data, wherein the original image data includes image data related to the target event and image data unrelated to the target event; classifying the third image data through a large model to obtain fourth image data and fifth image data, wherein the fourth image data is image data indicated by the large model to contain the target event, and the fifth image data is image data indicated by the large model to not contain the target event; extracting features of the fourth image data and the fifth image data through the feature extraction model to obtain second image features and third image features; storing the second image features into the positive experience library and storing the third image features into the negative experience library.

6. The method of claim 1, wherein The first image feature is compared with image features in a preset positive experience library, including: Determine that the first image feature includes a first feature set; Determine that the image features in the positive experience library correspond to a second feature set of the first feature set, wherein the type number difference between the second feature set and the first feature set is less than a target value; Determine the first comparison result of comparing the first image feature with the image features in the preset positive experience library through the second feature set.

7. The method of claim 6, wherein Determine the first comparison result of comparing the first image feature with the image features in the preset positive experience library through the second feature set, including: Split the first feature set and the second feature set into a plurality of feature groups, wherein each feature group includes features of the same type of feature in the first feature set and features of the same type of feature in the second feature set; In the case where the similarity between the features in each feature group is greater than or equal to the similarity threshold value corresponding to the same type of feature, it is determined that the first comparison result is a comparison success; In the case where the similarity between the features in at least one feature group is less than the similarity threshold value corresponding to the same type of feature, it is determined that the first comparison result is a comparison failure.

8. A device for confirming a target event, characterized in that, Including: The recognition module is configured to recognize first image data collected by the intelligent monitoring device through a target detection algorithm to obtain second image data including a target event indicated by the target detection algorithm; The extraction module is configured to extract features from the second image data through a feature extraction model to obtain first image features; The comparison module is configured to compare the first image features with image features in a preset positive experience library, and in the case where the obtained first comparison result is a comparison success, confirm that the second image data contains the target event, wherein the image features in the positive experience library are image features containing target events verified by a large model.

9. A computer readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program runs the method described in any one of claims 1 to 7 when running. 10.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to run the method described in any one of claims 1 to 7 through the computer program. The memory stores a computer program, and the processor is configured to run the method described in any one of claims 1 to 7 through the computer program.

Images