Image recognition alarm method, device, medium, equipment and system

Abstract

The invention discloses an image recognition alarm method and device, a medium, equipment and a system. The image recognition alarm method comprises the following steps: performing preliminary reasoning on an original image based on a preset computer vision small model to obtain a preliminary reasoning result corresponding to the original image; taking the original image of which the preliminary reasoning result meets a preset preliminary screening condition as the input of a preset computer vision large model, and performing verification reasoning on the original image to obtain a verification reasoning result corresponding to the original image; and outputting alarm information associated with the original image in response to matching of the verification reasoning result and the corresponding preliminary reasoning result. Through the above method, rapid and accurate image recognition reasoning can be achieved based on a small amount of data, the efficiency, accuracy and scene compatibility of image recognition alarm are effectively improved by using secondary verification reasoning of a small model and a large model, the method is suitable for an abnormal scene where material collection is difficult, and the efficiency and accuracy of image recognition alarm are improved. And application of image identification alarm can be realized.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an image recognition alarm method, device, medium, equipment and system. Background Art

[0002] At present, the analysis of video images is mainly achieved through computer vision (CV) algorithms, which decode the video into a series of frame images for inference analysis, and then judge the recognition results of the video images through specific business rules to achieve specific image detection, classification, segmentation and target recognition tasks.

[0003] The widely used computer algorithms are mainly based on training small models such as YOLO (You Only Look Once) series models, Inception series models, ResNet (Residual Network) series models, etc. However, these small models have significant limitations and shortcomings in image recognition: on the one hand, in order to achieve a high recognition accuracy and adapt to complex classification requirements, tens of thousands of data are usually required to train the small model. The process requires the collection and annotation of a large amount of materials, which requires a large amount of manpower and computing power consumption, and requires a high amount of training data. It is difficult to effectively apply it in video analysis and alarm tasks in abnormal scenes such as emergency scenes, safety violation scenes, and special activity scenes where image materials are difficult to collect. It is costly and has poor flexibility and compatibility. On the other hand, the generalization ability of the recognition of the small model is also poor. After changing the deployment location and video source of the relevant recognition facilities, the accuracy will be greatly reduced, and it is necessary to collect materials again for training and tuning. The compatibility is poor, the relevant deployment work is cumbersome, and the application is difficult, resulting in high deployment and tuning costs and complex operation processes. Summary of the invention

[0004] The present application mainly provides an image recognition alarm method, device, medium, equipment and system, aiming to solve the technical problem of high requirements for the amount of training data in existing image recognition.

[0005] In order to solve the above technical problems, the technical solution adopted by the present application is: to provide an image recognition alarm method. The image recognition alarm method includes: performing preliminary reasoning on the original image based on a preset small computer vision model to obtain a preliminary reasoning result of the original image; using the original image whose preliminary reasoning result meets the preset preliminary screening conditions as the input of the preset large computer vision model to perform verification reasoning on the original image to obtain a verification reasoning result corresponding to the original image; in response to the verification reasoning result matching the corresponding preliminary reasoning result, outputting an alarm information associated with the corresponding original image; wherein the parameter magnitude of the small computer vision model is smaller than the parameter magnitude of the large computer vision model.

[0006] In some embodiments, the preliminary reasoning result includes a first type and a first position information of the target in the original image, and the verification reasoning result includes a second type and a second position information of the target in the original image; in response to the verification reasoning result matching the corresponding preliminary reasoning result, outputting the warning information associated with the corresponding original image includes: in response to the first type and the second type being the same, and the overlap between the first position information and the second position information being greater than a preset overlap threshold, outputting the warning information.

[0007] In some embodiments, the preliminary reasoning result includes a first category label of the target event in the original image, and the verification reasoning result includes a second category label of the target event in the original image; in response to the verification reasoning result matching the corresponding preliminary reasoning result, outputting the warning information associated with the corresponding original image includes: in response to the first category label and the second category label being the same, outputting the warning information.

[0008] In some embodiments, the small computer vision model is obtained by training a small image recognition model based on the first sample set; and the large computer vision model is obtained by fine-tuning a pre-trained large model based on the second sample set.

[0009] In some embodiments, the pre-trained large model includes a Grounding DINO model and / or a CLIP model.

[0010] In some embodiments, the first sample set and the second sample set are the same preset sample set, and the preset sample set includes a first preset number of positive samples and a second preset number of negative samples.

[0011] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide an image recognition alarm device, which includes: a first recognition module, used to perform preliminary reasoning on the original image based on a preset small computer vision model to obtain a preliminary reasoning result of the original image; a second recognition module, used to use the original image whose preliminary reasoning result meets the preset preliminary screening conditions as the input of the preset large computer vision model to perform verification reasoning on the original image to obtain a verification reasoning result corresponding to the original image; a verification module, used to output an alarm information associated with the corresponding original image in response to the verification reasoning result matching the corresponding preliminary reasoning result; wherein the parameter magnitude of the small computer vision model is smaller than the parameter magnitude of the large computer vision model.

[0012] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a storage medium, on which program data is stored, characterized in that when the program data is executed by a processor, the steps of the image recognition alarm method as described above are implemented.

[0013] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide a computer device, which includes a processor and a memory connected to each other, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the image recognition alarm method as mentioned above.

[0014] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide a monitoring system, which includes a camera and a computer device as mentioned above, wherein the computer device is communicatively connected to the camera, and the camera is used to continuously acquire original images.

[0015] The beneficial effects of the present application are as follows: Different from the prior art, the present application discloses an image recognition alarm method, device, medium, equipment and system. The present application uses a preset small computer vision model to perform preliminary reasoning on the original image, which can quickly screen out the original images with alarm risks, reduce the amount of data to be processed by subsequent large model reasoning, reduce the requirements for the large model reasoning speed, and improve the reasoning efficiency. By performing secondary verification reasoning on the screened original images through a preset large computer vision model, the alarm information associated with the corresponding original image can be determined more accurately and reliably. At the same time, based on the scheme of the present application, only a small amount of material is needed to train the small model and fine-tune the large model to achieve a high image recognition accuracy rate, which can significantly reduce the material requirements and reduce the consumption of manpower and computing power, making the scheme of the present application particularly applicable when it is difficult to collect materials in abnormal scenes. In addition, due to the introduction of a large computer vision model and the use of the excellent generalization capabilities of the large model, this application can still achieve a high accuracy rate after changing the deployment location and camera, avoiding the re-collection, training and deployment optimization of materials, which is conducive to improving the scene compatibility of image recognition alarms and realizing the application of image recognition alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0017] Figure 1 It is a flowchart of an embodiment of an image recognition alarm method provided by the present application;

[0018] Figure 2 It is a structural schematic diagram of an embodiment of an image recognition alarm device provided by the present application;

[0019] Figure 3 It is a structural schematic diagram of an embodiment of a storage medium provided by the present application;

[0020] Figure 4 is a structural schematic diagram of an embodiment of a computer device provided by the present application;

[0021] Figure 5 It is a structural diagram of an embodiment of a monitoring system provided by the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The terms "first", "second", "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0024] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0025] This application provides an image recognition alarm method, see Figure 1 , Figure 1 1 is a flow chart of an embodiment of an image recognition alarm method provided by the present application, the image recognition alarm method comprising:

[0026] Step 10: Perform preliminary reasoning on the original image based on the preset computer vision model to obtain preliminary reasoning results of the original image.

[0027] Step 20: The original image whose preliminary reasoning result meets the preset preliminary screening conditions is used as the input of the preset computer vision large model to verify the reasoning of the original image and obtain the verification reasoning result corresponding to the original image.

[0028] Step 30: In response to verifying that the reasoning result matches the corresponding preliminary reasoning result, outputting warning information associated with the corresponding original image.

[0029] In this embodiment, the alarm information is warning data related to specific scenes or conditions, such as abnormal behavior, security threats, environmental changes, etc. The original image is image data used to determine whether there is alarm information, which can be obtained in real time through a camera, screenshot tools, etc., or by extracting historical data such as databases or archive files. For example, in video analysis alarms, the video can be decoded into frame images, and each frame image is used as the original image.

[0030] In this embodiment, the warning information in the original image is specifically identified by image recognition, and the rapid detection and response of the warning information is achieved by performing image recognition analysis on the original image. Among them, image recognition refers to the use of computers and artificial intelligence technology to analyze and understand images to automatically identify objects, scenes, and patterns in the image. For different image warning tasks, image recognition technologies with different focuses can be used, such as target detection, image classification, biometrics, and image matching. Among them, target detection and image classification are particularly widely used in image recognition warnings. For example, target detection can be used to detect abnormal behavior objects, disaster situations, and product defects in images, and image classification can be used to distinguish between scenes such as security threat scenes, congestion scenes, and abnormal situation scenes and normal scenes.

[0031] In this embodiment, image recognition adopts a secondary verification reasoning method, that is, preliminary reasoning based on a small computer vision model and verification reasoning based on a large computer vision model. Among them, the parameter magnitude of the small computer vision model is smaller than the parameter magnitude of the large computer vision model. For example, optionally, the small computer vision model can be established by training small image recognition models such as the YOLO series model, the Inception series model, the ResNet series model, the RetinaNet series model, and the VGG (Visual Geometry Group Network) series model with a parameter magnitude of 100,000 or 1 million. These small models have a higher reasoning speed and lower computing resource requirements. When training these models for image recognition, a small number of sample images that meet the basic image quality requirements and type matching requirements can usually be used to obtain an image recognition model with basic recognition capabilities.

[0032] In this embodiment, more sample images can also be used to train these models to achieve a more accurate image recognition effect, as long as the reasoning efficiency and accuracy of the trained model and the difficulty of collecting the corresponding materials are weighed. For example, in actual tests, model training with 80 positive sample images and 80 negative sample images can meet the requirements for preliminary reasoning ability, and model training with 200 positive and negative sample images and 200 sub-sample images can achieve better preliminary reasoning ability, but when this value is exceeded, the response speed of the model is not as expected, so the number of positive and negative samples can be selected within 80 to 200. Among them, the requirements for preliminary reasoning ability can be determined by setting thresholds or ranges for indicators such as accuracy, generalization ability, and response speed. Traditional image recognition that only uses small models usually requires more than 10,000 positive and negative sample images to achieve a good accuracy rate. However, the secondary verification reasoning architecture of this embodiment has a lower reliance on the small computer vision model, and about one hundred positive and negative sample images can usually meet the requirements of preliminary reasoning capabilities, so that the obtained small computer vision model can have preliminary screening capabilities while saving resource costs and image collection costs.

[0033] In this embodiment, compared with the small computer vision model, the large computer vision model has a more complex network structure and more parameters. For example, optionally, the large computer vision model can be fine-tuned based on the Grounding DINO model, CLIP (Contrastive Language-Image Pre-Training, Contrastive Language-Image and Training Model) model, M6 model and other pre-trained large models with tens of millions of parameters and hundreds of millions of parameters. The obtained large computer model can provide more accurate recognition results in image recognition tasks, but at the same time requires more computing resources and longer reasoning time. In this regard, the secondary verification reasoning method of this scheme can effectively reduce the amount of data that the large computer vision model needs to process, thereby reducing the consumption of computing resources and the reasoning time of the large computer vision model as a whole. Since the original image input into the large computer vision model is obtained by screening the small computer vision model, only a small number of video frames that may cause alarms are retained, so the amount of data that the large computer vision model needs to process is greatly reduced, which can effectively reduce the requirements for the reasoning speed and resource occupancy of the large computer vision model. It can also filter out the original images that are misidentified by the small computer vision model, and the obtained alarm information is more accurate and effective. In addition, since the computer vision big model is established based on fine-tuning the pre-trained big model, it usually also has the generalization ability of the pre-trained big model and better scene compatibility, so that the original images obtained in different scenes and video sources can be recognized with a high accuracy. There is no need to collect materials again for model training, which reduces the workload of deployment and tuning, and is conducive to the application of the image recognition alarm method of this embodiment.

[0034] In this embodiment, the amount of data required to obtain the computer vision large model based on fine-tuning the pre-trained large model is also extremely low. After experiments based on this solution, usually about 100 positive and negative sample image materials can achieve a good fine-tuning effect. More sample images can also be used to fine-tune these pre-trained large models to achieve a more accurate effect, as long as the reasoning efficiency and accuracy of the trained model and the difficulty of collecting the corresponding materials are weighed. For example, in actual tests, model training with 80 positive sample images and 200 negative sample images and sub-sample images can meet the requirements for verification reasoning ability. Model training with 200 positive and negative sample images and sub-sample images can achieve better preliminary reasoning ability, but when this value is exceeded, the response speed of the model is not as expected, so the number of positive and negative samples can be selected within 80 to 200. Among them, the requirements for verification reasoning ability can be determined by setting thresholds or ranges for indicators such as accuracy, generalization ability, and response speed. The requirements for each indicator of the verification reasoning ability need to be more stringent than the requirements for the preliminary reasoning ability to achieve a faster, more accurate, and more generalized verification reasoning effect.

[0035] Optionally, the pre-trained large model includes a Grounding DINO model and / or a CLIP model.

[0036] In this optional embodiment, the Grounding DINO model is an image recognition model based on deep learning, which can more accurately identify and classify objects in images by combining scene context information and image content. The model adopts a novel multi-task learning framework, which can not only perform target detection, but also scene understanding and image segmentation. In terms of target detection, the model improves the detection accuracy of small targets by fusing feature maps of different scales. In terms of scene understanding, the Grounding DINO model enhances the recognition ability of key elements in the scene by introducing an attention mechanism. The image segmentation part utilizes the structure of the Fully Convolutional Networks (FCN) to achieve accurate classification of each pixel in the image, and performs well in handling complex scenes and small target detection. In addition, the Grounding DINO model also integrates a lightweight post-processing module to further improve the accuracy and robustness of the detection results.

[0037] In this optional embodiment, the CLIP model is a comparative language-image pre-training model, which is jointly trained by large-scale text and image data sets to learn rich visual and language features. The core of the CLIP model is that it can understand the association between image content and natural language description, which enables it to use language information to guide the extraction and understanding of image features when processing image recognition tasks. In this embodiment, the CLIP model is used in an image recognition alarm system to improve the accuracy and efficiency of recognition. By fine-tuning the CLIP model, it can be adapted to specific image recognition alarm needs, such as anomaly detection for specific scenes or recognition of specific objects. Since the CLIP model already has a strong cross-modal understanding ability, in practical applications, it can quickly adapt to new image recognition tasks without large-scale retraining, which greatly reduces the cost of model deployment and maintenance.

[0038] In this optional embodiment, different pre-trained large models can be used for different image recognition alarm scenarios. For example, when performing target detection, the use of the Grounding DINO model is more appropriate, while when performing image classification, the CLIP model may provide better performance. In addition, according to actual needs, the advantages of different pre-trained large models can be combined to form a hybrid model to achieve better recognition effects. For example, the advantages of the Grounding DINO model in target detection and scene understanding can be combined with the ability of the CLIP model in cross-modal understanding to build a model that can simultaneously perform accurate target detection and image classification. In practical applications, this hybrid model can better adapt to diverse image recognition alarm scenarios, improve the overall performance and robustness of the system, and meet the application needs of different businesses and scenarios.

[0039] In this embodiment, optionally, on the basis that the aforementioned computer vision small model is obtained based on the training of the image recognition small model, and the computer vision large model is obtained based on the fine-tuning of the pre-trained large model, the computer vision small model can be obtained by training the first sample set and the computer vision large model can be obtained by fine-tuning the second sample set. Among them, the first sample set usually needs to include images in different scenes to ensure that the model can cover various possible recognition situations. These images can be static or dynamic video frames, and they are carefully selected to reflect the diversity in the real world. For example, the first sample set can include urban traffic scenes, industrial monitoring scenes, natural environment scenes, indoor environment scenes and other situations. Through such a diverse sample set, the computer vision small model can learn features in different environments during the training process, thereby improving its robustness and adaptability in practical applications. When constructing the second sample set, in addition to including scenes similar to the first sample set, special attention should be paid to edge cases that may trigger false positives or false negatives. Image samples of these edge cases are crucial for fine-tuning the computer vision large model because they help the model learn more refined discrimination capabilities. For example, the second sample set may include scenes with drastic lighting changes, occlusions, fast-moving objects, low-contrast or blurred images, etc. Through these edge case samples, the large computer vision model can improve its recognition accuracy of complex scenes during the fine-tuning process, thereby reducing the occurrence of false positives and negatives in practical applications.

[0040] In this embodiment, the first sample set and the second sample set can be the same preset sample set. On this basis, the construction of the preset sample set needs to comprehensively consider the coverage of the scene, the representativeness of the sample and the diversity of the data to ensure that the trained model can adapt to various practical application environments. The preset sample set includes a first preset number of positive samples and a second preset number of negative samples. The first preset number and the second preset number can be adjusted according to the needs of the actual application scenario. For example, in some occasions where the recognition accuracy is extremely high, the number of positive samples can be appropriately increased to ensure that the model can learn enough positive features; and in occasions where the false alarm rate is low, the number of negative samples can be appropriately reduced to reduce the model's excessive sensitivity to negative features. In addition, the construction of the sample set should also take into account the balance of the data to avoid deviations in the recognition of a certain category of the model due to the imbalance of the number of samples. In actual operation, the model performance under different sample set configurations can be evaluated by methods such as cross-validation to find the optimal sample set combination. In this way, it can be ensured that the image recognition alarm system can maintain efficient and accurate recognition capabilities under different environments and conditions.

[0041] In this embodiment, the preset sample set can be obtained through project channels, from relevant experiments or real scenes, or by simulating similar scenes, or extracted from open source galleries, network images, and third-party databases. In addition, data enhancement technology can also be used to expand the diversity of samples, such as increasing the model's adaptability to different changes through metric learning, meta-learning, transfer learning, and rotation, scaling, cropping, color transformation, etc. Through the above method, the recognition accuracy and generalization ability of small and large computer vision models can be effectively improved, while reducing dependence on a large number of samples and reducing training costs.

[0042] In this embodiment, after performing preliminary reasoning to obtain preliminary reasoning results and performing verification reasoning to obtain verification reasoning results, matching confirmation can be performed based on the secondary reasoning results to output the alarm information associated with the original image. Among them, the alarm information is based on the type and severity of the alarm information. The system can automatically select an appropriate alarm method, such as notifying relevant personnel through SMS, email, application push or on-site alarm, and the alarm information uses different triggering methods for different image recognition services. For example, in the target detection service, when the target object, abnormal behavior, or abnormal state is detected, the corresponding alarm information is triggered. In the image classification service, when it is determined that there is an abnormal type, classification error, or classification that does not meet the expected classification, the corresponding alarm information is triggered.

[0043] In this embodiment, the output mode of the alarm information can be real-time or timed, depending on the specific needs of the application scenario. For example, in a security monitoring system, real-time alarms are necessary to ensure timely response to potential security threats, while in some non-emergency monitoring situations, such as traffic flow analysis, timed alarms may be more appropriate to reduce unnecessary interference. In addition, the system can also self-learn and optimize based on the feedback of the alarm information, and continuously adjust and improve the performance of the image recognition alarm method and device by analyzing the accuracy of the alarm and the effectiveness of the response, thereby achieving more intelligent and automated monitoring and management.

[0044] In this embodiment, the image recognition alarm is applied in different business scenarios, such as the aforementioned target detection business and image classification business. The secondary verification reasoning method will also be different. Among them, in the target detection business, the model reasoning can obtain the type and location information of the target in the original image, so it can be judged whether the type and location information in the two reasoning results match, so as to output the alarm information associated with the corresponding original image. In the image classification business, the model reasoning can obtain the category label of the original image, so it can be judged whether the category labels in the two reasoning results are the same.

[0045] Optionally, in one embodiment, the preliminary reasoning result includes a first type and a first position information of the target in the original image, and the verification reasoning result includes a second type and a second position information of the target in the original image. In response to the verification reasoning result matching the corresponding preliminary reasoning result, outputting warning information associated with the corresponding original image includes: in response to the first type and the second type being the same, and the overlap of the first position information and the second position information being greater than a preset overlap threshold, outputting warning information.

[0046] In this optional embodiment, the implementation method of the image recognition alarm of the present application in application scenarios including various target detection is specifically described. Among them, when the first type and the second type are the same, it indicates that the detected targets belong to the same category, and the overlap of the position information reflects the positioning accuracy of the target in the image. When the overlap exceeds the preset threshold, it can be considered that the target is accurately identified, and the system will output the corresponding alarm information. Among them, the setting of the preset threshold can be determined by experiments and statistical analysis, and adjusted according to the requirements of accuracy and specific application scenarios. For example, in a security monitoring scenario, if the preliminary reasoning result shows that there is a "suspicious person" in the image, and the verification reasoning result confirms this judgment, and the overlap of the position information exceeds the set threshold, the system will trigger an alarm and notify the security personnel to conduct further inspection or intervention. In this way, the image recognition alarm system can effectively assist monitoring personnel, improve monitoring efficiency and response speed, while reducing false alarms and missed alarms, and ensuring the reliability of the monitoring system.

[0047] In this optional embodiment, the overlap can be obtained by a variety of numerical standards, such as the intersection-and-union ratio, Euclidean distance, Manhattan distance or cosine similarity of the detection frame corresponding to the first position information and the detection frame corresponding to the second position information. These numerical standards can quantify the degree of matching of the position information, thereby providing a basis for judging whether the target is accurately identified. For example, the intersection-and-union ratio is a common method to measure the overlap degree of two detection frames. It evaluates the overlap degree by calculating the ratio of the intersection area to the union area of ​​the two detection frames. When the intersection-and-union ratio exceeds a preset threshold, it can be considered that the positioning of the target is accurate. The Manhattan distance takes into account the distance between the center points of the detection frames and is suitable for scenes where the direction and speed of target movement need to be considered. Cosine similarity evaluates the similarity of the two detection frames from the direction, which is suitable for situations where the target may have rotational changes in the image. Through these different numerical standards, the system can flexibly adapt to various complex scenes and improve the accuracy and reliability of alarms.

[0048] In this optional embodiment, taking the calculation of the intersection-and-union ratio as an example, it can be specifically based on the formula Determine, among which, B A =(x A ,y A ,w A ,h A ), B A is the target frame coordinate of the first position information, (x A ,y A ) is the coordinate of the target frame of the first position information relative to the upper left corner of the original image, w A is the width of the target frame of the first position information, h A is the height of the target frame of the first position information, B B =(x B ,y B ,w B ,h B ), B B is the target frame coordinate of the second position information, (x B ,y B ) is the coordinate of the target frame of the second position information relative to the upper left corner of the original image, w B is the width of the target frame coordinates of the second position information, h B is the height of the target frame coordinates of the first position information, Area(B A ∩B B ) is the area of ​​the intersection between the target frame coordinates of the first position information and the target frame coordinates of the second position information, Area(B A ∪B B) is the area of ​​the union of the target frame coordinates of the first position information and the target frame coordinates of the second position information. A , B B ) is greater than a certain threshold value such as 0.5, the preliminary reasoning result and the verification reasoning result are judged to match, and then the corresponding alarm information is output.

[0049] Optionally, in one embodiment, the preliminary reasoning result includes a first category label of the target event in the original image, and the verification reasoning result includes a second category label of the target event in the original image. In response to the verification reasoning result matching the corresponding preliminary reasoning result, outputting warning information associated with the corresponding original image includes: in response to the first category label and the second category label being the same, outputting warning information.

[0050] In this optional embodiment, the implementation method of the image recognition alarm of the present application in application scenarios including various image classifications is specifically described. Among them, the sameness of the first category label and the second category label indicates that the two reasoning results are consistent, and the category of the target in the image is confirmed. In practical applications, for example, in the scene of judging a person falling, the small computer vision model classifies the original image according to "normal scene" and "personnel fall scene". If the preliminary reasoning result identifies that there is a person falling in the image, the original image in which the person falling is identified is input into the large computer vision model for verification reasoning. If the verification reasoning result also confirms that there is a person falling, the corresponding alarm information will be output to notify the service personnel to provide assistance. This double verification mechanism greatly improves the accuracy and reliability of the image recognition alarm system, ensuring the efficient operation of image classification and related monitoring applications.

[0051] See also Figure 2 , Figure 2 It is a structural diagram of an embodiment of an image recognition alarm device provided by the present application.

[0052] The image recognition alarm device 40 includes: a first recognition module 41, which is used to perform preliminary reasoning on the original image based on a preset computer vision small model to obtain a preliminary reasoning result of the original image; a second recognition module 42, which is used to use the original image whose preliminary reasoning result meets the preset preliminary screening conditions as the input of the preset computer vision large model to perform verification reasoning on the original image to obtain a verification reasoning result corresponding to the original image; a verification module 43, which is used to output warning information associated with the corresponding original image in response to the verification reasoning result matching the corresponding preliminary reasoning result; wherein the parameter magnitude of the computer vision small model is smaller than the parameter magnitude of the computer vision large model.

[0053] Optionally, in some embodiments, the preliminary reasoning result includes a first type and a first position information of the target in the original image, and the verification reasoning result includes a second type and a second position information of the target in the original image; the verification module 43 is also used to: output an alarm message in response to the first type and the second type being the same, and the overlap between the first position information and the second position information being greater than a preset overlap threshold.

[0054] Optionally, in some embodiments, the preliminary reasoning result includes a first category label of the target event in the original image, and the verification reasoning result includes a second category label of the target event in the original image; the verification module 43 is also used to: output an alarm message in response to the first category label and the second category label being the same.

[0055] Optionally, in some embodiments, the small computer vision model is obtained by training a small image recognition model based on the first sample set; and the large computer vision model is obtained by fine-tuning a pre-trained large model based on the second sample set.

[0056] Optionally, in some embodiments, the pre-trained large model includes a Grounding DINO model and / or a CLIP model.

[0057] Optionally, in some embodiments, the first sample set and the second sample set are the same preset sample set, and the preset sample set includes a first preset number of positive samples and a second preset number of negative samples.

[0058] Since the embodiments of the device part correspond to the embodiments of the above-mentioned method, please refer to the above-mentioned method embodiment for the introduction of the image recognition alarm device 40 provided in the embodiment of the present invention. The embodiment of the present invention will not be repeated here, and it has the same beneficial effects as the above-mentioned image recognition alarm method.

[0059] See also Figure 3 , Figure 3 It is a structural diagram of an embodiment of the storage medium provided by the present application.

[0060] The storage medium 50 stores program data 51. When the program data 51 is executed by the processor, the following is achieved: Figure 1 The described image recognition alarm method.

[0061] The program data 51 is stored in a storage medium 50, and includes a number of instructions for enabling a network device (such as a router, a personal computer, a server, or other network device) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application.

[0062] Optionally, the storage medium 50 may be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or other medium that can store the program data 51 .

[0063] See also Figure 4 , Figure 4 It is a structural schematic diagram of an embodiment of a computer device provided by the present application.

[0064] The computer device 60 includes a processor 62 and a memory 61 connected to each other. The memory 61 stores a computer program. When the processor 62 executes the computer program, the following is achieved: Figure 1 The image recognition alarm method described herein. The memory 61 may include the storage medium 50, or may be other independently developed memory.

[0065] See also Figure 5 , Figure 5 It is a structural schematic diagram of an embodiment of the monitoring system provided by the present application.

[0066] The monitoring system 70 includes a camera 71 and a Figure 4 The described computer device 60 is in communication connection with a camera 71, and the camera 71 is used to continuously acquire original images. The monitoring system 70 can be a system for monitoring a specific area, such as a shopping mall, a school, a factory, etc. The camera 71 can be fixed or mobile, and can perform real-time monitoring or timed snapshots as needed. By analyzing the original images provided by the camera 71, the computer device 60 can promptly detect abnormal events, such as a person falling, the appearance of controlled items, and the appearance of fire characteristics, and trigger a corresponding alarm mechanism. In addition, the monitoring system 70 can be equipped with a computer device 60 with a storage medium 50, or can be equipped with a computer device 60 without a storage medium 50, or the monitoring system 70 is a type of system that can directly read the storage medium 50.

[0067] Different from the prior art, the present application discloses an image recognition alarm method, device, medium, equipment and system. By using a preset computer vision small model to perform preliminary reasoning on the original image, the original image with an alarm risk can be quickly screened out, the amount of data to be processed by the subsequent large model reasoning can be reduced, the requirement for the large model reasoning speed can be reduced, and the reasoning efficiency can be improved. By using a preset computer vision large model to perform secondary verification reasoning on the screened original image, the alarm information associated with the corresponding original image can be determined more accurately and reliably. At the same time, based on the scheme of the present application, only a small amount of material is needed to train the small model and fine-tune the large model to achieve a high image recognition accuracy rate, which can significantly reduce the material demand, reduce the consumption of manpower and computing power, and make the scheme of the present application particularly applicable in the case of difficulty in collecting materials in abnormal scenes. In addition, due to the introduction of a large computer vision model and the use of the excellent generalization ability of the large model, the present application can still achieve a high accuracy rate after changing the deployment location and camera, avoiding the re-collection, training and deployment optimization of the material, which is conducive to improving the scene compatibility of image recognition alarms and realizing the application of image recognition alarms.

[0068] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, storage medium embodiment, computer equipment embodiment and monitoring system embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0069] The present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0070] In the several embodiments provided in this application, it should be understood that the disclosed methods, devices, storage media, computer equipment, and monitoring systems can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0071] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image recognition alarm method, characterized in that: include: Performing preliminary reasoning on the original image based on a preset small computer vision model to obtain a preliminary reasoning result of the original image; The original image whose preliminary reasoning result satisfies the preset preliminary screening condition is used as the input of the preset computer vision large model to perform verification reasoning on the original image to obtain a verification reasoning result corresponding to the original image; In response to the verification reasoning result matching the corresponding preliminary reasoning result, outputting warning information associated with the corresponding original image; Among them, the parameter magnitude of the small computer vision model is smaller than the parameter magnitude of the large computer vision model.

2. The image recognition alarm method according to claim 1, characterized in that: The preliminary reasoning result includes a first type and a first position information of the target in the original image, and the verification reasoning result includes a second type and a second position information of the target in the original image; In response to the verifying reasoning result matching the corresponding preliminary reasoning result, outputting warning information associated with the corresponding original image includes: In response to the first type and the second type being the same and the overlap between the first location information and the second location information being greater than a preset overlap threshold, the warning information is output.

3. The image recognition alarm method according to claim 1, characterized in that: The preliminary reasoning result includes a first category label of the target event in the original image, and the verification reasoning result includes a second category label of the target event in the original image; In response to the verifying reasoning result matching the corresponding preliminary reasoning result, outputting warning information associated with the corresponding original image includes: In response to the first category label and the second category label being the same, outputting the warning information.

4. The image recognition alarm method according to claim 1, characterized in that: The computer vision small model is obtained by training the image recognition small model based on the first sample set; The computer vision large model is obtained by fine-tuning the pre-trained large model based on the second sample set.

5. The image recognition alarm method according to claim 4, characterized in that: The pre-trained large model includes a Grounding DINO model and / or a CLIP model.

6. The image recognition alarm method according to claim 4, characterized in that: The first sample set and the second sample set are the same preset sample set, and the preset sample set includes a first preset number of positive samples and a second preset number of negative samples.

7. An image recognition alarm device, characterized in that: include: A first recognition module, used for performing preliminary reasoning on the original image based on a preset computer vision small model to obtain a preliminary reasoning result of the original image; A second recognition module is used to use the original image whose preliminary reasoning result meets the preset preliminary screening condition as an input of a preset computer vision large model to perform verification reasoning on the original image to obtain a verification reasoning result corresponding to the original image; a verification module, configured to output warning information associated with the corresponding original image in response to the verification reasoning result matching the corresponding preliminary reasoning result; Among them, the parameter magnitude of the small computer vision model is smaller than the parameter magnitude of the large computer vision model.

8. A storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the steps of the image recognition alarm method according to any one of claims 1 to 6 are implemented.

9. A computer device, characterized in that: The method comprises a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the image recognition alarm method according to any one of claims 1 to 6 are implemented.

10. A monitoring system, characterized in that: The monitoring system comprises a camera and the computer device as claimed in claim 9, wherein the computer device is in communication connection with the camera, and the camera is used to continuously acquire raw images.

Images