Video analysis alarm method, device, system, equipment and medium

Abstract

The invention relates to the technical field of video analysis, in particular to a video analysis alarm method, device, system, equipment and medium, a first alarm result sent by video analysis equipment after the video analysis equipment analyzes a real-time video is received, the first alarm result comprises a scene name, and according to the scene name, the video analysis equipment analyzes the real-time video according to the scene name; and extracting a target scene judgment rule and a target large model cue word corresponding to the scene name, and according to the target scene judgment rule and the target large model cue word, performing secondary rechecking on the first alarm result by using a preset multi-modal large model to obtain a second alarm result. Visibly, after the first alarm result sent after the video analysis device analyzes the real-time video is obtained, the first alarm result is re-checked for the second time through the preset multi-mode large model, so that the video analysis alarm problem is further accurately and effectively identified on the premise of controlling the overall cost, and the accuracy of the alarm result is improved.

Description

Technical Field

[0001] The present invention relates to the field of video analysis technology, and in particular to a video analysis alarm method, device, system, equipment and medium. Background Art

[0002] In recent years, with the continuous development of new-generation information technologies such as artificial intelligence, cloud computing, big data, and the Internet of Things, video analysis has gradually been used extensively in scenarios such as smart cities, smart parks, and smart communities to replace manual inspections and other tasks.

[0003] However, due to factors such as equipment performance and cost, traditional video analysis can only load models with small parameter scale for analysis, which will also lead to large-scale erroneous analysis alarms, thereby reducing the accuracy of the final alarm. Therefore, how to accurately and effectively identify video analysis alarm problems and improve the accuracy of alarm results has become an urgent problem to be solved. Summary of the invention

[0004] Based on this, it is necessary to provide a video analysis alarm method, device, system, equipment and medium to address the above technical problems, so as to solve the problem that the existing technology cannot accurately and effectively identify video analysis alarms, thereby reducing the accuracy of the alarm results.

[0005] A first aspect of an embodiment of the present application provides a video analysis alarm method, the video analysis alarm method comprising: Receiving a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name; According to the scene name, extracting the target scene determination rule and the target large model prompt word corresponding to the scene name; According to the target scene determination rule and the target large model prompt word, the first alarm result is rechecked using a preset multimodal large model to obtain a second alarm result.

[0006] A second aspect of an embodiment of the present application provides a video analysis alarm device, the video analysis alarm device comprising: A receiving module, configured to receive a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name; An extraction module, used for extracting target scene determination rules and target large model prompt words corresponding to the scene name according to the scene name; The review module is used to perform a secondary review on the first alarm result according to the target scene determination rule and the target large model prompt word using a preset multimodal large model to obtain a second alarm result.

[0007] A third aspect of an embodiment of the present application provides a video analysis alarm system, the video analysis alarm system comprising a camera, a video analysis device, and a cloud platform; The camera is used to collect real-time video within the target monitoring area; The video analysis device is used to perform video analysis on the real-time video, obtain a first alarm result, and report it to the cloud platform; The cloud platform is used to receive a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name; according to the scene name, a target scene determination rule and a target large model prompt word corresponding to the scene name are extracted; according to the target scene determination rule and the target large model prompt word, a preset multimodal large model is used to perform a second review on the first alarm result to obtain a second alarm result.

[0008] In a fourth aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the video analysis alarm control method as described in the second aspect when executing the computer program.

[0009] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the video analysis alarm control method as described in the second aspect is implemented.

[0010] In summary, the present invention provides a video analysis alarm method, apparatus, system, equipment and medium, which receives a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name, and according to the scene name, extracts a target scene determination rule and a target large model prompt word corresponding to the scene name, and according to the target scene determination rule and the target large model prompt word, performs a secondary review of the first alarm result using a preset multimodal large model to obtain a second alarm result. It can be seen that after obtaining the first alarm result sent by the video analysis device after analyzing the real-time video, the present application performs a secondary review of the first alarm result using a preset multimodal large model, so as to further accurately and effectively identify the video analysis alarm problem under the premise of controlling the overall cost, thereby improving the accuracy of the alarm result. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0012] Figure 1 It is a structural diagram of a video analysis alarm system provided by an embodiment of the present invention; Figure 2 It is a flowchart of a video analysis alarm method provided by an embodiment of the present invention; Figure 3 It is a structural schematic diagram of a video analysis alarm device provided by an embodiment of the present invention; Figure 4 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0015] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0016] As used in the present specification and the appended claims, the term “if” may be interpreted as “when” or “uponce” or “in response to determining”, depending on the context. Similarly, the phrases “if it is determined” or “if matched to [described condition or event]” may be interpreted as meaning “upon determination” or “in response to determination” or “uponce matched to [described condition or event]” or “in response to matching to [described condition or event]”, depending on the context.

[0017] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0018] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0019] It should be understood that the order of execution of the steps in the following embodiments does not imply a precedence of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0020] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0021] See also Figure 1 , Figure 1 This is a structural diagram of a video analysis alarm system provided by an embodiment of the present invention. The video analysis alarm system includes at least a camera, a video analysis device and a cloud platform. The camera, the video analysis device and the cloud platform can be directly or indirectly connected by wired or wireless communication, which is not specifically limited in this application.

[0022] A camera is any device with a camera function, which is set at a position where it can capture a target area. And send the real-time video taken by itself to the video analysis device (which can be run on the video analysis device, and the video analysis device can be set on an external server, a computer or the camera). The number of cameras can be one or more, and this application does not make specific restrictions here.

[0023] The video analysis device can be a smart phone, a video analysis box, a tablet computer, a laptop computer, a desktop computer, a smart home appliance, an intelligent voice interaction device, an intelligent vehicle-mounted device, etc. It receives the real-time video sent by the camera and uses image analysis technology to analyze the real-time video to determine whether there is a situation that meets the scene determination rule in the real-time video stream. Among them, the scene determination rule is a pre-set alarm event. When the alarm event is met, the first alarm result is obtained, and then the first alarm result is transmitted to the cloud platform.

[0024] The number of cloud platforms can be one or more. The cloud platform is used to provide the user end with relevant business services of the company or enterprise. By receiving the first alarm result sent by the video analysis device after analyzing the real-time video, since the first alarm result includes the scene name, the cloud platform will extract the target scene judgment rule and the target large model prompt word corresponding to the scene name according to the scene name, and then use the preset multimodal large model to conduct a secondary review of the first alarm result according to the target scene judgment rule and the target large model prompt word, and obtain the second alarm result, so as to send the alarm information corresponding to the second alarm result to the video analysis alarm system, so that the video analysis alarm system performs corresponding processing. The cloud platform may include an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and may also include cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The cloud platform can be a platform with security management and property management capabilities such as smart cities and smart communities, and this application does not make any restrictions on this.

[0025] Through the architecture of the above-mentioned video analysis alarm system, the first alarm result obtained by the video analysis equipment when analyzing the real-time video can be re-checked through the preset multimodal large model, which is beneficial to improve the recognition efficiency and accuracy of alarm information in different scenarios, reduce the false alarm rate, and thus improve the accuracy of the final video analysis alarm result.

[0026] It can be understood that the structural diagram of the video analysis alarm system described in the embodiment of the present application is for more clearly illustrating the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the system structure and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0027] It can be seen that the above Figure 1 The system architecture shown is only an example and is not specifically limited in the embodiments of the present application.

[0028] See also Figure 2 , is a flow chart of a video analysis alarm method provided by an embodiment of the present invention. The video analysis alarm method can be applied to Figure 1 Video analysis alarm system in Figure 2 As shown, the video analysis alarm method may include the following steps.

[0029] S201: Receive a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name.

[0030] In step S201, the real-time video in the target monitoring area is collected, wherein the target monitoring area can be set according to the needs of the user, or according to the needs of the use scenario, and this application does not make any restrictions on this. After acquiring the real-time video, the real-time video is sent to the video analysis device so that the video analysis device receives the real-time video and analyzes the real-time video, and then determines whether an alarm event exceeding the threshold of the scene determination rule occurs in the real-time video based on the scene determination rule. If there is an alarm event exceeding the threshold of the scene determination rule, an alarm is triggered, and a first alarm result is generated, and the first alarm result is reported to the cloud platform so that the cloud platform receives the first alarm result sent by the video analysis device after analyzing the real-time video. Among them, the scene determination rule can be implemented by cloud platform programming, remote configuration or local configuration, and the embodiment of this application does not limit the implementation method of the scene determination rule. Among them, the first alarm result can be presented in the form of an alarm sheet, and the alarm sheet includes the alarm object, the alarm occurrence time, the scene name, the alarm screenshot and other contents.

[0031] Exemplarily, when monitoring the scene of office personnel, the camera collects real-time personnel video in the target office area and sends it to the video analysis device. The video analysis device analyzes the real-time personnel video and determines that the current office personnel are not in the predetermined target office area based on the scene judgment rule. Then, an alarm will be triggered and a first alarm result will be generated. Then, the office personnel who are not in the predetermined target office area will be reported to the cloud platform so that the cloud platform can determine whether the alarm is true. It should be noted that in addition to office buildings, similar rules also exist for alarms in residential projects, industrial parks, schools and other places, which are not specifically limited in this embodiment. For example, within the target area, if someone is identified to appear in a certain monitoring area for the first time, it is determined that an alarm event occurs in the real-time video; or, if someone is identified to not wear a helmet in a certain monitoring area for the first time, it is determined that an alarm event occurs in the real-time video. The above only lists examples of alarm events, but is not limited to this. It should be known that any recognition situation obtained by type recognition and behavior recognition of objects is covered within the scope of protection of this disclosure. The object type recognition and behavior recognition disclosed in the present invention can be applied to regional anti-theft monitoring (such as anti-theft of vehicles, cultural relics, etc.), regional personnel monitoring (people are not allowed to appear in certain areas), construction specification monitoring (certain construction areas must wear corresponding safety equipment), concentration monitoring (such as monitoring whether there is cheating in the examination room, monitoring whether there is anyone sleeping or talking in class, etc.), etc. According to different real-time video scenes, the algorithms or rules used are different, which may be leaving the sleeping post, smoking, high temperature, overflowing trash cans, etc., but the scene differences do not affect the overall process, and this application does not make any limitations on this.

[0032] In this embodiment, the cloud platform tests the accuracy of the alarm results in combination with business processes including data reception, data storage and data processing, achieving full coverage testing of cloud platform monitoring alarm processing scenarios. Through the above steps, this method can greatly improve the efficiency of video analysis alarms, reduce false alarm rates, and thus improve the accuracy of subsequent alarm results.

[0033] S202: According to the scene name, extract the target scene determination rule and the target large model prompt word corresponding to the scene name.

[0034] In step S202, the present application extracts the target scene determination rules and target large model prompt words corresponding to the scene name according to the scene name in the first alarm result, so that the first alarm result can be reviewed and verified more quickly later. Among them, the scene name is mainly used to distinguish the types of objects or behaviors to be identified, such as the above-mentioned personnel, vehicles, leaving the post, smoking, etc., and then can effectively distinguish different types of objects or behaviors, which is convenient for subsequent data processing and analysis. Of course, in other embodiments, the scene name can also be an identification of a specific environment or situation, which is usually used to classify or describe a certain monitoring area, and can be set according to actual conditions. This application does not make any restrictions on this. For example, in a monitoring system, there may be scene names such as "parking lot", "store", "school", "road", etc., which can help the system quickly understand the target area of ​​monitoring. The scene determination rule is used to describe the logic or criteria for how to identify or detect related behaviors or events in a specific scene. These rules may include threshold settings or logical judgments for specific behaviors.

[0035] For example, in a parking lot scenario, the decision rule may be "If it is detected that the vehicle in the parking area is not parked within the normal time, it is determined to be abnormal parking." In a store scenario, the decision rule may be "If it is detected that a customer stays at the checkout counter for more than five minutes and does not check out, it is determined to be suspicious behavior." Large model prompts are keywords or phrases used to guide or prompt large-scale machine learning models (such as natural language processing models). These prompts can help the model better understand user intent or data context. For example: For the "parking lot" scenario, prompts may include "vehicle parking", "abnormal behavior detection", "surveillance video", etc. For the "store" scenario, prompts may include "customer behavior", "shopping patterns", "theft monitoring", etc.

[0036] It should be noted that before obtaining the scene name in the first alarm result, the first alarm result needs to be cleaned to remove noise information and useless words, such as repeated time information, server information, and words without special meaning. This application does not impose any restrictions on this.

[0037] In an embodiment of the invention, according to the scene name, the target scene determination rule and the target large model prompt word corresponding to the scene name are extracted, including: Matching the scene name with the scene name configured by the scene item in the preset video scene library; If the match is successful, then extracting the target scene determination rule and the target large model prompt word corresponding to the scene name from the preset video scene library; If the match fails, the scene name and the preset prompt word are input into the preset large language model. Obtaining a reference scene determination rule and a reference large model prompt word, and using the reference scene determination rule and the reference large model prompt word as a target scene determination rule and a target large model prompt word; The reference scene determination rule and the reference large model prompt word are bound to the scene name to obtain a new scene item, and the preset video scene library is updated using the new scene item.

[0038] Specifically, a video scene library is first pre-established in the cloud platform. The video scene library is used to store and manage various scene information related to video surveillance. It usually contains descriptions, features, judgment rules and related data of multiple video scenes, aiming to support applications such as video analysis and intelligent monitoring. By configuring the video scene library, accurate detection of alarm events can be achieved later. Table 1 shows the video scene library: In the embodiments of the present application, considering that in actual scenarios, there is a lot of preliminary preparation work to be done to issue an alarm for a specific scenario, the cloud platform can flexibly configure the scene items in the video scene library to adapt to the changes in the video analysis alarm requirements, which can more efficiently and comprehensively cover the testing of various demand scenarios for cloud platform monitoring and alarm, and associate each scene as a whole, making the test alarm results more referenceable, thereby effectively solving the problem of cumbersome and inefficient testing of cloud platform monitoring and alarm results. It should be noted that this application does not impose any restrictions on the number of scene items in the video scene library.

[0039] After the video scene library is established, the scene name of the first alarm result is matched with the scene name configured by the scene item in the video scene library. If the match is successful, the target scene judgment rule and the target large model prompt word corresponding to the scene name are directly extracted from the video scene library to execute the subsequent operation process. If the match fails, the scene name and the preset prompt word are input into the preset large language model. The large language model is a deep learning model trained with a large amount of text data. It can generate natural language text or understand the meaning of language text. The corresponding reference scene judgment rule and reference large model prompt word are generated through the large language model, and then the reference scene judgment rule and the reference large model prompt word are input into the preset large language model. The rules and reference large model prompt words are used as the target scene judgment rules and target large model prompt words. Among them, the setting of preset prompt words requires the pre-determination of the scene type, and then a detailed analysis of each scene is performed to identify the key features and behaviors related to the scene. Then, based on the scene characteristics, relevant keywords and phrases are collected. These keywords should cover the main activities of the scene, possible abnormal behaviors and security-related content. Finally, the collected prompt words are classified and organized for subsequent use. For example, if the scene is an electric vehicle charging station, the preset prompt words can be electric vehicle, charging, charging pile, monitoring, safety, abnormality, stay, charging status, staying too long, etc.

[0040] It should be noted that the preset prompt words are only examples of structural integrity. For example: please identify the scene name based on the video, give the scene determination rules in the format of "[XX] situation exists in the picture and lasts for [YY] time", "[XX] situation exists in the picture", and give the prompt words for use in the multimodal large model in the format of "Based on the above image, determine the [number of people] in the picture, only feedback the [number of people] number, do not feedback the rest of the content". This application does not make any restrictions on this.

[0041] For example, the scene name is parking lot. If the corresponding parking lot also appears in the video scene library, the target scene judgment rule and target large model prompt word corresponding to the parking lot are extracted, that is, the target scene judgment rule: if a parked vehicle is detected to be parked at an abnormal time, it is judged as abnormal parking behavior; target large model prompt words: vehicle parking, abnormal behavior, parking lot monitoring. If there is no matching parking lot in the video scene library, and the prompt words are pre-set as vehicle prompt words, the parking lot and vehicle prompt words are input into the large language model, and then the corresponding reference scene judgment rule and reference large model prompt words are generated, that is, the reference scene judgment rule: if a parked vehicle is detected to be parked at an abnormal time, it is judged as abnormal parking behavior; reference large model prompt words: vehicle parking, abnormal behavior, parking lot monitoring.

[0042] In the embodiment of the present application, when the scene name of the first alarm result does not match the scene name configured by the scene item in the video scene library, the reference scene determination rule and the reference large model prompt word generated by the large language model are bound to the above scene name to obtain a new scene item, and the new scene item is entered into the video scene library to complete the update of the video scene library. In the above manner, there is no need to manually update the data, which improves the update efficiency and accuracy of the video scene library, reduces false positives and false negatives, and improves the reliability and accuracy of subsequent alarm results.

[0043] In this embodiment, according to the scene name, the target scene determination rules and target large model prompt words corresponding to the scene name are extracted, which can ensure fast and accurate alarm behavior recognition and response, thereby enhancing the overall performance and user experience of video analysis alarms, so that video analysis alarms can be better performed in the future.

[0044] S203: According to the target scene determination rule and the target large model prompt word, the first alarm result is secondarily reviewed by using a preset multimodal large model to obtain a second alarm result.

[0045] In step S203, the multimodal big model is a deep learning model that combines multimodal information such as text, images, videos, and audio for training. Here, only the ability to parse images is required, and the other video and audio capabilities are not required. The role of the multimodal big model here is to reduce false alarms. By using the target scene judgment rules and the target big model prompt words, the preset multimodal big model can conduct a second review of the first alarm result, which can effectively improve the accuracy of the alarm result and ensure more reliable monitoring results in complex scenes.

[0046] In an embodiment of the invention, a second review is performed on the first alarm result using a preset multimodal large model to obtain a second alarm result, including: Inputting the target large model prompt word, the target scene determination rule and the first alarm result into a preset multimodal large model, so that the preset multimodal large model performs image analysis to obtain a first analysis result, wherein the first analysis result is represented by a matching result of the first alarm screenshot and the scene name; If the first alarm screenshot matches the scene name, determining that the first parsing result is true and determining that it is a real alarm; If the first alarm screenshot does not match the scene name, the first parsing result is determined to be false and is judged to be a false alarm.

[0047] In this embodiment, when the first alarm result includes a first alarm screenshot, and the first alarm screenshot is a clear alarm screenshot, the target large model prompt word, the target scene judgment rule and the first alarm result are used as input parameters of the multimodal large model, so that the multimodal large model performs image analysis on the first alarm screenshot to obtain a first analysis result, wherein the first analysis result is represented by the matching result of the first alarm screenshot and the scene name, that is, if the first alarm screenshot matches the scene name, the first analysis result is determined to be true, and the first alarm result is determined to be a true alarm; if the first alarm screenshot does not match the scene name, the first analysis result is determined to be false, and the first alarm result is determined to be a false alarm.

[0048] For example, if it is a school monitoring scene, the target large model prompt words are school, students, and playground. The target scene judgment rule is that there should be normal student activities in the school area. The scene name is school. The first alarm screenshot shows a group of students on the playground, and there are suspicious people outside the playground. The first alarm result is that there are suspicious people outside the playground, then an alarm is issued. If the output is that the school situation is shown in the first alarm screenshot, but the suspicious person is outside the playground, it is determined that the first analysis result is true (school activities are normal, but attention should be paid to external situations), and then the first alarm result is determined to be a true alarm; if it is a traffic intersection monitoring scene, the scene name is traffic intersection, the first alarm screenshot shows a car running a red light, the target large model prompt words are traffic, red light, and violation, and the target scene judgment rule is that vehicles must not enter the intersection when the red light is on. The first alarm result is that there is a car running a red light, then an alarm is issued. If the output is that the vehicle in the first alarm screenshot is driving normally, it is determined that the first analysis result is false, and then the first alarm result is a false alarm.

[0049] In an embodiment of the invention, a second review is performed on the first alarm result using a preset multimodal large model to obtain a second alarm result, and the method further includes: Inputting the target large model prompt word, the target scene determination rule and the first alarm result into a preset multimodal large model, so that the preset multimodal large model performs image analysis to obtain a second analysis result, wherein the second analysis result is represented by a matching result of the first alarm screenshot and the scene name, and a matching result of the second alarm screenshot and the scene name; If the first alarm screenshot matches the scene name, and the second alarm screenshot matches the scene name, determining that the second analysis result is true and determining that it is a real alarm; If the first alarm screenshot does not match the scene name, and the second alarm screenshot does not match the scene name, determining that the second analysis result is false and determining it to be a false alarm; If the first alarm screenshot matches the scene name, and the second alarm screenshot does not match the scene name, determining that the second analysis result is false and suspicious, and determining it as a false and suspicious alarm; If the first alarm screenshot does not match the scene name, and the second alarm screenshot matches the scene name, the second analysis result is determined to be true or doubtful, and is determined to be a true or doubtful alarm.

[0050] In this embodiment, when the first alarm result includes a first alarm screenshot, and the first alarm screenshot is an unclear alarm screenshot, it is necessary to determine whether the clarity of the first alarm screenshot is lower than a preset clarity threshold, wherein the preset clarity threshold can be set according to actual conditions, and the present application does not impose any limitation on this. For example, rainy days, nights, backlighting, and the like may cause the first alarm screenshot to have a lower contrast. If the clarity of the first alarm screenshot is lower than the preset clarity threshold, the first alarm screenshot is enhanced, such as by adjusting the contrast and brightness, etc., so as to obtain a second alarm screenshot having a clarity higher than the preset clarity threshold. It should be noted that a convolution operation can be performed using a Laplace mask, and then the standard deviation can be calculated, and the resulting value can represent the clarity of the image. The clarity can also be calculated in other ways, and the present application does not impose any limitation on this.

[0051] After obtaining the second alarm screenshot, the target large model prompt word, the target scene determination rule and the first alarm result are used as input parameters of the multimodal large model, so that the multimodal large model performs image analysis on the first alarm screenshot and the second alarm screenshot, thereby obtaining a second analysis result, wherein the second analysis result is represented by the matching result of the first alarm screenshot and the scene name, and the matching result of the second alarm screenshot and the scene name, that is, if the first alarm screenshot matches the scene name, the first analysis result is determined to be true, and the first alarm result is determined to be a true alarm; if the first alarm screenshot does not match the scene name, and the second alarm screenshot matches the scene name, then the first analysis result is determined to be true, and the first alarm result is determined to be a true alarm; If the first alarm screenshot does not match the scene name, and the second alarm screenshot does not match the scene name, then the second analysis result is determined to be true, and the first alarm result is determined to be a true alarm. If the first alarm screenshot does not match the scene name, and the second alarm screenshot does not match the scene name, then the second analysis result is determined to be false, and the first alarm result is determined to be a false alarm. If the first alarm screenshot matches the scene name, and the second alarm screenshot does not match the scene name, then the second analysis result is determined to be false and questionable, and the first alarm result is determined to be a false and questionable alarm. If the first alarm screenshot does not match the scene name, and the second alarm screenshot matches the scene name, then the second analysis result is determined to be true and questionable, and the first alarm result is determined to be a true and questionable alarm.

[0052] In the above manner, the multimodal large model is used to analyze the first alarm screenshot or the first alarm screenshot and the second alarm screenshot to determine whether the first alarm result is true or false or questionable. This process relies on combining the target large model prompt words and target scene judgment rules with the actual detection results, and re-checking the first alarm result to reduce the false alarm rate, thereby improving the accuracy of the final video analysis alarm result.

[0053] In an embodiment of the invention, after obtaining the second alarm result, the method further includes: If it is a real alarm, a first alarm work order is generated, wherein the first alarm work order is used to instruct the user to handle the alarm event of the current scene; If it is a false alarm, the alarm event will be recorded and archived; If it is a false doubtful alarm or a true doubtful alarm, a second alarm work order is generated, wherein the second alarm work order is used to instruct the user to re-check the second alarm result.

[0054] Specifically, for real alarms, the cloud platform generates a first alarm work order, so that the cloud platform can notify the property personnel based on the first alarm work order to promptly and accurately handle the alarm events in the current scene, such as returning to work, cleaning, eliminating early fires, etc., thereby improving the quality and efficiency of alarm event processing. For false alarms, the cloud platform records and archives the alarm events corresponding to the false alarms. For false or true alarms, the cloud platform generates a second alarm work order, so that the cloud platform can notify the property personnel based on the second alarm work order to review, confirm and handle the second alarm results. In the above manner, the efficiency and accuracy of alarm processing can be effectively improved, resource allocation can be optimized, user experience can be enhanced, and support can be provided for the continuous improvement of the video analysis alarm system.

[0055] In an embodiment of the invention, after obtaining the second alarm result, the method further includes: Performing statistical analysis on the first analysis result or the second analysis result to determine whether there are a large number of false alarm events in the target scene corresponding to the first alarm result; If there are a large number of false alarm events in the target scene corresponding to the first alarm result, the target scene judgment rule and the historical alarm screenshot corresponding to the first alarm result are input into the preset multimodal large model, so that the preset multimodal large model adjusts the threshold and generates a judgment rule adjustment suggestion; The judgment rule adjustment suggestion is sent to the user, so that after the user confirms, the target scene judgment rule in the preset video scene library is updated.

[0056] Specifically, after obtaining the first analysis result or the second analysis result, a statistical analysis is performed on the first analysis result or the second analysis result to determine whether there are a large number of false alarm events in the target scene corresponding to the first alarm result (that is, a large number of analysis results are false). If there are a large number of false alarm events in the target scene corresponding to the first alarm result, the target scene judgment rule and historical alarm screenshots corresponding to the second alarm result are input into the preset multimodal large model. The multimodal large model analyzes the common characteristics and adjusts the threshold to generate a judgment rule adjustment suggestion. The generated judgment rule adjustment suggestion is sent to the system administrator, who determines whether to adopt it. If adopted, the target scene judgment rule in the preset video scene library is updated in a timely and accurate manner, thereby reducing the false alarm rate, improving user experience, and realizing dynamic optimization decision-making of the rules, thereby improving the accuracy and efficiency of identifying alarm results.

[0057] In summary, the present invention provides a video analysis alarm method, apparatus, system, equipment and medium, which receives a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name, and according to the scene name, extracts a target scene determination rule and a target large model prompt word corresponding to the scene name, and according to the target scene determination rule and the target large model prompt word, performs a secondary review of the first alarm result using a preset multimodal large model to obtain a second alarm result. It can be seen that after obtaining the first alarm result sent by the video analysis device after analyzing the real-time video, the present application performs a secondary review of the first alarm result using a preset multimodal large model, so as to further accurately and effectively identify the video analysis alarm problem under the premise of controlling the overall cost, thereby improving the accuracy of the alarm result.

[0058] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram of the structure of the video analysis alarm device provided by an embodiment of the present invention. In this embodiment, the terminal includes various units for executing Figure 2 For details, please refer to the steps in the corresponding embodiment. Figure 2 as well as Figure 2 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 3 The video analysis and alarm device 30 includes: a receiving module 31, an extraction module 32, and a review module 33.

[0059] A receiving module 31, configured to receive a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name; An extraction module 32, for extracting a target scene determination rule and a target large model prompt word corresponding to the scene name according to the scene name; The review module 33 is used to perform a secondary review on the first alarm result according to the target scene determination rule and the target large model prompt word using a preset multimodal large model to obtain a second alarm result.

[0060] Optionally, the extraction module 32 is specifically used for: Matching the scene name with the scene name configured by the scene item in the preset video scene library; If the match is successful, then extracting the target scene determination rule and the target large model prompt word corresponding to the scene name from the preset video scene library; If the match fails, the scene name and the preset prompt word are input into the preset large language model. Obtaining a reference scene determination rule and a reference large model prompt word, and using the reference scene determination rule and the reference large model prompt word as a target scene determination rule and a target large model prompt word; The reference scene determination rule and the reference large model prompt word are bound to the scene name to obtain a new scene item, and the preset video scene library is updated using the new scene item.

[0061] Optionally, the first alarm result further includes a first alarm screenshot, and the review module 33 is specifically used for: Inputting the target large model prompt word, the target scene determination rule and the first alarm result into a preset multimodal large model, so that the preset multimodal large model performs image analysis to obtain a first analysis result, wherein the first analysis result is represented by a matching result of the first alarm screenshot and the scene name; If the first alarm screenshot matches the scene name, determining that the first parsing result is true and determining that it is a real alarm; If the first alarm screenshot does not match the scene name, the first parsing result is determined to be false and is judged to be a false alarm.

[0062] Optionally, the first alarm result further includes a first alarm screenshot and a second alarm screenshot, where the second alarm screenshot is an enhanced alarm screenshot of the first alarm screenshot, and the review module 33 is further used to: Inputting the target large model prompt word, the target scene determination rule and the first alarm result into a preset multimodal large model, so that the preset multimodal large model performs image analysis to obtain a second analysis result, wherein the second analysis result is represented by a matching result of the first alarm screenshot and the scene name, and a matching result of the second alarm screenshot and the scene name; If the first alarm screenshot matches the scene name, and the second alarm screenshot matches the scene name, determining that the second analysis result is true and determining that it is a real alarm; If the first alarm screenshot does not match the scene name, and the second alarm screenshot does not match the scene name, determining that the second analysis result is false and determining it to be a false alarm; If the first alarm screenshot matches the scene name, and the second alarm screenshot does not match the scene name, determining that the second analysis result is false and suspicious, and determining it as a false and suspicious alarm; If the first alarm screenshot does not match the scene name, and the second alarm screenshot matches the scene name, the second analysis result is determined to be true or doubtful, and is determined to be a true or doubtful alarm.

[0063] Optionally, the review module 33 is specifically used for: If it is a real alarm, a first alarm work order is generated, wherein the first alarm work order is used to instruct the user to handle the alarm event of the current scene; If it is a false alarm, the alarm event will be recorded and archived; If it is a false doubtful alarm or a true doubtful alarm, a second alarm work order is generated, wherein the second alarm work order is used to instruct the user to re-check the second alarm result.

[0064] Optionally, the review module 33 is further used to: Performing statistical analysis on the first analysis result or the second analysis result to determine whether there are a large number of false alarm events in the target scene corresponding to the first alarm result; If there are a large number of false alarm events in the target scene corresponding to the first alarm result, the target scene judgment rule and the historical alarm screenshot corresponding to the first alarm result are input into the preset multimodal large model, so that the preset multimodal large model adjusts the threshold and generates a judgment rule adjustment suggestion; The judgment rule adjustment suggestion is sent to the user, so that after the user confirms, the target scene judgment rule in the preset video scene library is updated.

[0065] It should be noted that the information interaction, execution process and other contents between the above-mentioned units are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0066] Figure 4 Schematic diagram of the structure of a computer device provided by an embodiment of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps in any of the above-mentioned video analysis alarm embodiments are implemented.

[0067] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 4 These are merely examples of computer devices and do not constitute limitations on the computer devices. The computer devices may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, they may also include a network interface, a display screen, and an input system.

[0068] In one embodiment, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor in a computer device, the computer device can perform the steps of any embodiment of a video analysis alarm disclosed in the present invention, which will not be repeated here. The computer-readable storage medium can be non-volatile or volatile.

[0069] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0070] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0071] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0072] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0073] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A video analysis alarm method, characterized in that: include: Receiving a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name; According to the scene name, extracting the target scene determination rule and the target large model prompt word corresponding to the scene name; According to the target scene determination rule and the target large model prompt word, the first alarm result is rechecked using a preset multimodal large model to obtain a second alarm result.

2. The video analysis alarm method according to claim 1, characterized in that: The step of extracting a target scene determination rule and a target large model prompt word corresponding to the scene name according to the scene name includes: Matching the scene name with the scene name configured by the scene item in the preset video scene library; If the match is successful, then extracting the target scene determination rule and the target large model prompt word corresponding to the scene name from the preset video scene library; If the match fails, the scene name and the preset prompt word are input into the preset large language model. Obtaining a reference scene determination rule and a reference large model prompt word, and using the reference scene determination rule and the reference large model prompt word as a target scene determination rule and a target large model prompt word; The reference scene determination rule and the reference large model prompt word are bound to the scene name to obtain a new scene item, and the preset video scene library is updated using the new scene item.

3. The video analysis alarm method according to claim 1, characterized in that: The first alarm result also includes a first alarm screenshot. The second alarm result is obtained by performing a secondary review on the first alarm result according to the target scene determination rule and the target large model prompt word using a preset multimodal large model, including: Inputting the target large model prompt word, the target scene determination rule and the first alarm result into a preset multimodal large model, so that the preset multimodal large model performs image analysis to obtain a first analysis result, wherein the first analysis result is represented by a matching result of the first alarm screenshot and the scene name; If the first alarm screenshot matches the scene name, determining that the first parsing result is true and determining that it is a real alarm; If the first alarm screenshot does not match the scene name, the first parsing result is determined to be false and is judged to be a false alarm.

4. The method according to claim 3, characterized in that The first alarm result also includes a first alarm screenshot and a second alarm screenshot, wherein the second alarm screenshot is an enhanced alarm screenshot of the first alarm screenshot, and the second alarm result is obtained by performing a secondary review on the first alarm result using a preset multimodal large model according to the target scene determination rule and the target large model prompt word, and further includes: Inputting the target large model prompt word, the target scene determination rule and the first alarm result into a preset multimodal large model, so that the preset multimodal large model performs image analysis to obtain a second analysis result, wherein the second analysis result is represented by a matching result of the first alarm screenshot and the scene name, and a matching result of the second alarm screenshot and the scene name; If the first alarm screenshot matches the scene name, and the second alarm screenshot matches the scene name, determining that the second analysis result is true and determining that it is a real alarm; If the first alarm screenshot does not match the scene name, and the second alarm screenshot does not match the scene name, determining that the second analysis result is false and determining it to be a false alarm; If the first alarm screenshot matches the scene name, and the second alarm screenshot does not match the scene name, determining that the second analysis result is false and suspicious, and determining it as a false and suspicious alarm; If the first alarm screenshot does not match the scene name, and the second alarm screenshot matches the scene name, the second analysis result is determined to be true or doubtful, and is determined to be a true or doubtful alarm.

5. The video analysis alarm method according to claim 4, characterized in that: After obtaining the second alarm result, the method includes: If it is a real alarm, a first alarm work order is generated, wherein the first alarm work order is used to instruct the user to handle the alarm event of the current scene; If it is a false alarm, the alarm event will be recorded and archived; If it is a false doubtful alarm or a true doubtful alarm, a second alarm work order is generated, wherein the second alarm work order is used to instruct the user to re-check the second alarm result.

6. The video analysis alarm method according to claim 3 or 4, characterized in that: After obtaining the second alarm result, the method further includes: Performing statistical analysis on the first analysis result or the second analysis result to determine whether there are a large number of false alarm events in the target scene corresponding to the first alarm result; If there are a large number of false alarm events in the target scene corresponding to the first alarm result, the target scene judgment rule and the historical alarm screenshot corresponding to the first alarm result are input into the preset multimodal large model, so that the preset multimodal large model adjusts the threshold and generates a judgment rule adjustment suggestion; The judgment rule adjustment suggestion is sent to the user, so that after the user confirms, the target scene judgment rule in the preset video scene library is updated.

7. A video analysis alarm device, characterized in that: include: A receiving module, configured to receive a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name; An extraction module, used for extracting target scene determination rules and target large model prompt words corresponding to the scene name according to the scene name; The review module is used to perform a secondary review on the first alarm result according to the target scene determination rule and the target large model prompt word using a preset multimodal large model to obtain a second alarm result.

8. A video analysis alarm system, characterized in that: The video analysis alarm system includes a camera, a video analysis device and a cloud platform; The camera is used to collect real-time video within the target monitoring area; The video analysis device is used to perform video analysis on the real-time video, obtain a first alarm result, and report it to the cloud platform; The cloud platform is used to receive a first alarm result sent by a video analysis device after analyzing a real-time video, wherein the first alarm result includes a scene name; according to the scene name, a target scene determination rule and a target large model prompt word corresponding to the scene name are extracted; according to the target scene determination rule and the target large model prompt word, a preset multimodal large model is used to perform a second review on the first alarm result to obtain a second alarm result.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the video analysis alarm method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the video analysis alarm method according to any one of claims 1 to 6 is implemented.

Images