Alarm information processing method and device, electronic device and storage medium

Abstract

The application relates to an alarm information processing method and device, an electronic device and a storage medium, wherein the alarm information processing method comprises the following steps: determining target subgraph data and compressed whole graph data according to primary alarm information corresponding to a to-be-analyzed alarm image; inputting the target subgraph data and the compressed whole graph data into a lightweight image description model to output metadata information and further determine research and judgment clue metadata; determining event dynamic prompt words according to event type information and the research and judgment clue metadata; inputting the target subgraph data and the event dynamic prompt words into a target multi-modal large model to output a primary research and judgment result; and determining target hierarchical information and target suggestion information according to the primary research and judgment result and the compressed whole graph data. Through the application, the problem of how to construct a multi-level research and judgment reasoning framework integrating structured information processing capability and deep semantic understanding capability is solved, the alarm false alarm rate is reduced, the large model efficiency and the alarm accuracy in a complex scene are improved, and resources are saved.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to alarm information processing methods, apparatus, electronic devices, and storage media. Background Technology

[0002] In the field of security monitoring, computer vision-based behavior recognition algorithms are widely used for real-time detection of various abnormal events. These algorithms perform target detection and behavior classification on video streams, outputting basic alarm information including event type, target location, and timestamp. However, such algorithms face the challenge of high false alarm rates in complex scenarios, requiring further optimization.

[0003] While technologies utilizing multimodal large models for secondary verification exist, these technologies suffer from several shortcomings: First, requiring large models to handle the entire process from target recognition and scene understanding to behavior judgment simultaneously leads to excessive cognitive load, low efficiency, slow inference speed, and high resource consumption. Second, the information input is too raw and lacks effective guidance. For example, directly inputting high-resolution original images and fuzzy natural language questions into the large model can cause confusion when multiple similar targets exist in the scene, making it unsure which specific target to focus on, resulting in ambiguous referencing. Third, the "one-question-one-result" model lacks a progressive analysis mechanism for complex scenarios requiring multi-angle and multi-evidence comprehensive judgment, easily leading to misjudgments and a lack of incremental analysis capabilities, making it difficult to handle complex situations. Finally, the judgment results are singular and lack depth in decision support, mainly focusing on a binary judgment of "whether it is a false alarm." For confirmed events, it cannot further assess the risk level or provide handling suggestions, failing to meet the refined management needs of modern security systems. It is evident that the relevant technologies directly couple low-level perception results with high-level reasoning tasks, but lack a mechanism to structurally inject domain prior knowledge and intermediate semantic information into the reasoning process. Large models are forced to re-mine this information from pixel-level raw data in each reasoning, resulting in opaque and inefficient reasoning paths, and unreliability in complex situations.

[0004] Currently, no effective solution has been proposed for the problem of how to construct a multi-level judgment and reasoning framework that integrates structured information processing capabilities and deep semantic understanding capabilities in related technologies to reduce the false alarm rate of security incident alarms. Summary of the Invention

[0005] This application provides an alarm information processing method, apparatus, electronic device, and storage medium to at least address the problem in related technologies of how to construct a multi-level judgment and reasoning framework that integrates structured information processing capabilities and deep semantic understanding capabilities.

[0006] Firstly, embodiments of this application provide an alarm information processing method.

[0007] In some embodiments, the alarm information processing method includes:

[0008] Obtain the primary alarm information corresponding to the alarm image to be analyzed, and determine the target sub-image data and compressed full image data corresponding to the alarm image to be analyzed based on the primary alarm information;

[0009] The target sub-image data and the compressed full image data are input into a lightweight image description model, so that the lightweight image description model outputs metadata information based on the target sub-image data and the compressed full image data, and determines the analysis clue metadata based on the metadata information and the primary alarm information;

[0010] Based on the primary alarm information, the event type information is determined; based on the event type information and the analysis clue metadata, the event dynamic prompt words are determined.

[0011] The target subgraph data and the event dynamic prompt words are input into the target multimodal large model, so that the target multimodal large model outputs a preliminary judgment result based on the target subgraph data and the event dynamic prompt words;

[0012] Based on the preliminary assessment results and the compressed whole map data, the target classification information and target suggestion information output by the target multimodal large model are determined.

[0013] In some embodiments, the step of inputting the target sub-image data and the compressed full-image data into a lightweight image description model, so that the lightweight image description model outputs metadata information based on the target sub-image data and the compressed full-image data, and determining the analysis clue metadata based on the metadata information and the primary alarm information includes:

[0014] The target sub-image data is input into the lightweight image description model to determine text description information, and the compressed full image data is input into the lightweight image description model to determine scene label information. The metadata information includes the text description information and the scene label information.

[0015] Based on the text description information, the scene tag information, and the primary alarm information, determine the metadata of the analysis clues.

[0016] In some embodiments, determining event type information based on the primary alarm information, and determining event dynamic prompt words based on the event type information and the analysis clue metadata, includes:

[0017] Based on the primary alarm information, determine the event type information, and based on the event type information, determine the basic prompt word skeleton corresponding to the alarm image to be analyzed;

[0018] The metadata of the analytical clues is filled into the variable slots of the basic prompt word skeleton to determine the event dynamic prompt words.

[0019] In some embodiments, determining the target classification information and target suggestion information output by the target multimodal large model based on the initial assessment results and the compressed whole image data includes:

[0020] Based on the preliminary analysis results, the false alarm confidence level is determined. If, based on the false alarm confidence level, the alarm image to be analyzed is determined to be a non-high confidence false alarm, the information classification and handling prompt word template corresponding to the alarm image to be analyzed is determined based on the event type.

[0021] The compressed whole image data and the information classification and processing prompt word template are input into the target multimodal large model to determine the target classification information and target suggestion information output by the target multimodal large model.

[0022] In some embodiments, obtaining the primary alarm information corresponding to the alarm image to be analyzed, and determining the target sub-image data and compressed full-image data corresponding to the alarm image to be analyzed based on the primary alarm information, includes:

[0023] Obtain the primary alarm information corresponding to the alarm image to be analyzed, and draw the target rectangle in the alarm image to be analyzed based on the target bounding box coordinate information included in the primary alarm information, so as to determine the alarm annotation original image including the target rectangle;

[0024] Based on the area corresponding to the target rectangle, the original alarm annotation image is cropped to determine the target sub-image data, and the original alarm annotation image is compressed to determine the compressed full image data.

[0025] In some embodiments, cropping the original alarm annotation image based on the region corresponding to the target rectangle to determine target sub-image data, and compressing the original alarm annotation image to determine compressed full image data includes:

[0026] Based on the event type information, determine the alarm event type corresponding to the alarm image to be analyzed;

[0027] When the alarm event type is a static event type, the single frame of the alarm image to be analyzed and its corresponding single frame of the original alarm annotation image are determined. The single frame of the original alarm annotation image is cropped to determine the single frame target sub-image, and the single frame target sub-image is determined as the target sub-image data. The single frame of the original alarm annotation image is compressed to determine the single frame compressed full image, and the single frame compressed full image is determined as the compressed full image data.

[0028] When the alarm event type is a dynamic event type, multiple frames of the alarm image to be analyzed and their corresponding multiple frames of the original alarm annotation image are determined. The multiple frames of the original alarm annotation image are cropped to determine a target sub-image sequence, and the target sub-image sequence is determined as the target sub-image data. The multiple frames of the original alarm annotation image are compressed to determine a compressed full image sequence, and the compressed full image sequence is determined as the compressed full image data.

[0029] In some embodiments, when the alarm event type is a dynamic event type, determining multiple frames of the alarm images to be analyzed and their corresponding multiple frames of the original alarm annotation images includes:

[0030] When the alarm event type is a dynamic event type, the timestamp information and camera identification information contained in the primary alarm information are obtained, and multiple frames of original video frame images are obtained from the camera corresponding to the camera identification information based on the timestamp information.

[0031] Based on the multiple original video frame images and the preset key frame selection algorithm, the multiple alarm images to be analyzed and their corresponding multiple alarm annotation original images are determined.

[0032] Secondly, embodiments of this application provide an alarm information processing device.

[0033] In some embodiments, the alarm information processing device includes an initial data determination module, an analysis clue generation module, a dynamic prompt word determination module, a preliminary result output module, and an advanced result output module:

[0034] The initial data determination module is used to obtain the primary alarm information corresponding to the alarm image to be analyzed, and determine the target sub-image data and compressed full image data corresponding to the alarm image to be analyzed based on the primary alarm information.

[0035] The analysis clue generation module is used to input the target sub-image data and the compressed full image data into a lightweight image description model, so that the lightweight image description model outputs metadata information based on the target sub-image data and the compressed full image data, and determines the analysis clue metadata based on the metadata information and the primary alarm information;

[0036] The dynamic prompt word determination module is used to determine event type information based on the primary alarm information, and to determine dynamic prompt words for the event based on the event type information and the analysis clue metadata.

[0037] The initial result output module is used to input the target subgraph data and the event dynamic prompt words into the target multimodal large model, so that the target multimodal large model can output an initial judgment result based on the target subgraph data and the event dynamic prompt words;

[0038] The advanced result output module is used to determine the target classification information and target suggestion information output by the target multimodal large model based on the initial judgment results and the compressed whole map data.

[0039] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the alarm information processing method as described in the first aspect above.

[0040] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the alarm information processing method as described in the first aspect above.

[0041] Compared to related technologies, the alarm information processing method, apparatus, electronic device, and storage medium provided in this application obtain primary alarm information corresponding to the alarm image to be analyzed; determine the target sub-image data and compressed full-image data corresponding to the alarm image to be analyzed based on the primary alarm information; input the target sub-image data and compressed full-image data into a lightweight image description model, so that the lightweight image description model outputs metadata information based on the target sub-image data and compressed full-image data; determine the analysis clue metadata based on the metadata information and the primary alarm information; determine the event type information based on the primary alarm information; and determine the event dynamics based on the event type information and the analysis clue metadata. Dynamic prompts are input into the target subgraph data and event dynamic prompts, enabling the target multimodal large model to output preliminary judgment results based on the target subgraph data and event dynamic prompts. Based on the preliminary judgment results and compressed whole graph data, the target classification information and target suggestion information output by the target multimodal large model are determined. This solves the problem in related technologies of how to construct a multi-level judgment and reasoning framework that integrates structured information processing capabilities and deep semantic understanding capabilities. It can reduce the false alarm rate of security event alarms, improve the reasoning speed and efficiency of large models, reduce resource consumption, improve alarm accuracy in complex scenarios, and meet the refined management needs of modern security systems.

[0042] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0044] Figure 1 This is a hardware structure block diagram of a terminal for an alarm information processing method according to an embodiment of this application;

[0045] Figure 2 This is a flowchart of an alarm information processing method according to an embodiment of this application;

[0046] Figure 3 This is a flowchart of an alarm information processing method according to a preferred embodiment of this application;

[0047] Figure 4 This is a structural block diagram of an alarm information processing device according to an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0049] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0050] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0051] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. Taking running on a terminal as an example, Figure 1 This is a hardware structure block diagram of the terminal for the alarm information processing method according to an embodiment of the present invention. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0052] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the alarm information processing method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0053] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0054] This embodiment provides a method for processing alarm information. Figure 2 This is a flowchart of an alarm information processing method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0055] Step S201: Obtain the primary alarm information corresponding to the alarm image to be analyzed, and determine the target sub-image data and compressed full image data corresponding to the alarm image to be analyzed based on the primary alarm information.

[0056] In this embodiment, the system receives a primary alarm event packet from a front-end algorithm, which can be a small model algorithm. The primary alarm event packet includes primary alarm information corresponding to the alarm image to be analyzed. The primary alarm information is structured data and may include: event_id (event identifier information), image_url (alarm screenshot URL information), bbox (target bounding box coordinate information, formatted as [x1, y1, x2, y2]), event_type (event type information, specifically "illegal_parking"), timestamp (timestamp information), and camera_id (camera identifier information). Based on the primary alarm information, by downloading the original image data of the alarm image to be analyzed, and through processing steps such as annotation, cropping, and compression, target sub-image data and compressed full image data corresponding to the alarm image to be analyzed can be generated. The specific processing method will be described later.

[0057] Step S202: Input the target sub-image data and compressed full-image data into the lightweight image description model so that the lightweight image description model can output metadata information based on the target sub-image data and compressed full-image data, and determine the analysis clue metadata based on the metadata information and the primary alarm information.

[0058] Furthermore, the target sub-image data and compressed full-image data are input into a lightweight image description model. A lightweight image description model is a small model; a pool of lightweight visual models is maintained, each optimized for a specific target type. Metadata information is output from the lightweight image description model and combined with initial alarm information to automatically generate structured analytical clue metadata, serving as prior knowledge to guide subsequent reasoning in the larger model. As an example, the JSON format of the analytical clue metadata is shown below:

[0059]

[0060] Step S203: Determine the event type information based on the primary alarm information, and determine the event dynamic prompt words based on the event type information and the analysis clue metadata.

[0061] In this embodiment, the system no longer uses a fixed prompt word template, but instead employs a prompt word assembly engine. Based on this engine, a basic template skeleton is loaded, allowing a basic prompt word skeleton to be loaded according to event type information. Furthermore, metadata from the analysis clues is integrated into the basic prompt word skeleton, thereby generating highly customized, information-rich, dynamic event prompt words.

[0062] Step S204: Input the target subgraph data and event dynamic prompts into the target multimodal large model so that the target multimodal large model can output preliminary judgment results based on the target subgraph data and event dynamic prompts.

[0063] Furthermore, the target subgraph data and dynamic event cues are used together as input to the target multimodal large-scale model (such as GPT-4V). Based on precise guidance, the target multimodal large-scale model performs inference and outputs a preliminary assessment result that meets the requirements; this preliminary assessment result can be a JSON result. Thus, at this stage, the large-scale model, based on precise guidance from the target subgraph data and dynamic event cues, focuses on answering a clear and specific question to output a preliminary assessment result, avoiding information overload.

[0064] Step S205: Based on the preliminary assessment results and the compressed whole map data, determine the target classification information and target suggestion information output by the target multimodal large model.

[0065] The initial assessment results are analyzed to determine the circumstances under which the target multimodal large model is invoked again. In this case, the compressed full map data is used as the input to the target multimodal large model to obtain the output target classification information and target suggestion information.

[0066] Through the above steps, this embodiment of the application obtains the primary alarm information corresponding to the alarm image to be analyzed, determines the target sub-image data and compressed full-image data corresponding to the alarm image to be analyzed based on the primary alarm information, inputs the above data into a lightweight image description model to determine the output metadata information, and determines the judgment clue metadata based on the metadata information and the primary alarm information. Thus, before inputting visual and textual information into the large model, it automatically generates and integrates structured "judgment clue metadata" (such as target description, attributes, scene tags). Furthermore, based on the event type information and the judgment clue metadata, it determines precise event dynamic prompt words to guide the large model in efficient reasoning. Specifically, it involves... Target sub-image data and dynamic event prompts are input into the target multimodal large model to determine the initial assessment results. Based on the initial assessment results and the compressed full image data, the target classification information and target suggestion information output by the target multimodal large model are determined. In this way, alarm-related images are analyzed through a dedicated lightweight image description model to generate structured assessment clue metadata. The large model is transformed from a passive statistical pattern matcher into an active constrained inference engine, improving assessment consistency, reducing the false judgment rate, and enhancing the interpretability of the results. The multimodal large model prompts are dynamically assembled in combination with the clue metadata. Based on this, a progressive security alarm event assessment process from authenticity to classification / suggestion is realized.

[0067] In some embodiments, step S202 includes:

[0068] Step S2021: Input the target sub-image data into the lightweight image description model to determine the text description information, and input the compressed full image data into the lightweight image description model to determine the scene label information. The metadata information includes text description information and scene label information.

[0069] A lightweight image description model is a small model. This application maintains a lightweight visual model pool containing multiple lightweight image description models, each model being analyzed and optimized for a specific target type. For example:

[0070] 1. vehicle_attr_model: Used to identify the vehicle's color, model (SUV / sedan / truck), and direction of travel.

[0071] 2. person_attr_model: Used to identify a person's clothing color, whether they are carrying luggage, and their general posture.

[0072] 3. general_object_model: Used to identify common objects (such as boxes, packages, tools).

[0073] Optionally, the target sub-image data is input into the lightweight image description model. The system infers the main target type (such as vehicle, person, or object) in the target sub-image based on event type information or uses a general target detector to determine it. Then, the corresponding attribute recognition model is called to convert the structured attributes output by the model (such as {color: "silver", type: "SUV", orientation: "eastward") into a natural language description using a preset text template. Inputting the target sub-image data into the lightweight image description model generates a text description. For example, for an illegal parking event, the following text description might be generated: "A silver SUV, facing east, is parked next to the sidewalk curb." In some optional embodiments, a preset attribute recognition rule base can also be called to generate the text description. Further, the compressed full-image data is input into the lightweight image description model to perform scene classification to determine scene label information. The scene label information includes one or more scene labels, such as: ["city road", "sidewalk", "night"]. Thus, metadata information including text description information and scene label information is obtained.

[0074] Step S2022: Determine the metadata of the clues for analysis based on the text description information, scene tag information, and primary alarm information.

[0075] Metadata information is combined with primary alarm information to assemble structured metadata for analyzing clues. As an example, the JSON format of the metadata for analyzing clues is shown below:

[0076]

[0077] Through the above steps, this embodiment of the application generates "analysis clue metadata," systematically transforming the target perception results into semantically rich structured descriptions—textual description information—before calling the large model for complex reasoning, and incorporating key scene knowledge—scene tag information. This pre-constructs a high-quality "working memory" or "case summary" for the multimodal large model, enabling it to skip time-consuming low-level feature extraction and semantic association steps and directly enter the deep logical analysis stage, significantly improving the accuracy and speed of analysis, and making the entire reasoning process more interpretable and predictable.

[0078] In some embodiments, step S203 includes:

[0079] Step S2031: Determine the event type information based on the primary alarm information, and determine the basic prompt word skeleton corresponding to the alarm image to be analyzed based on the event type information.

[0080] In this embodiment, event type information is determined based on the primary alarm information. A basic prompt word skeleton corresponding to the alarm image to be analyzed is loaded based on the event type information. This skeleton includes reserved variable slots, such as: "You are a security analyst. Please analyze the image based on the following clues: {CLUES}. Focus on targets within the {ENHANCED_BBOX} area and determine whether they constitute '{EVENT_TYPE_CHINESE}'. Your output must be in strict JSON format: {'is_correct': true / false, 'confidence': a floating-point number between 0 and 1, 'reason': 'your analysis reason'}."

[0081] Step S2032: Populate the metadata of the analyzed clues into the variable slots of the basic prompt word skeleton to determine the event dynamic prompt words.

[0082] In this embodiment, the system no longer uses a fixed prompt word template, but instead employs a prompt word assembly engine. Key fields from the analysis clue metadata generated by the engine are naturally and coherently populated into the variable slots of the basic prompt word skeleton. For example, `target_description` and `scene_context` from the JSON format example of the analysis clue metadata are integrated and filled into `{CLUES}`. This ultimately generates highly customized, information-rich dynamic event prompt words, such as: "You are a security analyst. Please analyze the image based on the following clue: The scene is a sidewalk on a nighttime city road. The system detected a silver SUV parked on the curb. Please focus on the target within the red rectangle area and determine whether it constitutes 'illegal parking'. Your output must be in strict JSON format: `{'is_correct': true / false, 'confidence': a floating-point number between 0 and 1, 'reason': 'your analysis reason'}`."

[0083] Through the above steps, the key fields in the generated analysis clue metadata are naturally and coherently filled into the variable slots of the basic prompt word skeleton, ultimately generating highly customized and information-rich event dynamic prompt words, which accurately guide the large model to perform efficient reasoning, and is highly feasible.

[0084] In some embodiments, step S205 includes:

[0085] Step S2051: Determine the false alarm confidence result based on the initial analysis result. If the false alarm confidence result indicates that the alarm image to be analyzed is a non-high confidence false alarm, determine the information classification and handling prompt word template corresponding to the alarm image to be analyzed based on the event type.

[0086] In this embodiment, the initial judgment result returned by the large model is parsed to determine the false alarm confidence result. Taking the JSON result as an example, if is_correct is false and confidence is higher than a preset threshold, it is determined to be a high-confidence false alarm based on the false alarm confidence result, the process terminates, and the final conclusion is output. If is_correct is true, or is_correct is false but confidence is lower than the preset threshold, the alarm image to be analyzed is determined to be a non-high-confidence false alarm based on the false alarm confidence result, the process continues, and at this time, the information classification and handling prompt word template corresponding to the alarm image to be analyzed is determined according to the event type. Preferably, the preset threshold can be 0.7.

[0087] Step S2052: Input the compressed whole map data and information classification processing prompt word template into the target multimodal large model to determine the target classification information and target suggestion information output by the target multimodal large model.

[0088] Furthermore, by using visual context to compress the entire image data more completely, and based on the target multimodal large model, loading information classification and handling prompt word templates for the corresponding event type, this template requires the large model to comprehensively assess factors such as event severity and scenario risk. The target multimodal large model is then invoked, outputting target classification information and target suggestion information. Examples of target classification information and target suggestion information are as follows:

[0089]

[0090] Through the above steps, this application embodiment allows a lightweight, specialized model or rule engine to handle the initial information extraction and clue structuring (such as target attribute description and scene classification), while the heavyweight, general-purpose large model, after identifying high-quality clues based on false positive confidence results, focuses on its strengths in complex logical reasoning, contextual understanding, and final decision-making. This architecture leverages the advantages of different model types, minimizing the system's dependence on expensive large model computing power and overall response latency while ensuring judgment accuracy. In some embodiments, step S201 includes:

[0091] Step S2011: Obtain the primary alarm information corresponding to the alarm image to be analyzed. Based on the target bounding box coordinate information included in the primary alarm information, draw the target rectangle in the alarm image to be analyzed to determine the original alarm annotation image including the target rectangle.

[0092] In this embodiment, the original image of the alarm to be analyzed, corresponding to the URL information of the alarm screenshot, is downloaded. Based on the target bounding box coordinate information, the target bounding box coordinate region is expanded using a preset algorithm (e.g., adaptively calculated based on the ratio of the target bounding box area to the original image area), resulting in expanded coordinates. Based on the expanded coordinates, a target rectangle corresponding to the expanded coordinates is drawn in the alarm image to be analyzed, highlighting it with a bright color, thus obtaining the original alarm annotation image including the target rectangle.

[0093] Step S2012: Based on the area corresponding to the target rectangle, crop the original alarm annotation image to determine the target sub-image data, and compress the original alarm annotation image to determine the compressed full image data.

[0094] Based on the area corresponding to the target rectangle, the target sub-image corresponding to the target rectangle area is precisely cropped from the original alarm annotation image, and the target sub-image data including the target sub-image is determined. The target sub-image removes a large amount of irrelevant background and focuses on the target. The resolution of the original alarm annotation image is compressed (e.g., scaled to 1024 pixels on the long side) to generate a compressed full image, and the compressed full image data including the compressed full image is determined.

[0095] Through the above steps, this embodiment dynamically calculates the clipping region based on the proportion of the target bounding box in the original image, generates the target rectangle and alarm annotation original image by magnification, and then generates target sub-image data, so that the large model focuses on the core target, effectively solving the problem of misjudgment caused by the target being too small or the attention being scattered; and, the alarm annotation original image is compressed in resolution to generate compressed whole image data with more complete visual context, and the image information is more accurate and complete. While ensuring the context information in the hierarchical stage, the number of visual tokens is greatly reduced, and information purification and noise reduction are completed at the input end.

[0096] In some embodiments, step S2012 includes:

[0097] Step S2112: Determine the alarm event type corresponding to the alarm image to be analyzed based on the event type information.

[0098] This application embodiment distinguishes alarm event types and determines the alarm event type corresponding to the alarm image to be analyzed based on the event type information. Alarm event types may include static event types, such as "item left behind", and dynamic event types, such as "vehicle driving in the wrong direction".

[0099] Step S2212: When the alarm event type is a static event type, determine the single-frame alarm image to be analyzed and its corresponding single-frame alarm annotation original image, crop the single-frame alarm annotation original image to determine the single-frame target sub-image, and determine the single-frame target sub-image as target sub-image data; compress the single-frame alarm annotation original image to determine the single-frame compressed whole image, and determine the single-frame compressed whole image as compressed whole image data.

[0100] When the alarm event type is a static event type, obtain the single-frame alarm image to be analyzed and its corresponding single-frame alarm annotation original image, crop the single-frame alarm annotation original image to determine the single-frame target sub-image, and determine the single-frame target sub-image as the target sub-image data; compress the single-frame alarm annotation original image to determine the single-frame compressed whole image, and determine the single-frame compressed whole image as the compressed whole image data.

[0101] Step S2312: When the alarm event type is a dynamic event type, determine multiple frames of alarm images to be analyzed and their corresponding multiple frames of alarm annotation original images, crop the multiple frames of alarm annotation original images to determine the target sub-image sequence, and determine the target sub-image sequence as the target sub-image data; compress the multiple frames of alarm annotation original images to determine the compressed whole image sequence, and determine the compressed whole image sequence as the compressed whole image data.

[0102] When the alarm event type is a dynamic event type, a video recording is retrieved based on the video source corresponding to the camera identification information. Multiple keyframes are extracted from the video recording as multiple alarm images to be analyzed, and their corresponding original alarm annotation images are further determined. The original alarm annotation images are then cropped to determine a target sub-image sequence, which is then used as the target sub-image data. Finally, the original alarm annotation images are compressed to determine a compressed full image sequence, which is then used as the compressed full image data.

[0103] Through the above steps, this embodiment intelligently distinguishes between static and dynamic events based on event type and triggers different preprocessing procedures. Static events are analyzed based on single-frame image analysis, while dynamic events are analyzed by pulling video streams and extracting multi-frame time-series images. By providing different optimized input strategies for different event types, the system significantly improves its utilization efficiency of large model resources and overall throughput while ensuring the effectiveness of the analysis.

[0104] In some embodiments, step S2312 includes:

[0105] Step S2412: When the alarm event type is a dynamic event type, obtain the timestamp information and camera identification information contained in the primary alarm information, and obtain multiple frames of original video frame images from the camera corresponding to the camera identification information based on the timestamp information.

[0106] Specifically, when the alarm event type is a dynamic event type, the time offset before and after is calculated based on the event type information and timestamp information, and a video recording is pulled from the video source corresponding to the camera identification information. The video recording includes multiple original video frame images.

[0107] Step S2512: Based on the multiple original video frame images and the preset key frame selection algorithm, determine the multiple alarm images to be analyzed and their corresponding multiple alarm annotation original images.

[0108] Based on a preset key frame selection algorithm, such as selecting key change frames at equal time intervals or based on optical flow, multiple alarm images to be analyzed are extracted from multiple original video frame images, and their corresponding original alarm annotation images are determined.

[0109] Through the above steps, this application provides a method for determining the original image of multi-frame alarm annotations for specific dynamic event types, namely, a judgment mechanism based on time-series multi-frames, which is highly feasible and improves the utilization efficiency of the entire system for large model resources.

[0110] Furthermore, the embodiments of this application can further generate a final assessment report. Continuing with the above example, at this stage, the large model no longer needs to identify vehicle color or determine whether it is parked, as these are already "known facts." Its task is to conduct a higher-level contextualized risk assessment. At this point, the system integrates the initial assessment results (authenticity, confidence level, and rationale) and the secondary assessment results (target-level information and target suggestion information, risk level, and handling suggestions) to generate a complete final assessment report, which is then pushed to the security management platform.

[0111] The embodiments of this application will be described and illustrated below through preferred embodiments.

[0112] Figure 3 This is a flowchart of an alarm information processing method according to a preferred embodiment of this application. Figure 3 As shown, the alarm information processing method includes the following steps:

[0113] Step S301: Obtain the primary alarm information corresponding to the alarm image to be analyzed, and determine the target sub-image data and compressed full image data corresponding to the alarm image to be analyzed based on the primary alarm information;

[0114] Step S302: Input the target sub-image data into the lightweight image description model to determine text description information, and input the compressed full image data into the lightweight image description model to determine scene label information. The metadata information includes the text description information and the scene label information.

[0115] Step S303: Determine the metadata of the analysis clues based on the text description information, the scene tag information, and the primary alarm information;

[0116] Step S304: Determine the event type information based on the primary alarm information, and determine the basic prompt word skeleton corresponding to the alarm image to be analyzed based on the event type information;

[0117] Step S305: Fill the analysis clue metadata into the variable slots of the basic prompt word skeleton to determine the event dynamic prompt word;

[0118] Step S306: Input the target subgraph data and the event dynamic prompt words into the target multimodal large model, so that the target multimodal large model can output the initial judgment result based on the target subgraph data and the event dynamic prompt words;

[0119] Step S307: Based on the preliminary judgment results and the compressed whole map data, determine the target classification information and target suggestion information output by the target multimodal large model.

[0120] This application's embodiments do not overload the large model with complex problems all at once. Instead, the system generates key clues through preprocessing, breaking down the analysis process into multiple logically progressive stages. At each stage, the large model is provided with appropriate "hints," guiding it to complete the analysis task efficiently and accurately. This two-stage design allows the system to handle two different types of tasks: "fact confirmation" and "risk assessment." Especially in the second stage, the large model can make more realistic, tiered judgments based on known facts, combined with comprehensive factors such as time, location, and scenario, rather than simply a "yes / no" answer. This significantly improves the accuracy, efficiency, and decision-making value of secondary analysis of security alarms.

[0121] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0122] This embodiment also provides an alarm information processing device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0123] Figure 4 This is a structural block diagram of an alarm information processing device according to an embodiment of this application, such as... Figure 4 As shown, the device includes an initial data determination module 10, a clue generation module 20, a dynamic prompt word determination module 30, a preliminary result output module 40, and an advanced result output module 50.

[0124] The initial data determination module 10 is used to obtain the primary alarm information corresponding to the alarm image to be analyzed, and to determine the target sub-image data and compressed full image data corresponding to the alarm image to be analyzed based on the primary alarm information.

[0125] The analysis clue generation module 20 is used to input the target sub-image data and compressed full image data into the lightweight image description model, so that the lightweight image description model can output metadata information based on the target sub-image data and compressed full image data, and determine the analysis clue metadata based on the metadata information and the primary alarm information.

[0126] The dynamic prompt word determination module 30 is used to determine the event type information based on the primary alarm information, and to determine the event dynamic prompt words based on the event type information and the analysis clue metadata.

[0127] The initial result output module 40 is used to input the target subgraph data and event dynamic prompt words into the target multimodal large model, so that the target multimodal large model can output the initial judgment results based on the target subgraph data and event dynamic prompt words;

[0128] The advanced result output module 50 is used to determine the target classification information and target suggestion information output by the target multimodal large model based on the initial analysis results and compressed whole map data.

[0129] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0130] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0131] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0132] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0133] Obtain the primary alarm information corresponding to the alarm image to be analyzed, and determine the target sub-image data and compressed full image data corresponding to the alarm image to be analyzed based on the primary alarm information;

[0134] The target sub-image data and compressed full-image data are input into the lightweight image description model so that the lightweight image description model can output metadata information based on the target sub-image data and compressed full-image data, and determine the analysis clue metadata based on the metadata information and the primary alarm information.

[0135] Based on the initial alarm information, determine the event type information; based on the event type information and the metadata of the analyzed clues, determine the event dynamic prompt words.

[0136] Input the target subgraph data and event dynamic prompts into the target multimodal large model so that the target multimodal large model can output preliminary judgment results based on the target subgraph data and event dynamic prompts;

[0137] Based on the preliminary assessment results and the compressed whole map data, the target classification information and target suggestion information output by the target multimodal large model are determined.

[0138] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0139] Furthermore, in conjunction with the alarm information processing methods in the above embodiments, this application embodiment can provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the alarm information processing methods in the above embodiments.

[0140] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0142] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for processing alarm information, characterized in that, Includes the following steps: Obtain the primary alarm information corresponding to the alarm image to be analyzed, and determine the target sub-image data and compressed full image data corresponding to the alarm image to be analyzed based on the primary alarm information; The target sub-image data and the compressed full image data are input into a lightweight image description model, so that the lightweight image description model outputs metadata information based on the target sub-image data and the compressed full image data, and determines the analysis clue metadata based on the metadata information and the primary alarm information; Based on the primary alarm information, the event type information is determined; based on the event type information and the analysis clue metadata, the event dynamic prompt words are determined. The target subgraph data and the event dynamic prompt words are input into the target multimodal large model, so that the target multimodal large model outputs a preliminary judgment result based on the target subgraph data and the event dynamic prompt words; Based on the preliminary assessment results and the compressed whole map data, the target classification information and target suggestion information output by the target multimodal large model are determined.

2. The alarm information processing method according to claim 1, characterized in that, The step of inputting the target sub-image data and the compressed full-image data into a lightweight image description model, so that the lightweight image description model outputs metadata information based on the target sub-image data and the compressed full-image data, and determines the analysis clue metadata based on the metadata information and the primary alarm information includes: The target sub-image data is input into the lightweight image description model to determine text description information, and the compressed full image data is input into the lightweight image description model to determine scene label information. The metadata information includes the text description information and the scene label information. Based on the text description information, the scene tag information, and the primary alarm information, determine the metadata of the analysis clues.

3. The alarm information processing method according to claim 2, characterized in that, The step of determining event type information based on the primary alarm information, and determining event dynamic prompt words based on the event type information and the analysis clue metadata, includes: Based on the primary alarm information, determine the event type information, and based on the event type information, determine the basic prompt word skeleton corresponding to the alarm image to be analyzed; The metadata of the analytical clues is filled into the variable slots of the basic prompt word skeleton to determine the event dynamic prompt words.

4. The alarm information processing method according to claim 3, characterized in that, The step of determining the target classification information and target suggestion information output by the target multimodal large model based on the preliminary judgment results and the compressed whole map data includes: Based on the preliminary analysis results, the false alarm confidence level is determined. If, based on the false alarm confidence level, the alarm image to be analyzed is determined to be a non-high confidence false alarm, the information classification and handling prompt word template corresponding to the alarm image to be analyzed is determined based on the event type. The compressed whole image data and the information classification and processing prompt word template are input into the target multimodal large model to determine the target classification information and target suggestion information output by the target multimodal large model.

5. The alarm information processing method according to any one of claims 1 to 4, characterized in that, The step of obtaining the primary alarm information corresponding to the alarm image to be analyzed, and determining the target sub-image data and compressed full-image data corresponding to the alarm image to be analyzed based on the primary alarm information, includes: Obtain the primary alarm information corresponding to the alarm image to be analyzed, and draw a target rectangle in the alarm image to be analyzed based on the target bounding box coordinate information included in the primary alarm information, so as to determine the original alarm annotation image including the target rectangle; Based on the area corresponding to the target rectangle, the original alarm annotation image is cropped to determine the target sub-image data, and the original alarm annotation image is compressed to determine the compressed full image data.

6. The alarm information processing method according to claim 5, characterized in that, The step of cropping the original alarm annotation image to determine target sub-image data based on the area corresponding to the target rectangle, and compressing the original alarm annotation image to determine compressed full image data includes: Based on the event type information, determine the alarm event type corresponding to the alarm image to be analyzed; When the alarm event type is a static event type, the single frame of the alarm image to be analyzed and its corresponding single frame of the original alarm annotation image are determined. The single frame of the original alarm annotation image is cropped to determine the single frame target sub-image, and the single frame target sub-image is determined as the target sub-image data. The single frame of the original alarm annotation image is compressed to determine the single frame compressed full image, and the single frame compressed full image is determined as the compressed full image data. When the alarm event type is a dynamic event type, multiple frames of the alarm image to be analyzed and their corresponding multiple frames of the original alarm annotation image are determined. The multiple frames of the original alarm annotation image are cropped to determine a target sub-image sequence, and the target sub-image sequence is determined as the target sub-image data. The multiple frames of the original alarm annotation image are compressed to determine a compressed full image sequence, and the compressed full image sequence is determined as the compressed full image data.

7. The alarm information processing method according to claim 6, characterized in that, When the alarm event type is a dynamic event type, determining the multiple frames of the alarm image to be analyzed and their corresponding multiple frames of the original alarm annotation image includes: When the alarm event type is a dynamic event type, the timestamp information and camera identification information contained in the primary alarm information are obtained, and multiple frames of original video frame images are obtained from the camera corresponding to the camera identification information based on the timestamp information. Based on the multiple original video frame images and the preset key frame selection algorithm, the multiple alarm images to be analyzed and their corresponding multiple alarm annotation original images are determined.

8. An alarm information processing device, characterized in that, It includes an initial data determination module, an analysis clue generation module, a dynamic prompt word determination module, a preliminary result output module, and an advanced result output module: The initial data determination module is used to obtain the primary alarm information corresponding to the alarm image to be analyzed, and determine the target sub-image data and compressed full image data corresponding to the alarm image to be analyzed based on the primary alarm information. The analysis clue generation module is used to input the target sub-image data and the compressed full image data into a lightweight image description model, so that the lightweight image description model outputs metadata information based on the target sub-image data and the compressed full image data, and determines the analysis clue metadata based on the metadata information and the primary alarm information; The dynamic prompt word determination module is used to determine event type information based on the primary alarm information, and to determine dynamic prompt words for the event based on the event type information and the analysis clue metadata. The initial result output module is used to input the target subgraph data and the event dynamic prompt words into the target multimodal large model, so that the target multimodal large model can output an initial judgment result based on the target subgraph data and the event dynamic prompt words; The advanced result output module is used to determine the target classification information and target suggestion information output by the target multimodal large model based on the initial judgment results and the compressed whole map data.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the alarm information processing method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the alarm information processing method according to any one of claims 1 to 7 when it is run.

Images