An event fusion analysis processing method and system

Abstract

The application relates to the technical field of event fusion processing, and provides an event fusion analysis processing method and system, which comprises the following steps: acquiring image information of a target event, constructing a corresponding target type list, detecting a target of interest from the image information of the target event, and generating image target description of the target event; generating corresponding image-text information prompt words according to the image target description of the target event and structured field information; constructing a knowledge base by gathering event types of interest, extracting vector features from the constructed knowledge base, and constructing a vector knowledge base; constructing a query vector according to the target event, searching the vector knowledge base by using the constructed query vector, generating model analysis prompt words according to a search result and the image-text information prompt words; obtaining a model analysis result according to the model analysis prompt words by using a large language model, and performing event fusion analysis processing according to the model analysis result. The application can realize fusion processing of image and text events under low computing resources.

Description

Technical Field

[0001] This invention relates to the field of event fusion processing technology, and in particular to an event fusion analysis and processing method and system. Background Technology

[0002] In the current process of collecting and processing social events, the common practice is to attach images as supplementary materials, and then manually review the relevant event descriptions and images to assign responsibility based on the reviewed descriptions and images. Although automated event processing systems have the capability to analyze event text descriptions, they are usually limited to the comprehensive utilization of keywords, summaries, and other information in the text descriptions.

[0003] How to provide a data source based on images, text, and historical events, and leverage the processing and comprehensive analysis capabilities of small and large models to make judgments on specific problems, achieve real-time comprehensive analysis of events, and provide reasonable suggestions for handling strategies has become an urgent technical problem to be solved. Summary of the Invention

[0004] In view of this, in order to overcome the shortcomings of the prior art, the present invention aims to provide an event fusion analysis and processing method and system.

[0005] According to a first aspect of the present invention, an event fusion analysis and processing method is provided, comprising:

[0006] The process involves acquiring image information of the target event, identifying the image information of the target event as scene labels, constructing a corresponding target type list based on the scene labels, detecting targets of interest from the image information of the target event based on the constructed target type list, and generating an image target description of the target event.

[0007] Obtain the structured field information of the target event, and generate corresponding image and text information prompts based on the image target description and structured field information of the target event;

[0008] A knowledge base is built by aggregating the types of events of interest, and a vector knowledge base is built by extracting vector features from the built knowledge base.

[0009] A query vector is constructed based on the target event. The constructed query vector is used to search the vector knowledge base. Based on the search results and graphic information prompts, a model is generated to analyze the prompts.

[0010] A large language model is used to obtain model analysis results based on model analysis prompts, and event fusion analysis is performed based on the model analysis results.

[0011] Preferably, in the event fusion analysis and processing method of the present invention, the process includes: acquiring image information of a target event; identifying the image information of the target event as scene tags; constructing a corresponding target type list based on the scene tags; detecting targets of interest from the constructed target type list; and generating an image target description of the target event.

[0012] An image classification model is trained using historical image data of the event and a set of event image labels. The trained image classification model is then used to identify the target event as a scene label in the set of event image labels based on the image information of the target event.

[0013] For each scene label in the image label set, construct a corresponding target object type map. Based on the constructed target object type map and the scene label corresponding to the target event, construct a target type list.

[0014] An object detection model is used to detect objects of interest from the image information of the target event based on a constructed list of target types. Based on the detected objects of interest and their number, an image object description of the target event is generated.

[0015] Preferably, in the event fusion analysis and processing method of the present invention, constructing a corresponding target object type map for each scene tag in the image tag set includes: taking the scene tag, the target object, and the correlation between the scene tag and the target object as components of the target object type map.

[0016] Preferably, in the event fusion analysis and processing method of the present invention, obtaining the structured field information of the target event and generating corresponding graphic and textual information prompts based on the image target description and structured field information of the target event includes: using the image target description and structured field information of the target event to replace the variables at the corresponding positions in the prompt template of the large language model to obtain the graphic and textual information prompts of the target event.

[0017] Preferably, in the event fusion analysis and processing method of the present invention, the structured field information of the target event includes the text description of the target event, time, location, and reporter.

[0018] Preferably, in the event fusion analysis and processing method of the present invention, a knowledge base is constructed by aggregating the event types of interest, and a vector knowledge base is constructed by extracting vector features from the constructed knowledge base, including:

[0019] Based on the needs of attention, we collect the types of events that are of concern, and gather detailed descriptions, handling strategies and reporting suggestions for each type of event.

[0020] A knowledge base is built based on the collected event types and the detailed descriptions, handling strategies, and reporting suggestions corresponding to each event type.

[0021] Semantic vector features are extracted from the constructed knowledge base to obtain the corresponding semantic vector features;

[0022] Based on the obtained semantic vector features and the corresponding processing strategies and reporting suggestions, a vector knowledge base is constructed.

[0023] Preferably, in the event fusion analysis and processing method of the present invention, a query vector is constructed based on the target event, the constructed query vector is used to search the vector knowledge base, and a model analysis prompt is generated based on the search results and graphic information prompts, including:

[0024] The actual values ​​are used to replace the parameters in the structured field information, scene labels, and image target descriptions of the target event to obtain the corresponding text description;

[0025] Semantic vector features are extracted from the obtained text description to obtain query vectors. The obtained query vectors are used to retrieve the constructed vector knowledge base, and the retrieval results are used as rules for the event knowledge base.

[0026] The model analyzes the prompts based on the event knowledge base rules and the graphic and textual information prompts.

[0027] Preferably, in the event fusion analysis and processing method of the present invention, a large language model is used to obtain model analysis results based on model analysis prompts, and event fusion analysis and processing is performed based on the model analysis results, including:

[0028] Based on the description of the target event, model analysis results are obtained by inputting model analysis prompts into the large language model.

[0029] The obtained model analysis results are output as an event reporting request body. The event is then analyzed through the event reporting request body, and the corresponding handling strategies and reporting suggestions are retrieved.

[0030] According to a second aspect of the present invention, an event fusion analysis and processing system is provided. This system includes an event fusion analysis and processing server, which is configured to: acquire image information of a target event; identify the image information of the target event as scene tags; construct a corresponding target type list based on the scene tags; detect targets of interest from the image information of the target event based on the constructed target type list; and generate an image target description of the target event; acquire structured field information of the target event; generate corresponding image-text information prompts based on the image target description and structured field information of the target event; construct a knowledge base by aggregating the event types of interest; construct a vector knowledge base by extracting vector features from the constructed knowledge base; construct a query vector based on the target event; retrieve the vector knowledge base using the constructed query vector; generate model analysis prompts based on the retrieval results and image-text information prompts; obtain model analysis results using a large language model based on the model analysis prompts; and perform event fusion analysis processing based on the model analysis results.

[0031] According to a third aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect of the present invention.

[0032] The event fusion analysis and processing method and system of this invention identifies the scene type of the target event, processes it based on the softmax value of the frequency of occurrence in historical event data, selects a reasonable number of key targets to achieve a visual description of the relevant target content, and extracts specific targets of interest according to the scene type, avoiding the general detection of irrelevant targets, and achieving fusion processing of image and text events; by constructing a model to analyze prompt words, it obtains the ability to comprehensively describe event information and reference basis; it fully utilizes the comprehensive knowledge processing capabilities of large models to realize customized handling strategies and reporting suggestions for events based on their descriptions and content, and provides comprehensive analysis. Through basic image classification and target detection, and a basic large language model, it achieves event description with low computing resources without the need for a larger-scale multimodal model. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of a system for an event fusion analysis and processing method applicable to embodiments of the present invention;

[0035] Figure 2 This is a flowchart illustrating the steps of an event fusion analysis and processing method according to an embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram illustrating the construction of a target object type map in an event fusion analysis and processing method according to an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the structure of the device provided by the present invention. Detailed Implementation

[0038] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0040] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0041] Figure 1 An exemplary system for an event fusion analysis and processing method applicable to embodiments of the present invention is shown. For example... Figure 1 As shown, the system may include an event fusion analysis and processing server 101, a communication network 102, and / or one or more event fusion analysis and processing clients 103. Figure 1 The example in the text is client 103, which is used for the fusion analysis and processing of multiple events.

[0042] The event fusion analysis and processing server 101 can be any suitable server used to store information, data, programs, and / or any other suitable type of content. In some embodiments, the event fusion analysis and processing server 101 can perform appropriate functions. For example, in some embodiments, the event fusion analysis and processing server 101 can be used for event fusion analysis and processing. As an optional example, in some embodiments, the event fusion analysis and processing server 101 can be used to perform event fusion analysis and processing by building a vector knowledge base and generating model analysis prompts. For example, the event fusion analysis and processing server 101 can be used to acquire image information of target events, identify the image information of target events as scene tags, construct a corresponding target type list based on the scene tags, detect targets of interest from the image information of target events based on the constructed target type list, and generate image target descriptions of target events; acquire structured field information of target events, and generate corresponding image and text information prompts based on the image target descriptions and structured field information of target events; construct a knowledge base by aggregating the event types of interest, and construct a vector knowledge base by extracting vector features from the constructed knowledge base; construct query vectors based on target events, and use the constructed query vectors to search the vector knowledge base, generating model analysis prompts based on the search results and image and text information prompts; use a large language model to obtain model analysis results based on the model analysis prompts, and perform event fusion analysis processing based on the model analysis results.

[0043] As another example, in some embodiments, the event fusion analysis and processing server 101 may send the event fusion analysis and processing method to the event fusion analysis and processing client 103 for user use, based on a request from the event fusion analysis and processing client 103.

[0044] As an optional example, in some embodiments, the event fusion analysis processing client 103 is used to provide a visual processing interface, which is used to receive a user's selection input operation for event fusion analysis processing, and to obtain and display the processing interface corresponding to the option selected by the selection input operation from the event fusion analysis processing server 101 in response to the selection input operation. The processing interface displays at least information about event fusion analysis processing and operation options for event fusion analysis processing.

[0045] In some embodiments, communication network 102 may be any suitable combination of one or more wired and / or wireless networks. For example, communication network 102 may include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. Event fusion analysis processing client 103 may be connected to communication network 102 via one or more communication links (e.g., communication link 104), which may be linked to event fusion analysis processing server 101 via one or more communication links (e.g., communication link 105). Communication links may be any communication link suitable for transmitting data between event fusion analysis processing client 103 and event fusion analysis processing server 101, such as network links, dial-up links, wireless links, hardwired links, any other suitable communication links, or any suitable combination of such links.

[0046] The event fusion analysis processing client 103 may include any one or more clients that present an interface related to event fusion analysis processing in a suitable form for user use and operation. In some embodiments, the event fusion analysis processing client 103 may include any suitable type of device. For example, in some embodiments, the event fusion analysis processing client 103 may include a mobile device, tablet computer, laptop computer, desktop computer, and / or any other suitable type of client device.

[0047] Although the event fusion analysis and processing server 101 is illustrated as a single device, in some embodiments, any suitable number of devices may be used to perform the functions performed by the event fusion analysis and processing server 101. For example, in some embodiments, multiple devices may be used to implement the functions performed by the event fusion analysis and processing server 101. Alternatively, cloud services may be used to implement the functions of the event fusion analysis and processing server 101.

[0048] Based on the above system, this invention provides an event fusion analysis and processing method, which will be described in the following embodiments.

[0049] Reference Figure 2 The diagram illustrates a flowchart of the steps of an event fusion analysis and processing method according to an embodiment of the present invention.

[0050] The event fusion analysis and processing method of this embodiment can be executed on the event fusion analysis and processing server. The event fusion analysis and processing method includes the following steps:

[0051] Step S201: Obtain image information of the target event, identify the image information of the target event as scene labels, construct a corresponding target type list based on the scene labels, detect the target of interest from the image information of the target event based on the constructed target type list, and generate an image target description of the target event.

[0052] As an optional example, the method of this embodiment of the invention uses historical image data of events and a set of event image labels to train an image classification model. Using the trained image classification model, the target event is identified as a scene label in the set of event image labels based on the image information of the target event.

[0053] To improve the recognition accuracy of the image classification model, this embodiment trains the model using images of the target event. The image classification model in this embodiment is a deep learning model for image analysis, capable of classifying an image, such as distinguishing whether the content in an image is small animals, flowers and trees, street vendors, a river scene, vehicles, or garbage. The image classification model can be a ResNet model, a ResNeXt model, or a YoLo model, etc., and this embodiment does not impose any restrictions on this.

[0054] For example, after acquiring image information of the target event, the method in this embodiment uses an image classification model to identify the image information of the target event as a type of a specified event image label set DL. An example of an event image label set DL is as follows: DL includes the following scenes: street vendors, markets, small motor vehicles, dump trucks, large trucks, buses, non-motorized vehicles, garbage, rivers, advertisements, clothes drying, stray dogs, etc. The identification process is to identify the image information of the target event as a specific scene label dl in the DL, such as identifying the acquired image information of the target event as a river scene in the event image label set DL.

[0055] After completing the scene label recognition for the target event, the method in this embodiment also needs to construct a corresponding target object type map for each scene label in the image label set. The scene label, the target object, and the correlation between the scene label and the target object are used as components of the target object type map. Based on the constructed target object type map and the scene label corresponding to the target event, a target type list is constructed. For example, this embodiment pre-constructs a target object type map Gdl related to each specific scene label dl in the event image label set DL. The constructed target object type map Gdl is: <scene label dl<-x_i->target Obj_i>, where x_i is the correlation between the image scene label type and the target Obj_i, and its value is taken from the softmax of the proportion of each correlation frequency. Specifically, the target list for each scene label is the top 3 to 5 targets with the highest correlation frequency, and the frequency must be greater than or equal to 0.2, i.e., a minimum of 3 and a maximum of 5, with the 4th and 5th targets having a frequency greater than 0.2. For example, among the targets related to the river scene, boats account for 0.55, aquatic plants 0.45, people 0.23, fishing 0.2, and bicycles 0.06 (there may be duplicates, so the sum is greater than 1). Selecting the first four and discarding bicycles (which have a value less than 0.2), we use Softmax([0.55,0.45,0.23,0.2]) = [0.30,0.27,0.22,0.21]. Softmax is a current technology and will not be elaborated upon here. Therefore, the obtained target object type map is: River <-0.30 -> Boat River <-0.27 -> Aquatic Plants River <-0.22 -> People River <-0.21 -> Fishing. Figure 3 This is a schematic diagram illustrating the construction of a target object type map in an event fusion analysis and processing method according to an embodiment of the present invention.

[0056] This embodiment constructs a target object type map Gdl corresponding to each specific scene label dl in the event image label set DL. The purpose is to focus on relevant targets within each scene of interest, while targets with low relevance are excluded from the attention list. For example, in a river scene, attention is paid to people, fishing, and boats, while attention is paid to grass, shrubs, and stones on the ground; in a garbage dump scene, attention is paid to information such as whether the garbage dump is full, while attention is paid to targets such as the garbage station and bicycles parked nearby.

[0057] After constructing the target type list, the method in this embodiment employs a target detection model to detect targets of interest from the image information of the target event based on the constructed target type list. Based on the detected targets of interest and their quantity, an image target description of the target event is generated. The target detection model in this embodiment is also a deep learning model for image analysis, capable of detecting specific target types and their locations in an image.

[0058] For example, the method in this embodiment obtains a list of target types of interest corresponding to the scene type dl of the image information of the target event and the constructed target object type map Gdl based on the specific scene type dl of the image information of the target event. The target detection model Md is used to detect the targets in the image information of the target event. The detected targets are limited to the part of interest in the target type list, and a series of detected targets are obtained.

[0059] Based on these detected targets and the number of each type, the image target description dl_desc is obtained. Here, dl_desc is a simple descriptive text, similar to "xx targets of type xx, xx targets of type yy". For example, in a river scene, 10 people, 3 fishing, 0 boats, and 3 aquatic plants were found. The accuracy of the quantifiers is not a concern here.

[0060] Step S202: Obtain the structured field information of the target event, and generate corresponding graphic and textual information prompts based on the image target description and structured field information of the target event.

[0061] As an optional example, the method of this embodiment uses the image target description and structured field information of the target event to replace the variables at the corresponding positions in the prompt template of the large language model, thereby obtaining the image and text information prompt of the target event. It should be noted that in this embodiment, the structured field information of the target event includes the text description, time, location, and reporter of the target event. Those skilled in the art can obtain other structured field information of the target event according to the needs of the actual application scenario when implementing the method of this invention; this embodiment does not impose any limitations on this.

[0062] For example, the method in this embodiment takes the event text description Edesc and the image target description dl_desc from the collected structured field information, and replaces the variables at the corresponding positions in the template according to the set large language model prompt word template to obtain the image and text information prompt word LM-q. The following uses a specific example to explain the replacement rule in detail:

[0063] "You are a handler receiving reports of social events. At {Event Discovery Time Etime}, you have discovered the following event description: {Event Text Description Edesc}. Based on the on-site image, target detection analysis reveals the current scene to be: {Scene Type dl}; Target: {Image Target Description dl_desc}. The relevant handling reference is: {event-kg-rule}."

[0064] Please provide a comprehensive analysis of this incident report, briefly describing the findings and the corresponding response strategy.

[0065] After substituting the aforementioned information, the graphic information prompt word LM-q is obtained as follows:

[0066] "You are a handler receiving reports of social events. At 10:23:45 on 2023-10-30, you discovered the following event description: There are fishermen by the river. Based on the on-site image, target detection analysis shows the current scene is: riverside; targets: 10 people, 3 fishing, 3 patches of aquatic plants, 0 boats. The relevant handling reference is: {event-kg-rule}."

[0067] Please provide a comprehensive analysis of this incident report, briefly describing the findings and the corresponding response strategy.

[0068] Step S203: Construct a knowledge base by aggregating the types of events of interest, and construct a vector knowledge base by extracting vector features from the constructed knowledge base.

[0069] As an optional example, the method of this embodiment of the invention gathers the event types of concern according to the needs of concern, collects the detailed description information, handling strategies and reporting suggestions corresponding to each event type; constructs a knowledge base based on the gathered event types and the detailed descriptions, handling strategies and reporting suggestions corresponding to each event type; extracts semantic vector features from the constructed knowledge base to obtain the corresponding semantic vector features; and constructs a vector knowledge base based on the obtained semantic vector features and the corresponding handling strategies and reporting suggestions.

[0070] For example, this embodiment of the method, by aggregating all relevant event types and their detailed descriptions, allows for the detailed collection and summarization of event types and their descriptions based on different national standards or points of interest in practical applications. Examples of event types and their detailed descriptions can be found below:

[0071] The category of urban appearance and environment mainly includes unauthorized construction, illegal extensions to slopes, unclean building facades, exposed garbage, accumulated garbage and construction waste, and damaged roads;

[0072] Advertising and promotional materials mainly include illegal small advertisements, unauthorized outdoor advertisements, unauthorized signs and plaques, and advertisements with non-standard language.

[0073] Construction management issues mainly include construction disturbances, construction dust, construction waste, construction site fencing problems, and construction encroachment on roads.

[0074] Street order issues mainly include unlicensed street vendors, morning (night) market management problems, homeless begging, illegal scrap metal collection, and businesses operating outside their premises.

[0075] Emergency incidents mainly include water supply pipe ruptures, gas pipe ruptures, road collapses, drainage pipe blockages, and mass incidents;

[0076] Other incidents mainly include illegal high-altitude suspended operations and street-side slaughtering.

[0077] It should be noted that when implementing the method of the present invention, those skilled in the art can collect other event types and corresponding detailed descriptions as needed in the actual scenario, and the method of this embodiment does not limit this.

[0078] After collecting event types and their corresponding detailed descriptions, the method in this embodiment organizes the handling strategies, reporting recommendations, and guidance schemes for the corresponding event types into a knowledge base. This part can describe in detail the handling strategies corresponding to various potential branch possibilities. For example, different strategies can be used for different ranges of numerical numbers.

[0079] Step S204: Construct a query vector based on the target event, use the constructed query vector to search the vector knowledge base, and generate a model to analyze prompt words based on the search results and graphic information prompt words.

[0080] As an optional example, the method in this embodiment replaces the parameters in the structured field information, scene labels, and image target descriptions of the target event with actual values ​​to obtain the corresponding text description; semantic vector features are extracted from the obtained text description to obtain a query vector; the obtained query vector is used to retrieve the constructed vector knowledge base, and the retrieval results are used as the event knowledge base rules; the event knowledge base rules and image and text information prompt words are used to generate a model to analyze prompt words.

[0081] For example, the method in this embodiment uses an existing BERT language model to extract semantic vector features from the retrieval portion of the entries in the knowledge base; the extracted corresponding vector features are stored in a vector database, along with corresponding handling strategies, reporting suggestions, and other supplementary information, thus forming a vector knowledge base; the structured field information "Event Description: {Event Text Description Edesc}, Scene: {Scene Type dl}, Target: {Image Target Description dl_desc}" is obtained, where the curly braces are replaced with actual values ​​to obtain the corresponding text description reqdesc; the text description reqdesc is used to extract semantic features from the same BERT language model to obtain a query vector, which is then used to retrieve the vector knowledge base; the best handling strategy and reporting suggestion at the semantic level are obtained from the above query, and this query result is defined as the event knowledge base rule event-kg-rule; the obtained event knowledge base rule event-kg-rule, combined with the image and text information prompt word LM-q, yields a comprehensive event description and handling strategy reporting suggestion as a model analysis prompt word, denoted as the large model analysis prompt word LM-Q.

[0082] A complete example of the large model analysis cue word LM-Q obtained according to the method of this embodiment is as follows:

[0083] "You are a staff member handling reports of social incidents. At 10:23:45 on October 30, 2023, you discovered the following incident description: There are anglers on the riverbank. Based on target detection analysis of the on-site images, the scene is: riverbank; targets: 10 people, 3 fishing, 3 patches of aquatic plants, 0 boats. Relevant references are: Fishing regulations: Fishing is prohibited near high-voltage power lines; fishing is prohibited in the Yangtze River and its tributaries. Handling strategy: Those fishing on riverbanks or in reservoirs should comply with the regulations of local management personnel and must not damage the surrounding ecological environment..."

[0084] Please provide a comprehensive analysis of this incident report, briefly describing the findings and the corresponding response strategy.

[0085] Step S205: Use a large language model to obtain model analysis results based on model analysis prompts, and perform event fusion analysis based on the model analysis results.

[0086] As an optional example, the method in this embodiment obtains model analysis results by inputting model analysis prompts into a large language model based on the description information of the target event; outputs the obtained model analysis results as a request body for event reporting; and performs event analysis and retrieves the corresponding handling strategies and reporting suggestions through the request body for event reporting.

[0087] The following specific examples illustrate the above steps of the method in the embodiments of the present invention in detail:

[0088] An example result for a phishing incident is as follows:

[0089] Event Name: River Fishing Activity

[0090] Event Time: 2023-10-30 10:23:45

[0091] Location of the incident: Riverside

[0092] Discovery content:

[0093] Ten people participated in the fishing activity by the river, three of whom were actually fishing.

[0094] There are three patches of aquatic plants on site, which are scattered.

[0095] No ships appeared.

[0096] Handling strategy:

[0097] Anglers are reminded to abide by relevant regulations, pay attention to environmental protection, and not litter.

[0098] Pay attention to the ecological environment along the river and stop any acts that damage the environment.

[0099] Strengthen safety management along the riverbank, set up safety signs, and prevent accidents.

[0100] Regularly inspect the riverbanks to understand the fishing activities being conducted and ensure they are compliant and legal.

[0101] Comprehensive analysis:

[0102] This incident involved riverside fishing, which was relatively small in scale, and no violations were found. In handling this matter, attention should be paid to environmental protection, safety, and the ecological environment to ensure the legal and orderly conduct of fishing activities. At the same time, communication should be maintained with the fishing enthusiast community to promote environmental awareness and improve the organization and management of fishing activities.

[0103] Recommended handling for reporting:

[0104] To ensure compliance with regulations for riverside fishing activities;

[0105] Pay attention to ecological environment and safety issues;

[0106] To ensure the normal conduct of people's leisure and entertainment activities.

[0107] "

[0108] In this embodiment, the model analysis results can be specified as JSON output to form an event reporting request body, thereby realizing event analysis and its automatic handling strategy and reporting suggestions for subsequent manual verification or allocation.

[0109] The event fusion analysis and processing method and system of this invention identifies the scene type of the target event, processes it based on the softmax value of the frequency of occurrence in historical event data, selects a reasonable number of key targets to achieve a visual description of the relevant target content, and extracts specific targets of interest according to the scene type, avoiding the general detection of irrelevant targets, and achieving fusion processing of image and text events; by constructing a model to analyze prompt words, it obtains the ability to comprehensively describe event information and reference basis; it fully utilizes the comprehensive knowledge processing capabilities of large models to realize customized handling strategies and reporting suggestions for events based on their descriptions and content, and provides comprehensive analysis. Through basic image classification and target detection, and basic large language models, it achieves event description with low computing resources without the need for larger-scale multimodal models.

[0110] like Figure 4 As shown, the present invention also provides a device including a processor 310, a communication interface 320, a memory 330 for storing a processor-executable computer program, and a communication bus 340. The processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 implements the aforementioned method for high-speed access to ORC external tables by running the executable computer program.

[0111] The computer program in memory 330, when implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0112] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected based on actual needs to achieve the purpose of this embodiment. Those skilled in the art can understand and implement this without any creative effort.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0114] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An event fusion analysis and processing method, characterized in that, The method includes: The process involves acquiring image information of the target event, identifying the image information of the target event as scene labels, constructing a corresponding target type list based on the scene labels, detecting targets of interest from the image information of the target event based on the constructed target type list, and generating an image target description of the target event. Obtain the structured field information of the target event, and generate corresponding image and text information prompts based on the image target description and structured field information of the target event; A knowledge base is built by aggregating the types of events of interest, and a vector knowledge base is built by extracting vector features from the built knowledge base. A query vector is constructed based on the target event. The constructed query vector is used to search the vector knowledge base. Based on the search results and graphic information prompts, a model is generated to analyze the prompts. A large language model is used to obtain model analysis results based on model analysis prompts, and event fusion analysis is performed based on the model analysis results. A query vector is constructed based on the target event. This query vector is then used to search the vector knowledge base. Based on the search results and text / image information prompts, a model is generated to analyze prompts, including: The actual values ​​are used to replace the parameters in the structured field information, scene labels, and image target descriptions of the target event to obtain the corresponding text description; Semantic vector features are extracted from the obtained text description to obtain query vectors. The obtained query vectors are used to retrieve the constructed vector knowledge base, and the retrieval results are used as rules for the event knowledge base. Based on the event knowledge base rules and graphic / textual information prompts, a model is generated to analyze prompts; A large language model is used to obtain model analysis results based on prompt words. Event fusion analysis is then performed based on these results, including: Based on the description of the target event, model analysis results are obtained by inputting model analysis prompts into the large language model. The obtained model analysis results are output as an event reporting request body. The event is then analyzed through the event reporting request body, and the corresponding handling strategies and reporting suggestions are retrieved.

2. The event fusion analysis and processing method according to claim 1, characterized in that, Acquire image information of the target event, identify the image information of the target event as scene labels, construct a corresponding target type list based on the scene labels, detect targets of interest from the constructed target type list, and generate an image target description of the target event, including: An image classification model is trained using historical image data of the event and a set of event image labels. The trained image classification model is then used to identify the target event as a scene label in the set of event image labels based on the image information of the target event. For each scene label in the image label set, construct a corresponding target object type map. Based on the constructed target object type map and the scene label corresponding to the target event, construct a target type list. An object detection model is used to detect objects of interest from the image information of the target event based on a constructed list of target types. Based on the detected objects of interest and their number, an image object description of the target event is generated.

3. The event fusion analysis and processing method according to claim 2, characterized in that, Construct a corresponding target object type map for each scene label in the image label set, including: incorporating scene labels, target objects, and the correlation between scene labels and target objects as components of the target object type map.

4. The event fusion analysis and processing method according to claim 1, characterized in that, Obtain the structured field information of the target event, and generate corresponding image and text information prompts based on the image target description and structured field information of the target event. This includes: using the image target description and structured field information of the target event, replacing the variables at the corresponding positions in the prompt template of the large language model, and obtaining the image and text information prompts of the target event.

5. The event fusion analysis and processing method according to claim 1, characterized in that, The structured field information for the target event includes a text description of the target event, time, location, and the person who reported it.

6. The event fusion analysis and processing method according to claim 1, characterized in that, A knowledge base is constructed by aggregating the types of events of interest. A vector knowledge base is then built by extracting vector features from the constructed knowledge base, including: Based on the needs of attention, we collect the types of events that are of concern, and gather detailed descriptions, handling strategies and reporting suggestions for each type of event. A knowledge base is built based on the collected event types and the detailed descriptions, handling strategies, and reporting suggestions corresponding to each event type. Semantic vector features are extracted from the constructed knowledge base to obtain the corresponding semantic vector features; Based on the obtained semantic vector features and the corresponding processing strategies and reporting suggestions, a vector knowledge base is constructed.

7. An event fusion analysis and processing system, characterized in that, The system includes an event fusion analysis and processing server, which is used to: acquire image information of a target event, identify the image information of the target event as scene tags, construct a corresponding target type list based on the scene tags, detect targets of interest from the image information of the target event based on the constructed target type list, and generate an image target description of the target event; Obtain structured field information of the target event, and generate corresponding image and text information prompts based on the image target description and structured field information of the target event; construct a knowledge base by aggregating the event types of interest, and construct a vector knowledge base by extracting vector features from the constructed knowledge base; A query vector is constructed based on the target event. The constructed query vector is used to search the vector knowledge base. Based on the search results and text and image information prompts, model analysis prompts are generated. A large language model is used to obtain model analysis results based on the model analysis prompts. Event fusion analysis is then performed based on the model analysis results. A query vector is constructed based on the target event. The constructed query vector is used to search the vector knowledge base. Based on the search results and image / text information prompts, a model is generated to analyze the prompts, including: replacing the parameters in the structured field information, scene tags, and image target descriptions of the target event with actual values ​​to obtain the corresponding text description; extracting semantic vector features from the obtained text description to obtain the query vector; using the obtained query vector to search the constructed vector knowledge base; and using the search results as rules for the event knowledge base. The system generates model analysis prompts based on event knowledge base rules and graphic / textual information prompts; it then uses a large language model to obtain model analysis results based on these prompts, and performs event fusion analysis based on these results. This includes: obtaining model analysis results by inputting model analysis prompts into the large language model based on the description information of the target event; outputting the obtained model analysis results as an event reporting request body; and then performing event analysis and retrieving the corresponding handling strategies and reporting suggestions based on the event reporting request body.

8. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method according to any one of claims 1-6.

Images