Urban event detection method, device and storage medium based on multimodal large model

Abstract

The present application discloses a method, device and storage medium for detecting urban events based on a multimodal large model, and relates to the field of computer application technology. The method includes: obtaining a public data set and an event data set of urban events; constructing a multimodal recognition model including a visual encoder, a linear projection layer and a large language model; establishing a stage training method for the multimodal recognition model, and training the multimodal recognition model in stages based on the public data set and the event data set; inputting the data to be tested into the trained multimodal recognition model to obtain event recognition results. The embodiment of the present application achieves the technical effect of reducing the number of multimodal recognition models in the process of urban event detection, improving the transfer learning ability and generalization ability of the multimodal recognition model to new scenes, saving the training time and resources of the multimodal recognition model, and making the maintenance and upgrade process of the multimodal recognition model relatively simple.

Description

Technical Field

[0001] The present application relates to the field of computer application technology, and in particular to a method, device and storage medium for detecting urban events based on a multimodal large model. Background Art

[0002] Urban event detection is a very important task in urban management. Cameras, drones and other equipment will capture a large amount of images, videos and other data. Through large models, these data can be analyzed to quickly detect abnormal events and notify city managers to take effective measures to handle these events in a timely manner to ensure the order of the city and the safety of citizens.

[0003] In the existing technology, several specific small models are often used to identify and analyze these tasks. However, when dealing with complex and changing urban events, small models often show insufficient transfer learning capabilities and generalization capabilities for new scenarios, which forces city managers to design and train dedicated models for different event tasks. This not only takes up a lot of time and resources, but also with the increase in the types of event tasks, the number of small models that need to be maintained and upgraded is also increasing, increasing the pressure and challenges of management and maintenance work.

[0004] In addition, since the detection process of urban event tasks often needs to take into account specific scenarios and needs, for example, a detection model that performs well in a commercial area may not be applicable in a residential area. Therefore, small models need to be constantly customized and adjusted, which increases the cost of model commercialization and makes rapid deployment more difficult. Summary of the invention

[0005] The embodiments of the present application provide a method, device and storage medium for urban event detection based on a multimodal large model, which solves the problem of poor model generalization ability and high economic cost in the existing technology for urban event detection. It achieves the technical effect of reducing the number of models in the process of urban event detection, improving the model transfer learning ability and generalization ability for new scenarios, saving model training time and resources, and making the model maintenance and upgrade process relatively simple.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting urban events based on a multimodal large model, comprising: acquiring and constructing a public data set and an event data set of urban events; wherein the event data set includes a text data set and a labeled data set; constructing a multimodal recognition model, wherein the multimodal recognition model includes a visual encoder, a linear projection layer and a large language model; inputting the public data set into the linear projection layer for training to achieve alignment of visual features with text features; performing initial training and fine-tuning training on the multimodal recognition model based on the public data set and the event data set; wherein the initial training uses the image data in the public data set and the text data set as input, and the fine-tuning training uses the image data in the public data set and the labeled data set as input; inputting the data to be tested into the trained multimodal recognition model to obtain an event recognition result, and if the event recognition result includes a regulatory event, an alarm is issued; wherein the event recognition result includes an event category, corresponding location information and camera information.

[0007] In combination with the first aspect, in a possible implementation, the text dataset includes: using a multimodal big model to generate description information and coordinate information of the image data in the public dataset; designing a question-and-answer dialogue corresponding to the image data based on the description information and the coordinate information; inputting the image data into the multimodal big model, and using the questions in the question-and-answer dialogue as context information, so that the multimodal big model generates a detailed text description of the image data as the text dataset.

[0008] In combination with the first aspect, in a second possible implementation, the annotated data set includes: acquiring city images using a camera device and setting event categories; classifying and annotating the city images based on the event categories; normalizing the location information of the city images and converting them into a string format; the annotated city images and their location information in a string format constitute the annotated data set.

[0009] In combination with the first aspect, in a third possible implementation method, the inputting the public dataset into the linear projection layer for training to achieve alignment of visual features with text features includes: setting the image data in the public dataset to a first preset size, and inputting it as an input image into the linear projection layer; using the input image to train the projection matrix of the linear projection layer to align the visual features of the input image with the text features; and using the trained projection matrix to convert the visual features into text tags.

[0010] In combination with the third possible implementation manner of the first aspect, in a fourth possible implementation manner, the initial training includes: setting the image data in the public dataset and the text dataset to a second preset size and inputting them into the visual encoder, the linear projection layer and the large language model to improve the ability to recognize complex visual details; using the trained linear projection layer to convert the visual features into text tags; establishing a mapping relationship between the visual features and the text tags, and inputting them into the large language model to recognize event information.

[0011] In combination with the third possible implementation manner of the first aspect, in a fifth possible implementation manner, the fine-tuning training includes: setting the image data in the public dataset and the annotated dataset to a second preset size and inputting them into the linear projection layer and the large language model to improve the reasoning ability of the multimodal recognition model; using the trained linear projection layer to convert the visual features into text tags; establishing a mapping relationship between the visual features and the text tags, and inputting them into the large language model to recognize event information.

[0012] In combination with the fourth possible implementation manner or the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner, the initial training and fine-tuning training of the multimodal recognition model based on the public data set and the event data set also includes: segmenting the image data at the second preset size as training samples of the multimodal recognition model; wherein the second preset size is larger than the first preset size.

[0013] In the second aspect, an embodiment of the present application provides an urban event detection device based on a multimodal large model, including: a data acquisition module, used to acquire public data based on an open source database, and to acquire urban events based on urban cameras; a server module, used to collect the public data to obtain a public data set; and describe and annotate the urban events to obtain an event data set; a model training module, used to train the multimodal large model based on the public data set and the event data set; an urban event detection module, used to input the data to be tested into the trained multimodal recognition model to obtain event recognition results, and perform alarm processing based on the event recognition results.

[0014] In a third aspect, an embodiment of the present application provides a device for executing a method for detecting urban events based on a multimodal large model, comprising: a processor; a memory for storing processor executable instructions; when the processor executes the executable instructions, a method for detecting urban events based on a multimodal large model as described in the first aspect and any one of the first aspects is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium, including a medium for storing a computer program or instruction. When the computer program or instruction is executed, a method for detecting urban events based on a multimodal large model as described in the first aspect and any one of the first aspects is implemented.

[0016] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0017] This application uses public data sets and multiple event data sets to train the multimodal recognition model, thereby enhancing the learning ability of the multimodal recognition model and its generalization ability for new scenarios. At the same time, image features and text features are used as model input features of the multimodal recognition model, and there is no need to train the model separately for each event. When adding new tasks, the trained multimodal recognition model can be used directly, which reduces the resource consumption of each training, improves the flexibility and versatility of the multimodal recognition model, saves the time of training the multimodal recognition model, and avoids the complexity of deploying and managing multiple multimodal recognition models, reducing the deployment and maintenance process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of a method for detecting urban events based on a multimodal large model provided in an embodiment of the present application;

[0020] Figure 2 A structural diagram of a multimodal recognition model provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the structure of an urban event detection device based on a multimodal large model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The present application embodiment provides a method for detecting urban events based on a multimodal large model. Figure 1 As shown, Figure 1 It is a flowchart of the urban event detection method based on the multimodal large model provided in an embodiment of the present application, including steps 101 to 105.

[0024] Step 101: Obtain and construct a public dataset and an event dataset of city events.

[0025] Step 102: construct a multimodal recognition model, which includes a visual encoder, a linear projection layer, and a large language model.

[0026] Step 103: Input the public dataset into the linear projection layer for training to align the visual features with the text features.

[0027] Step 104: Perform initial training and fine-tuning training on the multimodal recognition model based on the public dataset and the event dataset.

[0028] Step 105: Input the data to be tested into the trained multimodal recognition model to obtain event recognition results. If the event recognition results include regulatory events, an alarm is issued. The event recognition results include event categories, corresponding location information, and camera information.

[0029] This application uses public data sets and multiple event data sets of urban events to train different layers of the multimodal recognition model in stages and training methods, thereby enhancing the learning ability of the multimodal recognition model and the generalization ability of the multimodal recognition model for new scenarios. At the same time, the image features and text features in the data set are used as the model input features of the multimodal recognition model. There is no need to train the model separately for each event. When adding new tasks, the trained multimodal recognition model can be used directly, reducing the loss of training resources each time, improving the flexibility and versatility of the multimodal recognition model, saving time for training the multimodal recognition model, and avoiding the complexity of deploying and managing multiple multimodal recognition models, reducing the deployment and maintenance process.

[0030] In the embodiment of the present application, the event dataset includes a text dataset and a annotation dataset.

[0031] The process of obtaining the text dataset includes:

[0032] First, the multimodal large model is used to generate the description information and coordinate information of the image data in the public dataset.

[0033] For example, the public dataset here uses the open source dataset CoCo.

[0034] Generate dialogue text, input the open source dataset CoCo into the multimodal large model, simply describe the objects in the open source dataset CoCo, and obtain the description information and coordinate information of the image data corresponding to the objects.

[0035] Secondly, based on the description information and coordinate information, design a question-and-answer dialogue about the image data corresponding to the object. Ask about different information in the image data and give corresponding answers.

[0036] To answer different questions about an image, you need to understand the visual information in the image. Based on the answers generated by the question-answering dialogue, you can get multiple text messages describing the image.

[0037] Again, input the images in the open source dataset CoCo into the multimodal large model and select the above text information to output detailed image data.

[0038] Finally, the detailed image data is input into the multimodal large model, and the questions in the question-answering dialogue are used as context information to enable the multimodal large model to generate a detailed text description of the image data as a text dataset.

[0039] In the embodiment of the present application, the method for obtaining the annotated data set includes:

[0040] First, the camera equipment is used to obtain city images and set event categories, including 20 event categories such as occupying the road for business, drying clothes on the street, climbing over guardrails, fighting, etc.

[0041] Secondly, urban images are classified and labeled based on event categories.

[0042] Again, the location information of the city image is normalized and converted into a string format.

[0043] In an embodiment of the present application, the position information will be normalized to between [0, 1] and converted into a string format of [x1, y1], [x2, y2], where [x1, y1] represents the information of the upper left corner and [x2, y2] represents the information of the lower right corner.

[0044] Finally, the classified and annotated city images and their location information in string format constitute the annotated dataset.

[0045] It should be noted that in order to train the corresponding urban event detection model, this application constructs a multimodal recognition model, which specifically includes a visual encoder, a linear projection layer and a large language model.

[0046] Figure 2 The structure diagram of the multimodal recognition model provided in the embodiment of the present application is as follows: Figure 2As shown, in an embodiment of the present application, the vision encoder (Vision Encoder) uses the CLIP (English full name: Contrastive Language-Image Pre-Training, contrast language-image pre-training neural network) open source Vit-Large pre-training model, and the size of each patch is 14*14.

[0047] The linear projection layer is a Linear linear projection layer.

[0048] The large language model uses the Luoshu large model 13B model as the large language model.

[0049] At the same time, this application provides a multi-stage training method for a multimodal recognition model. Specifically, the public dataset is input into a linear projection layer for training, the purpose of which is to train a compatible visual encoder for a large language model, including:

[0050] The image data in the public dataset is set to a first preset size and is input as an input image into the linear projection layer to train the linear projection layer. The first preset size is 224*224.

[0051] The projection matrix of the linear projection layer is trained using the input image. Specifically, a projection matrix is ​​trained to map the visual features to vectors corresponding to the language model embedding space so as to align the visual features of the input image with the text features.

[0052] During this training process, since these large language models and visual encoders have been trained on large-scale datasets and have accumulated rich knowledge, the knowledge of the pre-trained models can be kept unaffected by freezing parameters.

[0053] It should be noted that the feature alignment process refers to aligning the data representations of different modalities (such as vision and language) into a common feature space. That is, data from different sources and in different forms can be understood and matched in the model in the same or similar way. This step fuses image and text data and processes them together.

[0054] The trained projection matrix is ​​used to convert visual features into textual tokens.

[0055] In step 104 mentioned in the embodiment of the present application, the initial training uses the image data in the public dataset and the text dataset as input, and the fine-tuning training uses the image data in the public dataset and the annotated dataset as input.

[0056] It should be noted that the initial training is high-resolution feature alignment training to improve the ability to recognize complex visual details, including:

[0057] The image data in the public dataset and the text dataset are set to a second preset size and input into the visual encoder to extract visual features. The second preset size is 336*336.

[0058] The trained linear projection layer is used to convert visual features into textual tokens.

[0059] A mapping relationship between visual features and text tags is established and input into a large language model to recognize event information.

[0060] Fine-tuning training aims to make the large language model have better reasoning ability. Fine-tuning training specifically includes:

[0061] The linear projection layer and the large language model are trained by using the public dataset and the annotated dataset. Meanwhile, the preset size of the graphic data is kept unchanged, which is still the second preset size.

[0062] The trained linear projection layer is used to convert visual features into textual tokens.

[0063] A mapping relationship between visual features and text tags is established and input into a large language model to recognize event information.

[0064] The training data and training objects of the above three stages (linear projection layer training, initial training and fine-tuning training) are different. They all use image text as the input of the multimodal recognition model and pictures as the input of the visual encoder to improve the visual features. At the same time, the text is preprocessed and converted into text tags, and then the visual features and text tags are merged into a string, and the merged result is input into the large language model.

[0065] Among them, in the above three stages of training, the settings of training hyperparameters are shown in Table 1:

[0066] Table 1

[0067]

[0068] Exemplarily, in the training process for the multimodal recognition model, the entire training process adopts autoregressive training. The autoregressive method predicts the next word based on the existing words. Its loss function is mainly used to evaluate the difference between the model prediction and the real word. The likelihood function is maximized during the training process so that the generated words are as close to the real words as possible.

[0069] In the embodiment of the present application, the multimodal recognition model is initially trained and fine-tuned based on the public data set and the event data set, and further includes:

[0070] The image data at a second preset size is sliced ​​and used as training samples for the multimodal recognition model, wherein the second preset size is larger than the first preset size.

[0071] Figure 3 A schematic diagram of the structure of a city event detection device based on a multimodal large model provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, including:

[0072] The data acquisition module is used to obtain public data based on open source databases and obtain urban events based on city cameras.

[0073] The server module is used to collect public data to obtain a public data set, and to describe and annotate city events to obtain an event data set.

[0074] The model training module is used to train large multimodal models based on public data sets and event data sets.

[0075] The urban event detection module is used to input the test data into the trained multimodal recognition model to obtain event recognition results and perform alarm processing based on the event results.

[0076] The urban event detection method, device and storage medium based on a multimodal large model provided in the embodiments of the present application can be applied to urban management events to help urban management personnel quickly detect illegal events such as illegal construction and occupying roads for business from a large number of videos, thereby improving the accuracy and speed of event detection while reducing the workload of manual review. In urban public safety, it is used for real-time monitoring of fires, traffic accidents, etc., and timely issuing warning information to users to reduce losses and ensure public safety.

[0077] In the deployment scheme of the embodiment of the present application, the trained model is connected to the camera device. After the camera captures the image or video, it is used as the input of the Luoshu multimodal large model. After the model is processed, if it contains a supervised event, the event name and location information in the picture, as well as the camera information, are returned to facilitate timely processing by the management personnel. If it does not contain a supervised event, no alarm is issued.

[0078] The embodiment of the present application obtains urban event data from urban cameras, drones, and open source databases, annotates the event categories and location information of urban event data, and then trains a multimodal large model to adapt it to user needs. Through the trained large model, the category and location information of the data to be obtained are obtained, and by building an event detection system, urban events are detected in real time. This application does not need to train a separate model for each event. When adding a new task, the trained model can be used directly to reduce the consumption of training resources, improve the versatility and flexibility of the model, save a large number of events, and avoid the complexity of deploying and managing multiple models, simplifying the deployment and maintenance process.

[0079] Although the present application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in this embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in sequence or in parallel according to the method shown in this embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment).

[0080] Some modules in the apparatus described in the present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0081] The devices or modules described in the above application embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0082] The methods, devices or modules described in this application can be implemented in the form of computer-readable program codes. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program codes (such as software or firmware) that can be executed by the (micro) processor, logic gates, switches, application-specific integrated circuits (English: Application Specific Integrated Circuit; Abbreviation: ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included in it for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.

[0083] An embodiment of the present application further provides a device, comprising: a processor; a memory for storing processor-executable instructions; when the processor executes the executable instructions, the method described in the embodiment of the present application is implemented.

[0084] The embodiments of the present application also provide a non-volatile computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed, the method described in the embodiments of the present application is implemented.

[0085] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist independently, or two or more modules may be integrated into one module.

[0086] The above storage media include but are not limited to random access memory (RAM), read-only memory (ROM), cache, hard disk (HDD) or memory card. The memory can be used to store computer program instructions.

[0087] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0088] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0089] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A method for detecting urban events based on a multimodal large model, characterized in that: include: Acquire and construct a public data set and an event data set of urban events; wherein the event data set includes a text data set and an annotation data set; Constructing a multimodal recognition model, the multimodal recognition model comprising a visual encoder, a linear projection layer, and a large language model; Inputting the public dataset into the linear projection layer for training to achieve alignment of visual features and text features; Performing initial training and fine-tuning training on the multimodal recognition model based on the public dataset and the event dataset; wherein the initial training uses the image data in the public dataset and the text dataset as input, and the fine-tuning training uses the image data in the public dataset and the annotated dataset as input; The data to be tested is input into the trained multimodal recognition model to obtain an event recognition result. If the event recognition result includes a regulatory event, an alarm is issued; wherein the event recognition result includes an event category, corresponding location information and camera information.

2. The method according to claim 1, characterized in that The text dataset includes: Generate description information and coordinate information of the image data in the public dataset using the multimodal large model; Designing a question-and-answer dialogue corresponding to the image data based on the description information and the coordinate information; The image data is input into the multimodal large model, and the questions in the question-answering dialogue are used as context information, so that the multimodal large model generates a detailed text description of the image data as the text dataset.

3. The method according to claim 1, characterized in that The annotated data set includes: Use camera equipment to obtain city images and set event categories; Classify and label the city image based on the event category; Normalizing the location information of the city image and converting it into a string format; The annotated city image and the location information in a string format constitute the annotated data set.

4. The method according to claim 1, characterized in that: The step of inputting the public dataset into the linear projection layer for training to achieve alignment of visual features with text features includes: Setting the image data in the public dataset to a first preset size and inputting it as an input image into the linear projection layer; Using the input image to train the projection matrix of the linear projection layer to align visual features of the input image with text features; The visual features are converted into text tags using the trained projection matrix.

5. The method according to claim 4, characterized in that The initial training includes: Setting the image data in the public dataset and the text dataset to a second preset size and inputting it into the visual encoder, the linear projection layer, and the large language model to improve the ability to recognize complex visual details; Using the trained linear projection layer, converting the visual features into text tags; A mapping relationship between the visual features and the text tags is established and input into the large language model to identify event information.

6. The method according to claim 4, characterized in that The fine-tuning training comprises: Setting the image data in the public dataset and the annotated dataset to a second preset size and inputting them into the linear projection layer and the large language model to improve the reasoning ability of the multimodal recognition model; Using the trained linear projection layer, converting the visual features into text tags; A mapping relationship between the visual features and the text tags is established and input into the large language model to identify event information.

7. The method according to claim 5 or 6, characterized in that: The performing initial training and fine-tuning training on the multimodal recognition model based on the public data set and the event data set further includes: The image data at the second preset size is sliced ​​and used as training samples for the multimodal recognition model; wherein the second preset size is larger than the first preset size.

8. An urban event detection device based on a multimodal large model, characterized in that: include: Data acquisition module, used to acquire public data based on open source databases and urban events based on urban cameras; A server module, used for collecting the public data to obtain a public data set; and describing and annotating the city events to obtain an event data set; A model training module, used for training a multimodal large model based on the public data set and the event data set; The urban event detection module is used to input the data to be tested into the trained multimodal recognition model to obtain event recognition results, and perform alarm processing based on the event recognition results.

9. A device for executing a method for detecting urban events based on a multimodal large model, characterized in that: include: processor; a memory for storing processor-executable instructions; When the processor executes the executable instructions, it implements the urban event detection method based on a multimodal large model as described in any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: It includes a computer program or instruction for storing, when the computer program or instruction is executed, the urban event detection method based on a multimodal large model as described in any one of claims 1 to 7 is implemented.

Images