Event recognition method, traffic event recognition method and event recognition system

Abstract

The invention discloses an event identification method, a traffic event identification method and an event identification system, and relates to the field of large model technology and monitoring. The method comprises the following steps: in a process of monitoring at least one moving object in a to-be-monitored area, acquiring an event video in the to-be-monitored area under the condition that occurrence of an event caused by a target moving object is monitored; identifying an event contained in the event video, and determining whether the event video contains the event; when the event video contains the event, intercepting the event video to obtain at least one event image containing the event; and identifying the event contained in the at least one event image by using the multi-modal large model to obtain the event type of the event. The technical problem that the event recognition effect is poor in the related technology is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large model and monitoring, in particular to an event identification method, a traffic event identification method and an event identification system. BACKGROUND

[0002] At present, with the increase of the complexity of the traffic monitoring system, it becomes more and more important to monitor and analyze traffic events in real time. The current scheme can identify traffic events by using the trajectory of the vehicle, but the trajectory of the vehicle is prone to be incomplete and intersected, and is not stable in a complex environment, resulting in poor effect of traffic event identification.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] The embodiments of the present application provide an event identification method, a traffic event identification method and an event identification system to at least solve the technical problem of poor effect of event identification in the related art.

[0005] According to an aspect of the embodiments of the present application, an event identification method is provided, including: in the process of monitoring at least one moving object in a to-be-monitored area, if an event caused by a target moving object is monitored, an event video in the to-be-monitored area is acquired; an event included in the event video is intercepted to obtain at least one event image containing the event; a large multi-modal model is used to identify the event contained in the at least one event image, and an event type of the event is obtained.

[0006] According to an aspect of the embodiments of the present application, a traffic event identification method is provided, including: in the process of monitoring at least one vehicle in a traffic area, if a traffic event caused by a target vehicle is monitored, an event video in the traffic area is acquired; a traffic event included in the event video is intercepted to obtain at least one event image containing the traffic event; a large multi-modal model is used to identify the traffic event contained in the at least one event image, and a traffic event type of the traffic event is obtained, wherein the large multi-modal model is used to generate description information describing the traffic event, and the description information is used to determine the traffic event type of the traffic event.

[0007] According to an aspect of some embodiments of the present application, there is provided a method for event identification, comprising: in a process of monitoring at least one moving object in a to-be-monitored area, acquiring an event video in the to-be-monitored area by invoking a first interface in a case that an event caused by a target moving object is monitored, wherein the first interface comprises a first parameter, and a parameter value of the first parameter comprises the event video; intercepting an event contained in the event video to obtain at least one event image containing the event; identifying the event contained in the at least one event image by using a multi-modal large model to obtain an event type of the event, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type of the event; and outputting the event type by invoking a second interface, wherein the second interface comprises a second parameter, and a parameter value of the second parameter comprises the event type.

[0008] According to another aspect of some embodiments of the present application, there is also provided an event identification system, comprising: an event monitoring module configured to monitor whether an event caused by a target moving object occurs in a process of monitoring at least one moving object in a to-be-monitored area; an event tracking module configured to acquire an event video in the to-be-monitored area, and intercept an event contained in the event video to obtain at least one event image containing the event in a case that the event monitoring module monitors that the event occurs; and an event identification module configured to identify the event contained in the at least one event image by using a multi-modal large model to obtain an event type of the event, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type of the event.

[0009] According to another aspect of some embodiments of the present application, there is also provided a computer terminal, comprising: a memory storing an executable program; and a processor configured to run the program, wherein the program is configured to execute the method in each of the embodiments of the present application when the program is running.

[0010] According to another aspect of some embodiments of the present application, there is also provided a computer-readable storage medium, comprising a stored executable program, wherein the computer-readable storage medium is configured to control a device where the computer-readable storage medium is located to execute the method in each of the embodiments of the present application when the executable program is running.

[0011] According to another aspect of some embodiments of the present application, there is also provided a computer program product, comprising a computer program, wherein the computer program is configured to implement the method in each of the embodiments of the present application when the computer program is executed by a processor.

[0012] According to another aspect of some embodiments of the present application, there is also provided a computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and the computer program is configured to implement the method in each of the embodiments of the present application when the computer program is executed by a processor.

[0013] According to another aspect of the embodiments of the present application, a computer program is also provided, which, when executed by a processor, implements the method in the various embodiments of the present application.

[0014] In the embodiments of the present application, in the process of monitoring at least one moving object in the to-be-monitored region, when an event caused by a target moving object is monitored to occur, an event video in the to-be-monitored region is acquired; the event in the event video is intercepted to obtain at least one event image containing the event; and the event in the at least one event image is identified by using a multi-modal large model to obtain an event type of the event, thereby improving the identification effect of the event. It is easy to note that, in the process of monitoring the at least one moving object, if the target moving object causes the event to occur, the event video can be intercepted to obtain at least one event image, and the multi-modal large model can be used to obtain rich information related to the event, thereby improving the classification effect of the event, obtaining an event type with higher accuracy, and providing more detailed and rich information for the alarm of the event, thereby improving the event identification effect and solving the technical problem of poor event identification effect in the related art.

[0015] It is easy to note that the above general description and the following detailed description are only for exemplifying and explaining the present application, and do not constitute a limitation on the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings described herein are used to provide further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is an application scenario schematic diagram of an event identification method according to an embodiment of the present application;

[0018] Figure 2 is a flowchart of an event identification method according to Embodiment 1 of the present application;

[0019] Figure 3 is a flowchart of a traffic event detection according to an embodiment of the present application;

[0020] Figure 4 is a flowchart of an event identification method according to Embodiment 2 of the present application;

[0021] Figure 5 is a flowchart of an event identification method according to Embodiment 3 of the present application;

[0022] Figure 6 is a flowchart of an event identification method according to Embodiment 4 of the present application;

[0023] Figure 7 is a schematic diagram of an event recognition device according to an embodiment of the application;

[0024] Figure 8 is a schematic diagram of a traffic event recognition device according to an embodiment of the application;

[0025] Figure 9 is a schematic diagram of an event recognition device according to an embodiment of the application;

[0026] Figure 10 is a schematic diagram of an event recognition device according to an embodiment of the application;

[0027] Figure 11 is a schematic diagram of an event recognition system according to an embodiment of the application;

[0028] Figure 12 is a structural block diagram of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0029] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should fall within the protection scope of the present application.

[0030] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units need not be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to such a process, method, product or device.

[0031] The technical scheme provided in the application is mainly implemented by using a large model technology. The large model refers to a deep learning model with a large number of model parameters, and can generally include hundreds of millions, billions, tens of billions, hundreds of billions or even tens of trillions of model parameters. The large model can also be referred to as a foundation model. A large model is pre-trained through a large amount of unlabeled corpus, and a pre-training model with hundreds of millions of parameters is output. Such a model can adapt to a wide range of downstream tasks, and the model has good generalization ability. For example, a large language model (LLM) and a multi-modal pre-training model.

[0032] It should be noted that the large model can be fine-tuned through a small amount of samples in actual application, so that the large model can be applied to different tasks. For example, the large model can be widely applied to the fields of natural language processing (NLP), computer vision, speech processing and the like, and can be applied to computer vision field tasks such as visual question answering (VQA), image captioning (IC), image generation and the like, and can be widely applied to natural language processing field tasks such as text-based sentiment classification, text summarization generation, machine translation and the like. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design and the like. In the embodiments of the present application, the data processing through a multi-modal large model in a traffic scenario is taken as an example for explanation and description.

[0033] First, some nouns or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

[0034] Ball machine: refers to a camera capable of rotating in all directions, also known as a spherical camera or a gimbal camera. Such a camera can generally rotate horizontally and tilt vertically, and is usually equipped with zooming capability to monitor targets at a distance. These characteristics of the ball machine make it very useful when monitoring a large area.

[0035] Latitude and longitude: latitude and longitude are the combination of longitude and latitude to form a coordinate system, called a geographic coordinate system, which is a spherical coordinate system that uses a three-dimensional spherical surface to define the space on the earth. It can indicate any position on the earth.

[0036] Multimodal large model: refers to a large artificial intelligence model that can process and integrate multiple types of data such as text, images, audio, etc. In the field of deep learning and artificial intelligence, the "multimodal" feature of the model means that it can be designed to understand and combine information from multiple sensors or data sources, while "large model" usually refers to a model with a huge number of parameters, possibly reaching tens of billions or even hundreds of billions.

[0037] Object detection: The task of object detection is to find the target (object) of interest in the image and determine the category and location of the target, which is one of the core problems in the field of computer vision.

[0038] Target tracking: An important research direction in the field of computer vision, the purpose of which is to track the motion trajectory of one or more target objects in a video sequence.

[0039] Traffic event: A traffic event refers to any irregular event that occurs in a traffic system, which may affect traffic flow, safety or operational efficiency. Traffic events can include a variety of situations, from minor traffic violations to major traffic incidents. Such as traffic incidents, road construction, traffic congestion, vehicle breakdowns, illegal behavior, etc.

[0040] At present, by taking advantage of the feature that the collision of moving objects in a traffic accident will cause at least one moving object to have a local change in the direction of motion, by constructing a motion direction graph and calculating the energy of the motion direction graph, the energy fluctuations of the motion direction graph are monitored on the time axis to detect traffic accidents. This method can detect events, but it is difficult to subdivide the types of accidents. Another method mainly uses the trajectory of moving vehicles to detect traffic accidents, but the trajectory of moving vehicles is prone to be incomplete and intersected, and is not stable in complex traffic environments. This method uses speed, position change, and motion direction change to weight the detection, which is difficult to subdivide the event type, and the accident threshold is not easy to determine, which directly affects the detection effect.

[0041] Embodiment 1

[0042] According to the embodiments of the present application, an event recognition method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] Considering that the model parameters of a large model are huge, and the operation resources of a mobile terminal are limited, the above-mentioned event recognition method provided by the embodiments of the present application can be applied to the application scenario as shown in Figure 1 but not limited to this. Figure 1is a schematic diagram of an application scenario of an event recognition method according to an embodiment of the present application, in the application scenario as shown Figure 1 The large model is deployed in the server 10, the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client device 20 can include, but is not limited to, a smartphone, a tablet computer, a notebook computer, a palm computer, a personal computer, a smart home device, a vehicle-mounted device, etc. The client device 20 can interact with the user through a graphical user interface to call the large model, and then implement the method provided in the embodiments of the present application.

[0044] In the embodiments of the present application, the system composed of the client device and the server can perform the following steps: the client device performs the following steps: in the process of monitoring at least one moving object in a to-be-monitored area, if an event caused by a target moving object is monitored, an event video in the to-be-monitored area is acquired; the event contained in the event video is intercepted to obtain at least one event image containing the event; a multi-modal large model is used to recognize the event contained in the at least one event image to obtain an event type of the event. It should be noted that, in the case that the running resources of the client device can meet the deployment and running conditions of the large model, the embodiments of the present application can be performed in the client device.

[0045] In the above running environment, the present application provides an event recognition method as shown in Figure 2 Figure 2 is a flowchart of an event recognition method according to Embodiment 1 of the present application. As shown in Figure 2

[0046] Step S202, in the process of monitoring at least one moving object in a to-be-monitored area, if an event caused by a target moving object is monitored, an event video in the to-be-monitored area is acquired;

[0047] The event video in the to-be-detected area can be an event video in the event occurrence area.

[0048] The to-be-monitored area can be a traffic scene area, a sports scene area, or a news scene area that needs to be monitored. The type of the to-be-monitored area is not limited, and the to-be-monitored area can be any area that needs to be monitored.

[0049] ​​The at least one moving object can be an object moving in the to-be-monitored area, and if an event caused by the target moving object occurs, the event can be confirmed by acquiring an event video in the area where the event occurs. In a traffic scene area, the moving object can be a vehicle, in a sports scene area, the moving object can be a sports player or a sports article held by the sports player, and in a news scene area, the moving object can be an object generating news content or a reporter. The type of the moving object is not limited herein.

[0050] The event can be an event caused by collision, stillness, or execution of a preset operation of the target moving object. In a traffic scene area, the event can include, but is not limited to, a traffic accident, road construction, traffic congestion, vehicle failure, and emergency lane parking. In a sports scene area, the event can include, but is not limited to, a player scoring a goal, a ball falling, and a game instruction being issued. In a news scene area, the event can include, but is not limited to, a reporter introducing news content and chasing a news occurrence site. The type of the event is not limited herein. The type of the event can be adjusted according to a specific scene.

[0051] In an optional embodiment, the to-be-monitored area can be monitored by a camera first. If an event caused by the target moving object is detected, the target moving object can be tracked and detected to acquire an event video in the area where the event occurs. Optionally, the to-be-monitored area can be monitored by a gun-type camera first. If an event caused by the target moving object is detected, the gun-type camera can be controlled to shoot an event video by acquiring shooting information in the gun-type camera.

[0052] Optionally, the event video can be identified to determine whether the event video contains an event.

[0053] In an optional embodiment, an event detection module can be connected to the camera. The event detection module can identify the event video according to a pre-set event identification logic to determine whether the event video contains an event.

[0054] The pre-set event identification logic can be to detect whether a specific scene or a specific object is contained in the event video. The event identification logic can be set according to specific event identification requirements. The event identification logic is not limited herein.

[0055] In an optional embodiment, the event video can be identified by a neural network model to determine whether the event video contains an event. If the event video contains an event, the type of the event can be determined by a multi-modal large model to enrich information of the event report. If the event video does not contain an event, other event videos waiting to be identified can be identified, or a new event video can be waited for.

[0056] Step S204, the event contained in the event video is intercepted to obtain at least one event image containing the event;

[0057] In an optional embodiment, in the case that the event is contained in the event video, an image containing the vehicle can be intercepted. For example, whether the target moving object is contained in the video frame in the event video can be determined by recognizing the video frame. If the target moving object is contained in the video frame, the video frame containing the target moving object can be intercepted to obtain at least one event image. The event video can also be intercepted according to a pre-set interception interval to obtain at least one event image containing the event. The event video can also be intercepted according to a pre-set interception logic to obtain at least one event image containing the event.

[0058] In another optional embodiment, in the case that the event is contained in the event video, the event video can be detected first. The video frame containing the event detected can be intercepted to obtain at least one event image. The video frame not containing the event can be discarded, so that the at least one event image is the event image containing the event. Here, only an example is given, and whether the video frame not containing the event is discarded or the video frame containing the event is completely saved is not limited here. The interception mode of the event video can be determined according to actual needs.

[0059] Step S206, the event contained in the at least one event image is recognized by using a multi-modal large model to obtain an event type of the event.

[0060] The multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type of the event.

[0061] The description information described above can be text, voice, etc. The type of the description information is not limited here.

[0062] The multi-modal large model described above refers to a large deep learning model capable of processing multiple different modalities (such as text, image, voice, etc.) data. These models are usually based on deep neural networks, capable of simultaneously processing different modal input data and performing joint learning and representation learning, thereby improving the performance and generalization ability of the model. Multi-modal large models have a wide range of applications in natural language processing, computer vision, speech recognition, etc. and can handle complex data and tasks.

[0063] In an alternative embodiment, the multi-modal large model can be used to identify the event contained in at least one event image, so as to generate text data of the event in the event image through the text generation capability of the multi-modal large model, so as to determine the event type of the event by analyzing the text data. The event type represented by the text data corresponding to the event can be determined by comparing the text data with the pre-set text data corresponding to the event type, and the event type of the event can also be determined by analyzing the text data in an artificial intelligence manner.

[0064] In another alternative embodiment, the multi-modal large model can be used to identify the event contained in at least one event image, so as to generate voice data of the event in the event image through the multi-modal large model, so as to determine the event type of the event by analyzing the voice data. The event type represented by the voice data corresponding to the event can be determined by comparing the voice data with the pre-set voice data corresponding to the event type, and the event type of the event can also be determined by analyzing the voice data in an artificial intelligence manner.

[0065] In yet another alternative embodiment, the multi-modal large model can be used to identify the event contained in at least one event image, so as to generate image data of the event in the event image through the multi-modal large model, so as to determine the event type of the event by analyzing the image data. The event type represented by the image data corresponding to the event can be determined by comparing the image data with the pre-set image data corresponding to the event type, and the event type of the event can also be determined by analyzing the image data in an artificial intelligence manner.

[0066] In a traffic scene, the multi-modal large model can be applied to generate a detailed description of the traffic event, realize event type subdivision, and provide more detailed and rich information for event warning. With the increasing complexity of the traffic system, real-time monitoring and analysis of traffic events become more and more important. The pan-tilt camera in the security monitoring system is used to realize all-around monitoring, and the multi-modal large model can realize automatic detection, detailed description and classification of traffic events, thereby improving the efficiency and response speed of traffic management.

[0067] In a sports scene, the multi-modal large model can be applied to generate a detailed broadcast content of the sports event, thereby improving the broadcast effect of the sports event.

[0068] In a news scene, the multi-modal large model can be applied to generate a detailed broadcast content of the news scene, thereby improving the broadcast effect of the news scene.

[0069] By the above steps, in the process of monitoring at least one moving object in the to-be-monitored region, when an event caused by a target moving object is monitored, an event video in the to-be-monitored region is obtained; the event in the event video is intercepted to obtain at least one event image containing the event; and the event in the at least one event image is identified by using the multi-modal large model to obtain an event type of the event, thereby improving the identification effect of the event. It is easy to note that, in the process of monitoring the at least one moving object, if the target moving object causes the event to occur, the event video can be intercepted to obtain at least one event image, and the multi-modal large model can be used to obtain rich information related to the event by using the image-to-text capability of the multi-modal large model, thereby improving the classification effect of the event and obtaining an event type with higher accuracy, so as to provide more detailed and rich information for the alarm of the event, thereby improving the event identification effect and solving the technical problem of poor event identification effect in the related art.

[0070] In the above embodiments of the present application, the multi-modal large model is used to identify the event in the at least one event image to obtain an event type of the event, including: inputting the at least one event image into the multi-modal large model and obtaining text data output by the multi-modal large model, wherein the text data is used to describe the event in the at least one event image; determining the event type of the event based on the text data and a preset association relationship, wherein the preset association relationship is used to represent the association relationship between different event types and different text data.

[0071] The number of the above event images can be one or multiple, and the number of the event images can be defined in advance or determined according to the actual interception of the event video.

[0072] The text data described above can be data generated by describing the event in the event image in the form of text. The text data can be text used to describe a traffic event in a traffic scene, the text data can also be text used to describe a sports event in a sports scene, and the text data can also be text used to describe a news event in a news scene. The scene described by the text data is not limited, and the event in the scene described by the text data is also not specifically limited.

[0073] For example, when the text data describes a traffic event in a traffic scene, the specific content of the text data can be a collision between two vehicles at an intersection, a vehicle congestion caused by road construction, a vehicle failure and stop running, an emergency lane with a vehicle, and the like. This is only an example and is not specifically limited.

[0074] For example, when the text data describes a news event in a news scenario, the specific content of the text data can be the disaster situation of a fire in a fire news, or the situation of various regions in a global news. These are merely examples and are not limited in this regard.

[0075] For example, when the text data describes a sports event in a sports scenario, the specific content of the text data can be the state of each player in a sports match or the score situation in a sports match. These are merely examples and are not limited in this regard.

[0076] The preset association relationship described above can be a type classification rule of the event. By combining the event contained in the at least one event image in the text data with the preset association relationship, the event can be accurately classified according to the type, thereby obtaining the event type of the event.

[0077] The event type described above includes but is not limited to a traffic accident, road construction, traffic congestion, vehicle failure, and emergency lane parking. The type classification rule can be adjusted and determined according to the application scenario and the effect of the multi-modal large model.

[0078] Taking the event type of traffic congestion as an example, when the text data indicates that a lane is completely occupied, the number of vehicles in the same lane exceeds a certain threshold, most vehicles travel at a low speed, and most vehicles are in a stationary state, the event type can be determined as traffic congestion according to the preset association relationship. Taking the emergency lane as an example, when the text data indicates that the vehicle is in a stopped state and is in an emergency lane, the event type can be determined as traffic congestion according to the preset association relationship.

[0079] In an optional embodiment, the at least one event image can be input into the multi-modal large model, so as to identify the image in the at least one event image by the multi-modal large model, so as to identify the event data appearing in the at least one event image, and output the text data of the event data by the multi-modal model, thereby obtaining the text data output by the multi-modal large model.

[0080] In the above embodiments of the present application, the method further includes: obtaining a training sample, wherein the training sample includes a training image and a label text, the training image contains any one of a plurality of events caused by different moving objects, and the label text is used to describe any one of the events contained in the training image; and training the pre-trained large model using the training sample to obtain the multi-modal large model.

[0081] The training image described above can be a pre-collected image of any one of a plurality of events caused by different moving objects. The number of training images can be one or more, and the image of the training image is not limited in this regard.

[0082] The label text can be pre-labeled text data of the event in the training image.

[0083] The training image can be obtained by intercepting event videos in the area where the event occurs caused by the target moving object when the occurrence of the event is monitored. The training image can also be automatically generated by artificial intelligence. The generation of the training image is not limited herein.

[0084] The label text of the training image can be determined by manual labeling or intelligent labeling.

[0085] In an optional embodiment, a training sample can be obtained, and a training image in the training sample can be input into a pre-trained large model. The pre-trained large model can output a predicted label text. A loss function between the predicted label text and the label text can be constructed to adjust model parameters of the pre-trained large model, thereby obtaining a multi-modal large model.

[0086] In the above embodiments of the present application, when the occurrence of the event caused by the target moving object is monitored, the event video in the area to be monitored is obtained, including: when the occurrence of the event is monitored, the event information of the event that has been generated is obtained, wherein the event information at least includes: geographic coordinate information of the event, first identification information of the first camera that monitors the event, and position information of the event in the monitoring image, the monitoring image is photographed by the first camera; based on the event information, the control instruction of the second camera is generated, wherein the shooting range of the second camera is greater than that of the first camera, and the control instruction is used to control the second camera to shoot the area where the event occurs; the control instruction is sent to the second camera, and the event video photographed by the second camera according to the control instruction is received.

[0087] The event information can be obtained by alarm, or received by a terminal device or by other means. Optionally, the event detection module can be used to monitor the event, and when the occurrence of the event is monitored, the event information of the event can be obtained to control the camera to shoot the event.

[0088] The geographic coordinate information of the event can be the longitude and latitude coordinates of the event location, which can be determined by the global positioning system (GPS) or other positioning technologies to accurately locate the position of the event.

[0089] When the event occurs, the geographic coordinate information can help quickly locate the event location; the first identification information of the first camera that monitors the event can quickly find the relevant monitoring image; and the position information of the event in the monitoring image can quickly locate the specific position of the event, thereby obtaining relatively complete event information.

[0090] The first camera can be used to monitor the area within the monitoring range of the first camera, so as to determine whether an event occurs in the area, and if an event occurs, event information of the event can be generated.

[0091] The first identification information of the first camera can be a unique identifier of the first camera, which can be a specific number or name. The first identification information can be used to determine the area where the event occurs by locating the area where the first camera is located. The first identification information can also facilitate the operator of the monitoring system to quickly locate the monitoring screen to view the actual situation of the event.

[0092] The second camera can be a camera with a larger shooting range, which can realize long-distance monitoring. For example, a spherical camera, a gimbal camera, a ball camera, and a panoramic camera, which are not limited here and can be selected according to actual conditions.

[0093] In an optional embodiment, the first camera can be a gun camera, and the second camera can be a ball camera. The gun camera is usually long and thin in shape, suitable for being fixed on a wall or a ceiling, and usually has a longer observation distance and a narrower viewing angle, suitable for monitoring a long distance or a narrow area. The ball camera is usually spherical in shape and is usually installed on a support that can rotate and tilt. It has a wider viewing angle and a more flexible observation range, suitable for monitoring a large area or a place where the observation direction needs to be changed frequently. The ball camera also usually has a shielding function and can automatically adjust the position of the lens to avoid being blocked.

[0094] The control instruction can be an instruction for zooming in on the second camera, and the instruction for zooming in can be an instruction for zooming in on the event occurrence area for display. Zooming in on the camera refers to zooming in on a specific area for display by the camera, so as to more clearly observe the details of the area.

[0095] For example, if it is necessary to zoom in on the event occurrence area for display for event shooting, the second camera can automatically zoom in on the area for display, so as to more clearly capture the details of the area. In a traffic scene, the license plate number of a vehicle in the area, a landmark building in the area, and specific information of a road can be captured, which are not limited here.

[0096] In an optional embodiment, in the case of monitoring the occurrence of an event, the occurrence area of the event can be quickly located by obtaining the event information of the event, so as to generate a control instruction according to the event information, and send the control instruction to the second camera. The zoom-in function of the second camera can be used to display the occurrence area of the event in a zoomed-in manner, so as to provide a clearer view and more detailed information for video shooting of the occurrence area of the event, thereby obtaining a clearer event video containing more information.

[0097] In the above embodiments of the present application, the control instruction of the second camera is generated based on the event information, including: determining the second camera corresponding to the geographic coordinate information from a plurality of cameras, wherein different cameras correspond to different geographic coordinate information; determining the shooting parameter of the second camera based on the first identification information and the position information; and generating the control instruction in a preset format based on the shooting parameter and the second identification information of the second camera.

[0098] The plurality of cameras described above can be a plurality of cameras connected to the monitoring system, and the second camera corresponding to the geographic coordinate system in the plurality of cameras can be determined according to the geographic coordinate information in the event information. The different geographic coordinate information corresponding to different cameras described above can be pre-set according to the installation position of different cameras.

[0099] The shooting parameter described above can be a required function, such as zoom-in, zoom-out, center point of the zoom-in position, etc., which is not limited here.

[0100] The preset format described above can be a pre-set fixed format, and the shooting parameter and the second identification information of the second camera can be processed according to the preset format to generate the control instruction. For example, the address of the interface service, the unique identification of the camera, the required function (zoom-in or zoom-out), the center point of the zoom-in position, etc. can be input in the zoom-in interface, and the fixed format is generated according to the need.

[0101] In an optional embodiment, after the first camera monitors the occurrence of an event, the second camera belonging to the same area as the first camera can be called according to the geographic coordinate information of the first camera. The shooting parameter of the second camera, i.e. the zoom-in or zoom-out operation and the center point of the zoom-in position of the second camera, can be determined according to the first identification information and the position information of the event, so as to process the shooting parameter and the second identification information according to the preset format to generate the control instruction of the second camera.

[0102] In the above embodiments of the present application, the method further includes: performing target detection on the plurality of video frames of the event video to determine whether the plurality of video frames contain the target moving object corresponding to the event; in a case where at least one video frame of the plurality of video frames contains the target moving object, performing target tracking on the target moving object based on the at least one video frame to determine whether the event video contains the event; and in a case where none of the plurality of video frames contains the target moving object, determining that the event video does not contain the event.

[0103] Since the target moving object is the main leading object of the event, target detection can be performed on the plurality of video frames of the event video to determine whether the plurality of video frames contain the target moving object corresponding to the event. If the plurality of video frames contain the target moving object corresponding to the event, target tracking can be performed on the target moving object based on at least one video frame of the plurality of video frames to determine whether the event video contains the event.

[0104] In an optional embodiment, in a case where none of the plurality of video frames contains the target moving object, since the video frame does not contain the leading object of the event, the event video does not contain the event.

[0105] It should be noted that the at least one video frame and the at least one image are not completely the same. In a case where the original video contains the event, the at least one image is at least one video frame containing the event target.

[0106] In a traffic scene, after obtaining the event video, the event video can be preprocessed to determine whether the event video contains the event. Target detection can be performed on the plurality of video frames of the event video to determine whether the plurality of video frames contain a vehicle corresponding to the event. In a case where at least one video frame of the plurality of video frames contains the vehicle, tracking detection can be performed on the vehicle to determine whether the event video contains the event. In a case where none of the plurality of video frames contains the vehicle, it can be determined that the event video does not contain the event.

[0107] In the above embodiments of the present application, the method further includes: obtaining a monitoring video obtained by photographing a to-be-monitored region by a first camera; identifying at least one moving object in the monitoring video to determine whether the monitoring video contains an event caused by a target moving object; and in a case where the monitoring video contains the event, determining that the event occurs.

[0108] The identification of the at least one moving object in the monitoring video can be used to monitor whether the at least one moving object causes the event in real time, so as to determine whether the monitoring video contains the event caused by the target moving object.

[0109] In an optional embodiment, in order to improve the accuracy of monitoring events, the event monitoring module can acquire a monitoring video obtained by the first camera shooting the region to be monitored, identify the monitoring video to determine whether the monitoring video contains an event, and if the monitoring video contains an event, it can be determined that an event has occurred. The event monitoring module can access the second camera to monitor and track the target through the second camera, so as to further monitor the event and report the event video containing the event when the event video is acquired.

[0110] Optionally, the monitoring video can be identified by some configuration information and event judgment logic of the field to determine whether the monitoring video contains an event.

[0111] In the above embodiments of the application, the method further includes: in the case where the monitoring video contains an event, the monitoring video is intercepted to obtain at least one monitoring image containing the event; the monitoring image is identified to determine the geographic coordinate information of the event and the position information of the event in the monitoring image; and the event information of the event is generated based at least on the geographic coordinate information, the position information and the first identifier of the first camera.

[0112] In an optional embodiment, in the case where the monitoring video contains an event, the monitoring video can be continuously intercepted or intercepted according to a predetermined interception mode to obtain at least one monitoring image containing the event, and the interval time of the interception can be set in advance. The interception mode of the monitoring video is not limited here, and the interception mode can be determined according to the actual situation.

[0113] In another optional embodiment, in the case where the monitoring video contains an event, the monitoring video can be intercepted to obtain at least one monitoring image containing the event, and the monitoring image can be identified to determine the geographic coordinate information of the event and the position information of the event in the monitoring image by any one or more of the position information of the first camera, the object information in the monitoring image, the road information and the position information of the target moving object.

[0114] Figure 3 is a flowchart of a traffic event detection according to an embodiment of the application. As shown in Figure 3 , the method includes:

[0115] Step S301, in the process of monitoring at least one moving object in the region to be monitored, in the case where an event caused by a target moving object is monitored, the event information of the event is acquired;

[0116] The event information includes at least: geographic coordinate information of the event, first identification information of the first camera that monitors the event, and position information of the event in a monitoring image, the monitoring image being photographed by the first camera.

[0117] The position information includes, but is not limited to, coordinates of the vehicle in the picture, latitude and longitude, and the first identification information of the first camera can be a camera number.

[0118] In step S302, a zoom-in command of the second camera is generated based on the event information.

[0119] The second camera can be a spherical camera.

[0120] The zoom-in command can be obtained by processing the event information through certain rule logic, and the zoom-in operation of the second camera can be performed through the zoom-in command.

[0121] In step S303, target detection and tracking can be performed on the video stream output after zooming-in to determine whether the vehicle is in the picture, and if the vehicle is in the picture, the picture of the event area can be intercepted to obtain at least one event image.

[0122] It should be noted that if the vehicle is in the picture, the event area in the video stream is intercepted. If the vehicle is not in the picture, it is suspected that the vehicle has left or the zoom-in has a problem, and the operation is abandoned, and the next event is waited for.

[0123] In step S304, a multi-modal large model can be used to identify the event in the at least one event image to obtain an event type of the event.

[0124] The at least one event image can be input to the multi-modal large model to generate text data of the at least one event image, and the event type of the event can be determined according to the text data and a subdivision rule of the event type.

[0125] The event type can include, but is not limited to, a traffic accident, road construction, traffic congestion, vehicle failure, and emergency lane parking.

[0126] In the present application, event detection and reporting of event-related information can be completed by an event detection module, a spherical camera zoom-in instruction can be generated according to the event-related information, and the spherical camera can be called to zoom-in. By zooming-in the event area, more rich visual information is provided for subsequent modules. Traditional event detection only has structured information, and in the present application, the event picture after zooming-in of the spherical camera is input to a multi-modal large model to generate a detailed description and a subdivision type of the event, and more rich information is reported.

[0127] The application applies the graph generation capability of a multi-modal large model to generate a detailed description of an event, realizes event type subdivision function, and provides more detailed and rich information for event alarm. In the application, the gun camera, the ball camera and the multi-modal large model are combined, the ball camera is used to magnify the event area detected by the gun camera, the remote, all-around coverage can be realized, and clearer event area images than the gun camera can be provided, thereby ensuring the quality of event description generated by the multi-modal large model.

[0128] In the application, a related script program can be written by using a program (python) language to realize the alignment process, and a C / C++ language is used in project landing to realize the positioning function. However, no limitation is made, and different languages can be selected to realize the function of the application according to actual conditions.

[0129] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions, and provide corresponding operation entrances for users to choose authorization or refusal.

[0130] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the action sequence described, because according to the application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the application.

[0131] From the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and necessary general hardware platform, and of course, it can also be realized by hardware. Based on such understanding, the technical solutions of the application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk), and includes a plurality of instructions to make a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the method of each embodiment of the application.

[0132] Embodiment 2

[0133] According to the embodiments of the present application, a traffic event identification method is also provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from this.

[0134] Figure 4 is a flowchart of a traffic event identification method according to Embodiment 2 of the present application, as shown in Figure 4 the method comprises the following steps:

[0135] Step S402, in the process of monitoring at least one vehicle in the traffic area, if a traffic event caused by a target vehicle is monitored, an event video in the traffic area is acquired;

[0136] The traffic area described above can be a highway, an urban road, or a traffic control area, and the traffic area is not limited here.

[0137] The vehicle type described above can be a motor vehicle, a non-motor vehicle, etc., and the vehicle type is not limited here.

[0138] Step S404, the traffic event contained in the event video is intercepted to obtain at least one event image containing the traffic event;

[0139] Step S406, the traffic event contained in the at least one event image is identified by using a multi-modal large model to obtain a traffic event type of the traffic event.

[0140] The multi-modal large model is used to generate description information describing the traffic event, and the description information is used to determine the traffic event type of the traffic event.

[0141] Through the above steps, in the process of monitoring at least one vehicle in the traffic area, when it is monitored that a traffic event caused by a target vehicle occurs, an event video in the traffic area is acquired; the traffic event contained in the event video is intercepted to obtain at least one event image containing the traffic event; and the traffic event contained in the at least one event image is identified by using the multi-modal large model to obtain a traffic event type of the traffic event, thereby improving the identification effect of the event. It is easy to note that in the process of monitoring at least one moving object, if the target moving object causes an event to occur, the event video can be intercepted to obtain at least one event image, and the multi-modal large model can be used to obtain rich information related to the event by using the ability of the multi-modal large model to generate text from images, thereby improving the classification effect of the event and obtaining an event type with higher accuracy, so as to provide more detailed and rich information for the alarm of the event, thereby improving the event recognition effect and further solving the technical problem of poor event recognition effect in the related art.

[0142] In the above embodiments of the present application, the multi-modal large model is used to identify the traffic event contained in the at least one event image to obtain the event type of the traffic event, which includes: inputting the at least one event image into the multi-modal large model and acquiring text data output by the multi-modal large model, wherein the text data is used to describe the traffic event contained in the at least one event image; determining the traffic event type of the traffic event based on the text data and a preset association relationship, wherein the preset association relationship is used to represent the association relationship between different traffic event types and different text data.

[0143] In an optional embodiment, the at least one event image can be input into the multi-modal large model to identify the traffic event in the at least one event image by the multi-modal large model, so as to identify the traffic event appearing in the at least one event image, and output the text of the traffic event by the multi-modal model, thereby obtaining the text data output by the multi-modal large model.

[0144] Optionally, a training sample is acquired, wherein the training sample includes: a training image and a label text, the training image contains any one of a plurality of traffic events caused by different moving objects, and the label text is used to describe any one of the traffic events contained in the training image; and the pre-training large model is trained by using the training sample to obtain the multi-modal large model.

[0145] It should be noted that the preferred implementation schemes and embodiments involved in the above embodiments of the present application are the same as the schemes, application scenarios and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0146] Embodiment 3

[0147] According to the embodiments of the present application, an event identification method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from this.

[0148] Figure 5 is a flowchart of an event identification method according to Embodiment 3 of the present application, as shown in Figure 5 the method comprises the following steps:

[0149] Step S502, in the process of monitoring at least one moving object in the to-be-monitored area, if an event caused by a target moving object is monitored, event information of the event is displayed on an operation interface;

[0150] Step S504, in response to an input instruction acting on the operation interface, an event video in the area where the event occurs is displayed on the operation interface;

[0151] The operation interface described above can be used to display the event video, wherein the operation interface can contain a variety of different controls for the user to operate, so as to display the event video on the operation interface.

[0152] Step S506, in response to an identification instruction acting on the operation interface, an event type of the event is displayed on the operation interface.

[0153] Wherein, the event type of the event is obtained by identifying the event contained in at least one event image using a multi-modal large model, and the at least one event image is obtained by intercepting the event video under the condition that the event video contains the event.

[0154] The identification instruction described above can be an instruction generated by the user operating the controls on the operation interface when the user needs to identify the event video.

[0155] By the above steps, in the process of monitoring at least one moving object in the to-be-monitored region, in the case that an event caused by a target moving object is monitored to occur, event information of the event is displayed on the operation interface; in response to an input instruction acting on the operation interface, event video in the region where the event occurs is displayed on the operation interface; and in response to a recognition instruction acting on the operation interface, an event type of the event is displayed on the operation interface, wherein the event type of the event is obtained by recognizing the event contained in at least one event image by using the multi-modal large model, and the at least one event image is obtained by intercepting the event video in the case that the event video contains the event. The recognition effect of the event is improved. It is easy to note that, in the process of monitoring the at least one moving object, if the target moving object causes the event to occur, the event video can be intercepted to obtain at least one event image, the multi-modal large model can be used to obtain rich information related to the event by using the image-to-text capability of the multi-modal large model, so as to improve the classification effect of the event and obtain an event type with higher accuracy, so as to provide more detailed and rich information for the alarm of the event, thereby improving the event recognition effect and solving the technical problem of poor event recognition effect in the related art.

[0156] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0157] Embodiment 4

[0158] According to the embodiments of the present application, an event recognition method is also provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in different order.

[0159] Figure 6 is a flowchart of an event recognition method according to Embodiment 4 of the present application, as shown in Figure 6 the method comprises the following steps:

[0160] Step S602, in the process of monitoring at least one moving object in the to-be-monitored region, in the case that an event caused by a target moving object is monitored to occur, event video in the to-be-monitored region is obtained by calling a first interface;

[0161] The first interface includes a first parameter, and the parameter value of the first parameter includes the event video.

[0162] The first interface can be an interface for data interaction between the cloud server and the client. The client can input the event video into the interface function as a first parameter of the interface function, so as to upload the event video to the cloud server.

[0163] In step S604, the event in the event video is intercepted to obtain at least one event image containing the event.

[0164] In step S606, the event in the at least one event image is recognized by using the multi-modal large model to obtain an event type of the event.

[0165] The multi-modal large model is configured to generate description information of the event, and the description information is configured to determine the event type of the event.

[0166] In step S608, the event type is output by calling the second interface.

[0167] The second interface includes a second parameter, and a parameter value of the second parameter includes the event type.

[0168] The second interface can be an interface for data interaction between the cloud server and the client. The cloud server can input the event type into the interface function as a second parameter of the interface function, so as to realize the purpose of issuing the event type to the client.

[0169] Through the above steps, in the process of monitoring at least one moving object in the to-be-monitored region, when an event caused by a target moving object is monitored, the event video in the to-be-monitored region is obtained by calling the first interface, the first interface includes a first parameter, and a parameter value of the first parameter includes the event video; the event in the event video is intercepted to obtain at least one event image containing the event; the event in the at least one event image is recognized by using the multi-modal large model to obtain an event type of the event, wherein the multi-modal large model is configured to generate description information of the event, and the description information is configured to determine the event type of the event; and the event type is output by calling the second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the event type, thereby improving the recognition effect of the event. It is easy to note that, in the process of monitoring the at least one moving object, if the target moving object causes the event to occur, the event video can be intercepted to obtain at least one event image, and the multi-modal large model can be used to obtain rich information related to the event, thereby improving the classification effect of the event, obtaining an event type with higher accuracy, and providing more detailed and rich information for the alarm of the event, thereby improving the event recognition effect and solving the technical problem of poor event recognition effect in the related art.

[0170] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same application scenarios and implementation processes as the schemes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0171] Embodiment 5

[0172] According to the embodiments of the present application, an event identification device for implementing the above event identification method is further provided, Figure 7 is a schematic diagram of an event identification device according to Embodiment 5 of the present application, as Figure 7 shown, the device 700 includes an acquisition module 702, an interception module 704, and an identification module 706.

[0173] The acquisition module is configured to, in the process of monitoring at least one moving object in the to-be-monitored area, acquire an event video in the to-be-monitored area when an event caused by a target moving object is monitored; the interception module is configured to intercept an event contained in the event video to obtain at least one event image containing the event; and the identification module is configured to identify the event contained in the at least one event image by using a multi-modal large model to obtain an event type of the event, wherein the multi-modal large model is configured to generate description information describing the event, and the description information is configured to determine the event type of the event.

[0174] It should be noted that the acquisition module 702, the interception module 704, and the identification module 706 correspond to steps S202 to S206 in Embodiment 1, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above modules can also be run in the server 10 provided in Embodiment 1 as part of the device.

[0175] In the above embodiments of the present application, the identification module is further configured to input the at least one event image into the multi-modal large model and acquire text data output by the multi-modal large model, wherein the text data is used to describe the event contained in the at least one event image; and determine the event type of the event based on the text data and a preset association relationship, wherein the preset association relationship is used to represent the association relationship between different event types and different text data.

[0176] In the above embodiments of the present application, the device further includes a training module.

[0177] The acquisition module is further configured to acquire a training sample, where the training sample includes a training image and a label text, the training image contains any one of a plurality of events caused by different moving objects, and the label text is used to describe any one of the events contained in the training image; and the training module is configured to train the pre-trained large model by using the training sample to obtain the multi-modal large model.

[0178] In the above embodiments of the present application, the acquisition module is further configured to acquire event information of the event in a case where the occurrence of the event is monitored, where the event information at least includes geographic coordinate information of the event, first identification information of a first camera that monitors the event, and position information of the event in a monitoring image, the monitoring image being captured by the first camera; the control instruction of the second camera is generated based on the event information, where the shooting range of the second camera is greater than that of the first camera; the control instruction is sent to the second camera, and an event video captured by the second camera according to the control instruction is received.

[0179] In the above embodiments of the present application, the generation module is further configured to determine the second camera corresponding to the geographic coordinate information from a plurality of cameras, where different cameras correspond to different geographic coordinate information; determine the shooting parameter of the second camera based on the first identification information and the position information; and generate the control instruction in a preset format based on the shooting parameter and the second identification information of the second camera.

[0180] In the above embodiments of the present application, the device further includes a detection module, a tracking module, and a determination module.

[0181] The detection module is configured to perform target detection on a plurality of video frames of the event video to determine whether the target moving object corresponding to the event is contained in the plurality of video frames; the tracking module is further configured to perform target tracking on the target moving object based on at least one video frame of the plurality of video frames in a case where the target moving object is contained in the at least one video frame, to determine whether the event is contained in the event video; and the determination module is further configured to determine that the event is not contained in the event video in a case where the target moving object is not contained in the plurality of video frames.

[0182] In the above embodiments of the present application, the acquisition module is further configured to acquire a monitoring video obtained by the first camera capturing the to-be-monitored region; the recognition module is further configured to recognize at least one moving object in the monitoring video to determine whether the event caused by the target moving object is contained in the monitoring video; and the determination module is further configured to determine that the occurrence of the event is monitored in a case where the event is contained in the monitoring video.

[0183] In the above embodiments of the present application, the device further includes a generation module.

[0184] The interception module is further configured to intercept the monitoring video to obtain at least one monitoring image containing the event if the monitoring video contains the event; the identification module is configured to identify the monitoring image to determine geographical coordinate information of the event and position information of the event in the monitoring image; and the generation module is further configured to generate event information of the event based at least on the geographical coordinate information, the position information, and first identification information of the first camera.

[0185] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same application scenarios and implementation processes as the scheme provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0186] Embodiment 6

[0187] According to the embodiments of the present application, a traffic event identification device for implementing the above traffic event identification method is further provided, Figure 8 is a schematic diagram of a traffic event identification device according to Embodiment 6 of the present application, as Figure 8 shown, the device 800 includes an acquisition module 802, an interception module 804, and an identification module 806.

[0188] The acquisition module is configured to acquire event video in a traffic area in a process of monitoring at least one vehicle in the traffic area, if a traffic event caused by a target vehicle is monitored; the interception module is configured to intercept the traffic event contained in the event video to obtain at least one event image containing the traffic event; and the identification module is configured to identify the traffic event contained in the at least one event image by using a multi-modal large model to obtain a traffic event type of the traffic event, wherein the multi-modal large model is configured to generate description information describing the traffic event, and the description information is configured to determine the traffic event type of the traffic event.

[0189] It should be noted that the above acquisition module 802, interception module 804, and identification module 806 correspond to steps S402 to S406 in Embodiment 2, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be a part of the device and can run in the server 10 provided in Embodiment 1.

[0190] In the above embodiments of the present application, the identification module is further configured to input the at least one event image into the multi-modal large model, and obtain text data output by the multi-modal large model, wherein the text data is used to describe the traffic event contained in the at least one event image; and determine the traffic event type of the traffic event based on the text data and a preset association relationship, wherein the preset association relationship is used to represent the association relationship between different traffic event types and different text data.

[0191] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0192] Embodiment 7

[0193] According to the embodiments of the present application, an event identification device for implementing the above event identification method is further provided, Figure 9 is a schematic diagram of an event identification device according to Embodiment 7 of the present application, as Figure 9 shown, the device 900 includes a first display module 902, a second display module 904, and a third display module 906.

[0194] The first display module is configured to display event information of an event on an operation interface in a case where the event caused by a target mobile object is monitored in the process of monitoring at least one mobile object in a to-be-monitored area; the second display module is configured to display an event video in an area where the event occurs on the operation interface in response to an input instruction acting on the operation interface; and the third display module is configured to display an event type of the event on the operation interface in response to an identification instruction acting on the operation interface, wherein the event type of the event is obtained by identifying the event contained in at least one event image by using a multi-modal large model, and the at least one event image is obtained by intercepting the event video in a case where the event is contained in the event video.

[0195] It should be noted that the first display module 902, the second display module 904, and the third display module 906 correspond to steps S502 to S506 in Embodiment 3, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above modules can also be a part of the device and can run in the server 10 provided in Embodiment 1.

[0196] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0197] Embodiment 8

[0198] According to the embodiments of the present application, an event identification device is also provided for implementing the above-mentioned event identification method, Figure 10 is a schematic diagram of an event identification device according to Embodiment 8 of the present application, as Figure 10 shown, the device 1000 includes an acquisition module 1002, a determination module 1004, an identification module 1006, and an output module 1008.

[0199] The acquisition module is configured to, in the process of monitoring at least one moving object in a to-be-monitored area, acquire an event video in the to-be-monitored area by calling a first interface when an event caused by a target moving object is monitored, wherein the first interface includes a first parameter, and a parameter value of the first parameter includes the event video. The determination module is configured to intercept an event included in the event video to obtain at least one event image containing the event. The identification module is configured to identify the event contained in the at least one event image by using a multi-modal large model to obtain an event type of the event, wherein the multi-modal large model is configured to generate description information describing the event, and the description information is configured to determine the event type of the event. The output module is configured to output the event type by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the event type.

[0200] It should be noted that the acquisition module 1002, the determination module 1004, the identification module 1006, and the output module 1008 correspond to steps S602 to S608 in Embodiment 4, and the four modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above-mentioned modules can also be run in the server 10 provided in Embodiment 1 as a part of the device.

[0201] It should be noted that the preferred embodiments involved in the above-mentioned embodiments of the present application have the same application scenarios, implementation processes as the scheme provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0202] Embodiment 9

[0203] According to the embodiments of the present application, an event identification device is also provided for implementing the above-mentioned event identification method, Figure 11 is a schematic diagram of an event identification system according to Embodiment 9 of the present application, as Figure 11 shown, the system 1100 includes an event monitoring module 1102, an event tracking module 1104, and an event identification module 1106.

[0204] The event monitoring module is configured to monitor whether an event caused by a target mobile object occurs in the process of monitoring at least one mobile object in the to-be-monitored region; the event tracking module is configured to, in the case that the event monitoring module monitors that the event occurs, acquire an event video in the to-be-monitored region, intercept an event included in the event video, and obtain at least one event image including the event; and the event identification module is configured to identify the event included in the at least one event image by using a multi-modal large model, and obtain an event type of the event, wherein the multi-modal large model is configured to generate description information describing the event, and the description information is configured to determine the event type of the event.

[0205] In the above embodiments of the present application, the event tracking module includes: an instruction generation module configured to, in the case that the event occurs is monitored, acquire event information of the event, and generate a control instruction of a second camera based on the event information, wherein the event information at least includes geographic coordinate information of the event, first identification information of a first camera that monitors the event, and position information of the event in a monitoring image, the monitoring image is photographed by the first camera, and a photographing range of the second camera is greater than a photographing range of the first camera; an instruction execution module configured to send the control instruction to the second camera, and receive an event video photographed by the second camera according to the control instruction; and a monitoring tracking module configured to identify the event video, determine whether the event video includes the event, and in the case that the event video includes the event, intercept the event video to obtain at least one event image including the event.

[0206] The first camera described above can be a gun camera, and the second camera can be a ball camera, and the photographing range of the second camera is greater than the photographing range of the first camera.

[0207] It should be noted that the preferred implementation schemes and embodiments involved in the above embodiments of the present application have the same application scenarios and implementation processes as the schemes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0208] Embodiment 10

[0209] The embodiments of the present application can provide an electronic device, which can be any one of the electronic devices in the electronic device group. Alternatively, in the present embodiment, the above-mentioned electronic device can also be replaced by a terminal device such as a mobile terminal.

[0210] Alternatively, in the present embodiment, the above-mentioned electronic device can be located in at least one network device of a plurality of network devices of a computer network.

[0211] In the present embodiment, the above-mentioned computer terminal can execute program codes in the method.

[0212] Alternatively, Figure 12is a structural block diagram of an electronic device according to an embodiment of the present application. As shown in Figure 12 The electronic device A can include one or more (only one is shown in the figure) processors 102, a memory 104, a storage controller, and a peripheral interface, wherein the peripheral interface is connected with a radio frequency module, an audio module, and a display.

[0213] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the methods in the above embodiments. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal A through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0214] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: in the process of monitoring at least one moving object in a to-be-monitored area, if an event caused by a target moving object is monitored, an event video in the to-be-monitored area is acquired; an event included in the event video is intercepted to obtain at least one event image containing the event; a multi-modal large model is used to identify the event contained in the at least one event image to obtain an event type of the event, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type of the event.

[0215] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: in the process of monitoring at least one vehicle in a traffic area, if a traffic event caused by a target vehicle is monitored, an event video in the traffic area is acquired; a traffic event included in the event video is intercepted to obtain at least one event image containing the traffic event; a multi-modal large model is used to identify the traffic event contained in the at least one event image to obtain a traffic event type of the traffic event, wherein the multi-modal large model is used to generate description information describing the traffic event, and the description information is used to determine the traffic event type of the traffic event.

[0216] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: in the process of monitoring at least one moving object in a to-be-monitored area, in the case that an event caused by a target moving object is monitored to occur, displaying event information of the event on an operation interface; in response to an input instruction acting on the operation interface, displaying an event video in an area where the event occurs on the operation interface; and in response to a recognition instruction acting on the operation interface, displaying an event type of the event on the operation interface, wherein the event type of the event is obtained by recognizing the event contained in at least one event image by using a multi-modal large model, and the at least one event image is obtained by intercepting the event video in the case that the event video contains the event.

[0217] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: in the process of monitoring at least one moving object in a to-be-monitored area, in the case that an event caused by a target moving object is monitored to occur, obtaining an event video in the to-be-monitored area by calling a first interface, wherein the first interface includes a first parameter, and a parameter value of the first parameter includes the event video; intercepting the event contained in the event video to obtain at least one event image containing the event; recognizing the event contained in the at least one event image by using a multi-modal large model to obtain an event type of the event, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type of the event; and outputting the event type by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the event type.

[0218] By adopting the embodiments of the present application, in the process of monitoring at least one moving object in a to-be-monitored area, in the case that an event caused by a target moving object is monitored to occur, an event video in the to-be-monitored area is obtained; the event contained in the event video is intercepted to obtain at least one event image containing the event; and the event contained in the at least one event image is recognized by using a multi-modal large model to obtain an event type of the event, thereby improving the recognition effect of the event. It is easy to note that, in the process of monitoring at least one moving object, if the target moving object causes the event to occur, the event video can be intercepted to obtain at least one event image, and the multi-modal large model can be used to obtain rich information related to the event, thereby improving the classification effect of the event, obtaining an event type with higher accuracy, and providing more detailed and rich information for the alarm of the event, thereby improving the event recognition effect and solving the technical problem of poor event recognition effect in the related art.

[0219] Those skilled in the art can understand that, as Figure 12The structure shown is only schematic, and the electronic device can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, or the like. The figure does not limit the structure of the electronic device. For example, the electronic device A can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure.

[0220] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing the relevant hardware of the terminal device, and the program can be stored in a computer readable storage medium, which can include a flash disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, etc.

[0221] Embodiment 11

[0222] The embodiments of the present application also provide a computer readable storage medium. Optionally, in the embodiment, the above computer readable storage medium can be used to save the program code executed by the method provided by the above embodiments.

[0223] Optionally, in the embodiment, the above storage medium can be located in any one of the electronic devices in the group of electronic devices in the computer network, or in any one of the mobile terminals in the group of mobile terminals.

[0224] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for executing the following steps: in the process of monitoring at least one moving object in the to-be-monitored area, if an event caused by a target moving object is monitored, an event video in the to-be-monitored area is acquired; the event in the event video is intercepted to obtain at least one event image containing the event; the event in the at least one event image is identified by using a multi-modal large model to obtain an event type of the event, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type of the event.

[0225] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: in the process of monitoring at least one vehicle in the traffic area, if a traffic event caused by a target vehicle is monitored, an event video in the traffic area is acquired; the traffic event contained in the event video is intercepted to obtain at least one event image containing the traffic event; the traffic event contained in the at least one event image is identified by using a multi-modal large model to obtain a traffic event type of the traffic event, wherein the multi-modal large model is used to generate description information describing the traffic event, and the description information is used to determine the traffic event type of the traffic event.

[0226] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: in the process of monitoring at least one mobile object in the to-be-monitored area, if an event caused by a target mobile object is monitored, event information of the event is displayed on an operation interface; in response to an input instruction acting on the operation interface, an event video in an area where the event occurs is displayed on the operation interface; in response to an identification instruction acting on the operation interface, an event type of the event is displayed on the operation interface, wherein the event type of the event is obtained by identifying the event contained in at least one event image by using a multi-modal large model, and the at least one event image is obtained by intercepting the event video containing the event.

[0227] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: in the process of monitoring at least one mobile object in the to-be-monitored area, if an event caused by a target mobile object is monitored, an event video in the to-be-monitored area is acquired by calling a first interface, wherein the first interface includes a first parameter, and a parameter value of the first parameter includes the event video; the event contained in the event video is intercepted to obtain at least one event image containing the event; the event contained in the at least one event image is identified by using a multi-modal large model to obtain an event type of the event, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type of the event; the event type is output by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the event type.

[0228] In the process of monitoring at least one mobile object in the to-be-monitored region, if an event caused by the target mobile object is monitored, an event video in the to-be-monitored region is acquired; the event in the event video is intercepted to obtain at least one event image containing the event; and the event in the at least one event image is identified by using the multi-modal large model to obtain the event type of the event, thereby improving the identification effect of the event. It is easy to note that, in the process of monitoring the at least one mobile object, if the target mobile object causes an event, the event video can be intercepted to obtain at least one event image, and the multi-modal large model can be used to obtain rich information related to the event, thereby improving the classification effect of the event, obtaining an event type with higher accuracy, providing more detailed and rich information for the alarm of the event, improving the event identification effect, and thereby solving the technical problem of poor event identification effect in the related art.

[0229] Embodiment 12

[0230] The embodiment of the present application further provides a computer program product. Optionally, the computer program product can include a computer program, and the computer program is used to implement the method provided by the above embodiment when the computer program is executed by a processor.

[0231] Embodiment 13

[0232] The embodiment of the present application further provides a computer program product. Optionally, the computer program product can include a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium can be used to store a computer program. The computer program is used to implement the method provided by the above embodiment when the computer program is executed by a processor.

[0233] Embodiment 14

[0234] The embodiment of the present application further provides a computer program. Optionally, the computer program is used to implement the method provided by the above embodiment when the computer program is executed by a processor.

[0235] The above serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0236] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0237] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0238] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0239] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0240] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and various program codes that can be stored in the medium.

[0241] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. An event recognition method characterized by, The method comprises: In the process of monitoring at least one moving object in a to-be-monitored area, when an event caused by a target moving object is monitored, an event video in the to-be-monitored area is acquired; The event in the event video is intercepted to obtain at least one event image containing the event; A multi-modal large model is used to identify the event contained in the at least one event image, to obtain an event type of the event, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type of the event.

2. The method of claim 1, wherein, The method further comprises: Acquiring a training sample, wherein the training sample comprises a training image and a label text, the training image contains any one of a plurality of events caused by different moving objects, and the label text is used to describe the any one event contained in the training image; A pre-training large model is trained using the training sample to obtain the multi-modal large model.

3. The method of claim 2, wherein, The method further comprises: When the event occurs, event information of the event that has been generated is acquired, wherein the event information at least comprises geographic coordinate information of the event, first identification information of a first camera that monitors the event, and position information of the event in a monitoring image, and the monitoring image is photographed by the first camera; Based on the event information, a control instruction of a second camera is generated, wherein the photographing range of the second camera is greater than that of the first camera, and the control instruction is used to control the second camera to photograph the area where the event occurs; 4. The method of claim 1, wherein, The control instruction is sent to the second camera, and the event video photographed by the second camera according to the control instruction is received. The method further comprises: From a plurality of cameras, the second camera corresponding to the geographic coordinate information is determined, wherein different cameras correspond to different geographic coordinate information; Based on the first identification information and the position information, the photographing parameters of the second camera are determined; 5. The method of claim 4, wherein, Based on the photographing parameters and the second identification information of the second camera, the control instruction is generated in a preset format. The method further comprises: ​ ​ 6. The method of claim 1, wherein, ​ target detection is performed on a plurality of video frames of the event video to determine whether the plurality of video frames contain the target moving object corresponding to the event; In a case where at least one video frame of the plurality of video frames contains the target moving object, target tracking is performed on the target moving object based on the at least one video frame to determine that the event is contained in the event video; In a case where none of the plurality of video frames contains the target moving object, it is determined that the event is not contained in the event video.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: obtaining a monitoring video obtained by photographing the to-be-monitored region by the first camera; identifying the at least one moving object in the monitoring video to determine whether the event caused by the target moving object is contained in the monitoring video; In a case where the event is contained in the monitoring video, it is determined that the event is monitored.

8. The method of claim 7, wherein, The method further comprises: In a case where the event is contained in the monitoring video, the monitoring video is intercepted to obtain at least one monitoring image containing the event; identifying the monitoring image to determine geographic coordinate information of the event and position information of the event in the monitoring image; generating event information of the event based at least on the geographic coordinate information, the position information, and first identification information of the first camera.

9. A method of identifying a traffic incident, characterized by includes: In the process of monitoring at least one vehicle in a traffic region, in a case where a traffic event caused by a target vehicle is monitored, an event video in the traffic region is obtained; The traffic event contained in the event video is intercepted to obtain at least one event image containing the traffic event; The traffic event contained in the at least one event image is identified by using a multi-modal large model to obtain a traffic event type of the traffic event, wherein the multi-modal large model is used to generate description information describing the traffic event, and the description information is used to determine the traffic event type of the traffic event.

10. The method of claim 9, wherein, The traffic event contained in the at least one event image is identified by using a multi-modal large model to obtain a traffic event type of the traffic event, comprising: inputting the at least one event image into the multi-modal large model and obtaining text data output by the multi-modal large model, wherein the text data is used to describe the traffic event contained in the at least one event image; determine the traffic event type of the traffic event based on the text data and a preset association relationship, wherein the preset association relationship is used to represent the association relationship between different traffic event types and different text data.

11. An event recognition method characterized by, includes: In the process of monitoring at least one moving object in a to-be-monitored region, in a case where an event caused by a target moving object is monitored, an event video in the to-be-monitored region is obtained by calling a first interface, wherein the first interface comprises a first parameter, and a parameter value of the first parameter comprises the event video; The event contained in the event video is intercepted to obtain at least one event image containing the event; The event type is obtained by using a multi-modal large model to recognize the event contained in the at least one event image, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type. The event type is output by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the event type.

12. An event recognition system characterized by, The method comprises: An event monitoring module is configured to monitor whether an event caused by a target mobile object occurs during monitoring of at least one mobile object in a to-be-monitored area; An event tracking module is configured to, in a case where the event monitoring module monitors that the event occurs, acquire an event video in the to-be-monitored area, and capture the event contained in the event video to obtain at least one event image containing the event; An event recognition module is configured to use a multi-modal large model to recognize the event contained in the at least one event image to obtain an event type of the event, wherein the multi-modal large model is used to generate description information describing the event, and the description information is used to determine the event type.

13. The system of claim 12, wherein, The event tracking module comprises: An instruction generation module is configured to, in a case where the event occurs is monitored, acquire event information of the event, and generate a control instruction of a second camera based on the event information, wherein the event information at least includes geographic coordinate information of the event, first identification information of a first camera that monitors the event, and position information of the event in a monitoring image, the monitoring image is captured by the first camera, and a capturing range of the second camera is greater than a capturing range of the first camera; An instruction execution module is configured to send the control instruction to the second camera, and receive the event video captured by the second camera according to the control instruction; A monitoring tracking module is configured to recognize the event video, determine whether the event video contains the event, and, in a case where the event video contains the event, capture the event video to obtain at least one event image containing the event.

14. An electronic device, comprising: The method comprises: A memory stores an executable program; A processor is configured to run the program, wherein the program performs the method of any one of claims 1 to 11 when the program is run.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein the computer-readable storage medium controls a device where the computer-readable storage medium is located to perform the method of any one of claims 1 to 11 when the executable program is run.

16. A computer program product, characterised in that, The computer program is executed by a processor to implement the method of any one of claims 1 to 11.

Images