Abnormal event detection method and system based on large model, electronic equipment and medium

Abstract

The invention relates to the field of anomaly detection, and provides an abnormal event detection method and system based on a large model, electronic equipment and a medium, and the method comprises the steps: employing a visual large model to process video content, and obtaining an event description text T = {t1, t2,..., tn}; preprocessing the event description text to obtain an entity set Ej, tagging part-of-speech for the text in the entity set, and obtaining a part-of-speech set Pj; an event is structured based on an entity set, a part-of-speech set and dependency syntactic analysis to serve as a triple, and a triple set of the event is constructed by analyzing named entities and part-of-speech analysis. When the method is used, the automatic marking function of the abnormal events is continuously optimized through the self-learning ability of the model, and the intelligent level of the system is improved. The novel video analysis technology is expected to play an important role in improving the response speed and accuracy of a monitoring system, and provides technical support for video analysis in other fields.

Description

Technical Field

[0001] The present invention relates to the field of anomaly detection, and more specifically, to an abnormal event detection method, system, electronic device and medium based on a large model. Background Art

[0002] With the rapid development of deep learning technology, using large models (such as Gemini) for video content understanding has become a hot research direction. These models can automatically extract meaningful features from large amounts of video data for identifying and classifying key events. In this context, key event extraction and anomaly detection in videos have become an important research direction. It has a wide range of applications, including surveillance systems, autonomous driving, sports event analysis, etc. Through the efficient processing capabilities of large models, the accuracy and efficiency of event recognition can be significantly improved.

[0003] In some known technical solutions, abnormal events are identified using manual or rule-based methods.

[0004] During manual identification, staff members stare at the monitoring screen for a long time, which can easily lead to visual fatigue and miss some key abnormal events. Secondly, it is inefficient. Humans cannot efficiently observe multiple video images at the same time, resulting in delayed information processing. Humans are highly subjective and may make different judgments due to personal experience, emotions and other factors. Moreover, manual identification makes it difficult to trace back and accurately analyze events.

[0005] The rule-based approach is difficult to adapt to changes in complex scenes. Changes in lighting (such as sunrise and sunset, cloud cover) and weather (such as rain, snow, and fog) will interfere with the judgment of such methods and lead to incorrect judgments. The detection of changes in object shape is not flexible enough and can only detect anomalies but cannot judge specific events.

[0006] The feature extraction capability of deep learning-based methods is limited. Traditional CNNs mainly focus on local feature extraction, such as extracting information such as the edges and textures of objects in video frames to judge behavior. However, for complex abnormal behaviors involving scene semantic understanding and long sequence action associations, feature extraction may not be comprehensive enough; insufficient use of contextual information, CNNs are usually based on frame analysis, and when processing video sequences, the order of actions and the contextual association of scenes are not accurate enough. The generalization ability is not high. Such methods are usually trained for specific scenarios, and performance is prone to degradation when scenes outside of training appear.

[0007] To this end, we propose an abnormal event detection method, system, electronic device and medium based on a large model and a preparation method thereof to solve the above problems. Summary of the invention

[0008] Technical issues to be solved

[0009] In view of the problems existing in the prior art, the purpose of the present invention is to provide an abnormal event detection method, system, electronic device and medium based on a large model, which solves the problem of complex identification of abnormal events.

[0010] Technical Solution

[0011] To solve the above problems, the present invention adopts the following technical solutions.

[0012] A method for detecting abnormal events based on a large model, comprising: using a large visual model to process video content, and obtaining an event description text T = {t1, t2, ..., t n}; Preprocess the event description text to obtain the entity set E j , and annotate the text in the entity set with parts of speech, and obtain the part of speech set P j ; Based on entity set, part of speech set and dependency syntactic analysis, the event is structured as a triple, and the triple set of events is constructed by analyzing named entities and part of speech analysis; the triple sets corresponding to the description texts of all events are merged to obtain a structured event list Use the pre-trained word vector model to convert the triples into vectors of fixed dimensions; prepare in advance a set of seed word vectors representing abnormal concepts; calculate the cosine similarity between the word vector in the event and the seed word vector; and determine whether there is an abnormal event based on the cosine similarity threshold θ.

[0013] In a new embodiment, the processing of video content using a large visual model includes:

[0014] Use Gaussian filter to denoise video images;

[0015] Adjust brightness and contrast, and resize to a uniform size;

[0016] Image segmentation technology is used to remove irrelevant background and highlight the foreground.

[0017] In a new embodiment, the preprocessing of the event description text includes:

[0018] Split the description text T into several phrases and filter out common words that are not important information;

[0019] Use recognition models to identify named entities in text;

[0020] After describing and identifying each event, we get the entity set E j .

[0021] In a new embodiment, the triplet includes:

[0022] The subject of the event, the person or object in the named entity;

[0023] The action or state of an event predicte, using a verb or adjective;

[0024] The object of the event object;

[0025] Among them, the triple set is: d j ={(s j1 ,p j1 ,o j1 ),(s j2 ,p j2 ,o j2 ),...,(s jk ,p jk ,o jk )};

[0026] (s jl ,p jl ,o jl ) indicates that in the text t j The lth triplet constructed.

[0027] In a new embodiment, the converting of the triples into vectors of fixed dimensions using the pre-trained word vector model includes:

[0028] The pre-trained word vector model uses Word2Vec;

[0029] For the input event triple (s jl ,p jl ,o jl );

[0030] Map the word vectors (s jl ',p jl ',o jl ')=f Word2Vec (s jl ,p jl ,o jl );

[0031] where s jl ' is the subject vector, p jl ' is the predicate vector, o jl ' is a vector of objects.

[0032] In a new embodiment, the seed word vector set representing abnormal concepts prepared in advance includes:

[0033] Seed word vector set A = {a1, a2, ..., a m}, where a i is the i-th abnormal seed word vector;

[0034] And the seed word vector is obtained through the pre-trained word vector model.

[0035] In a new embodiment, the cosine similarity is:

[0036] Among them, v1·v2 is the dot product of two vectors, |v1| and |v2| are the moduli of the two vectors respectively;

[0037] And use this to get sim(s jl ',a i )、sim(p jl ',a i )、sim(o jl ',a i );

[0038] The determining whether there is an abnormal event based on the cosine similarity threshold θ includes:

[0039] If there exists i such that sim(s jl ',a i )>θorsim(p jl ',a i )>θorsim(o jl ',a i )>θ, then event t is determined j There is an exception in .

[0040] An abnormal event detection system based on a large model, comprising:

[0041] The visual processing module is used to process the video content using a large visual model to obtain an event description text T = {t1, t2, ..., t n};

[0042] The text preprocessing module is used to preprocess the event description text to obtain the entity set E j , and annotate the text in the entity set with parts of speech, and obtain the part of speech set P j ;

[0043] The structuring module is used to structure the event as a triple based on the entity set, part-of-speech set and dependency syntax analysis, and to construct the triple set of the event by analyzing the named entity and part-of-speech analysis;

[0044] The merging module is used to merge the triple sets corresponding to the description texts of all events to obtain a structured event list

[0045] A conversion module, which is used to convert triples into vectors of fixed dimensions using a pre-trained word vector model;

[0046] A seed word vector module is used to prepare a set of seed word vectors representing abnormal concepts in advance;

[0047] The calculation module is used to calculate the cosine similarity between the word vector in the event and the seed word vector;

[0048] The judgment module is used to judge whether there is an abnormal event based on the cosine similarity threshold θ.

[0049] An electronic device, comprising:

[0050] one or more processors, memory, and displays;

[0051] The memory is used to store computer program code, wherein the computer program code includes computer instructions;

[0052] When one or more of the processors execute the computer instructions, the electronic device performs the above-mentioned large model-based abnormal event detection method.

[0053] A computer scale storage medium is used to store a computer program. When the computer program is executed, it is used to implement the above-mentioned abnormal event detection method based on a large model.

[0054] Beneficial effects: Compared with the prior art, the advantages of the present invention are:

[0055] 1. It solves the problem that traditional CNNs cannot fully understand complex semantics and contextual information. The large model can provide a more advanced semantic understanding and abstract representation of video events. It can capture complex situational information, relationships between objects, and contextual information of events in the video, rather than just being limited to local features of the image. It has stronger generalization capabilities, can adapt to different scenes and different types of video data, and can also have better detection capabilities for newly emerging abnormal situations.

[0056] 2. Solve the problem that traditional CNN has low generalization ability and is difficult to adapt to changes in complex scenarios. The large model and the semantic extraction model can be used as an organic whole. The former is responsible for extracting a wide range of events, while the latter performs targeted anomaly screening based on text data. When the scene changes, only the semantic extraction model needs to be adjusted. Compared with image data, the storage and computing costs are lower, and mature natural language processing technology can be used for analysis, which improves the efficiency of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a schematic diagram of Embodiment 1 of the present invention;

[0058] Figure 2 It is a schematic diagram of the second embodiment of the present invention. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments, and all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0060] Embodiment 1

[0061] The embodiments of the present application solve the problem of... by providing a method for detecting abnormal events based on a large model. When used, the large model and semantic extraction model technology can process a large amount of video data, significantly reducing the need for manual monitoring, thereby improving efficiency. Specifically, these technologies can accurately identify important events and abnormal situations in videos, and maintain a high recognition rate even in complex scenarios. In addition, the semantic extraction model can adjust the data set in time for different scenarios, so that it can adapt to dynamic environmental changes, thereby providing more efficient and stable performance.

[0062] The innovation of this study is to combine the powerful feature extraction capabilities of large models to develop an automated video event analysis method. This can not only identify regular events, but also detect and mark abnormal events, reducing the need for manual intervention.

[0063] In addition, the automatic marking function of abnormal events is continuously optimized through the self-learning ability of the model, which improves the intelligence level of the system. This new video analysis technology is expected to play an important role in improving the response speed and accuracy of monitoring systems, and also provide technical support for video analysis in other fields.

[0064] See also Figure 1 , a method for detecting abnormal events based on a large model, comprising S100, using a large visual model to process video content, and obtaining an event description text T = {t1, t2, ..., t n};. The visual big model receives the video V as input. The visual big model is a model based on deep learning. By learning a large amount of image data, it can extract various features in the image and understand and describe the image content.

[0065] The processing process of the visual big model is represented by function G, that is, for the input video V, after being processed by the visual big model, a series of event description texts T = {t1, t2, ..., t n}, where t j Represents the jth event description text. It can be expressed by the formula: T = G (V).

[0066] S200, preprocess the event description text to obtain the entity set Ej and annotate the text in the entity set with part-of-speech tags to obtain a part-of-speech set P j ;

[0067] First, preprocess the event description text. For the obtained event description text set T, split T into individual phrases. When splitting, use the forward maximum matching method for splitting, W = Tokenize(T). Also, filter out common words that do not carry important information. Important information includes named entities, such as person names, place names, and organization names, actions, time dates, and quantities. In "Apple Inc. released the iPhone", "Apple Inc.", "iPhone", and "released" are important information, while "le" is unimportant information. In "The meeting was held in Shanghai on November 10, 2024", "November 10, 2024", "Shanghai", and "held" are important information; in "House prices increased by 10%", "house prices", "increased", and "10%" are important information. Use the NER model to identify named entities in the text, use the Transformers library of Hugging Face to identify proper nouns, such as person names, place names, and organization names; use SpaCy for Chinese part-of-speech tagging to extract verbs; use the SpaCy library to extract tags containing time and quantity.

[0068] For each event description w j , after named entity recognition, obtain a set of entity sets E j . Named entity recognition (NER) obtains a set of entity sets by identifying proper nouns in the text, such as person names, place names, organization names, dates, etc. This set contains the text of the entity and the corresponding category tags. The entity set is usually a list containing tuples, and each tuple contains: entity text: the actual string of the entity in the text. entity category: the category to which the entity belongs, such as PERSON (person name), ORG (organization), GPE (place name), DATE (date), etc. The implementation methods include SpaCy and Transformers tools.

[0069] Next, use part-of-speech tagging (POS tagging) technology to determine the part-of-speech of each word in the text to better understand the grammatical structure of the text. For each event description text w j , after part-of-speech tagging, obtain a set of part-of-speech sets P j .

[0070] S300, structure the event based on the entity set, part-of-speech set, and dependency syntactic analysis as a triple, and construct a triple set of the event through analyzing named entities and part-of-speech analysis.

[0071] 1. Preprocess the text

[0072] Segment text into phrases and identify named entities through NER to identify people, places, organizations, etc.

[0073] 2. Part-of-speech tagging and dependency parsing

[0074] Mark the part of speech of each word, identify verbs, nouns, etc., and analyze the dependency relationship between words to determine the subject, predicate, object, etc.

[0075] 3. Construct event triples

[0076] Subject: The subject of the sentence is determined through dependency relationships, usually a noun or pronoun. Predicate: Verbs are identified through part-of-speech tagging, indicating the action or state of an event. Object: The object of the sentence is determined through dependency relationships, usually a noun. Attach time and quantity information to the predicate.

[0077] For example, if you input "Steve Jobs founded Apple in California in 2024 with 200 participants," the output is [('Steve Jobs', 'Founded (2024, 200)', 'Apple')].

[0078] S400: Merge the triple sets corresponding to the description texts of all events to obtain a structured event list.

[0079] S500, using a pre-trained word vector model to convert the triples into vectors of fixed dimensions; S600, preparing in advance a set of seed word vectors representing abnormal concepts; S700, calculating the cosine similarity between the word vector in the event and the seed word vector; S800, judging whether there is an abnormal event based on the cosine similarity threshold θ.

[0080] Using a large visual model to process video content includes:

[0081] Use Gaussian filter to denoise video images;

[0082] Adjust brightness and contrast, and resize to a uniform size;

[0083] Image segmentation technology is used to remove irrelevant background and highlight the foreground.

[0084] First, a Gaussian filter is used to denoise the image. Then the brightness and contrast are adjusted to improve the image quality. Then the image is resized to a uniform size to fit the model input. Finally, the image segmentation technology GrabCut algorithm is used to remove irrelevant background and highlight the foreground.

[0085] The event description text preprocessing includes:

[0086] Split the description text T into several phrases and filter out common words that are not important information;

[0087] Use recognition models to identify named entities in text;

[0088] After describing and identifying each event, we get the entity set E j .

[0089] The triples include:

[0090] The subject of the event, the person or object in the named entity;

[0091] The action or state of an event predicte, using a verb or adjective;

[0092] The object of the event object;

[0093] Among them, the triple set is: d j ={(s j1 ,p j1 ,o j1 ),(s j2 ,p j2 ,o j2 ),...,(s jk ,p jk ,o jk )}; where D represents the set of event triples formed by all events from 1 to m, and dj represents the j-th event triple set.

[0094] (s jl ,p jl ,o jl ) indicates that in the text t j The lth triplet constructed.

[0095] Using the pre-trained word vector model to convert triples into vectors of fixed dimension includes:

[0096] The pre-trained word vector model uses Word2Vec;

[0097] For the input event triple (s jl ,p jl ,o jl );

[0098] Map the word vectors (s jl ',p jl ',o jl ')=f Word2Vec (s jl ,p jl ,o jl );

[0099] where s jl ' is the subject vector, p jl' is the predicate vector, o jl ' is a vector of objects.

[0100] The seed word vector set representing abnormal concepts prepared in advance includes:

[0101] Seed word vector set A = {a1, a2, ..., a m}, where a i is the i-th abnormal seed word vector;

[0102] And the seed word vector is obtained through the pre-trained word vector model.

[0103] Prepare a set of seed word vectors representing abnormal concepts A = {a1, a2, ..., a m}, where a i is the i-th abnormal seed word vector, which are words related to abnormal events, such as "fire" and "explosion". The seed word vector can be obtained through the same pre-trained word vector model.

[0104] The cosine similarity is:

[0105] Among them, v1·v2 is the dot product of two vectors, |v1| and |v2| are the moduli of the two vectors respectively;

[0106] And use this to get sim(s jl ',a i )、sim(p jl ',a i )、sim(o jl ',a i ); where sim(sjl',ai) is the cosine similarity between the subject vector sjl' and the abnormal concept seed word vector ai; sim(pjl',ai) is the cosine similarity between the predicate vector sjl' and the abnormal concept seed word vector ai; sim(ojl',ai) is the cosine similarity between the object vector and the abnormal concept seed word vector ai;

[0107] Judging whether there is an abnormal event based on the cosine similarity threshold θ includes:

[0108] If there exists i such that sim(s jl ',a i )>θorsim(p jl ',a i )>θorsim(o jl ',a i )>θ, then event t is determined j There is an exception in .

[0109] By experimenting and adjusting on the training dataset, a suitable cosine similarity threshold θ is determined. jl ', predicate vector p jl ', object vector o jl ', if there exists i such that sim(s jl ',a i )>θorsim(p jl ',a i )>θorsim(o jl ',a i )>θ, then event t is determined j There is an exception in .

[0110] Embodiment 2

[0111] See also Figure 2 , an abnormal event detection system based on a large model, comprising:

[0112] The visual processing module is used to process the video content using a large visual model to obtain an event description text T = {t1, t2, ..., t n}; First, use a Gaussian filter to denoise the image. Then adjust the brightness and contrast to improve the image quality. Then resize the image to a uniform size to fit the model input. Finally, use the image segmentation technology GrabCut algorithm to remove irrelevant background and highlight the foreground.

[0113] The video is input to the visual big model, and the visual big model receives the video V as input. The visual big model is a model based on deep learning. By learning a large amount of image data, it can extract various features in the image and understand and describe the image content.

[0114] The processing process of the visual big model is represented by function G, that is, for the input video V, after being processed by the visual big model, a series of event description texts T = {t1, t2, ..., t n}, where t j Represents the jth event description text. It can be expressed by the formula: T = G (V).

[0115] The text preprocessing module is used to preprocess the event description text to obtain the entity set E j , and annotate the text in the entity set with parts of speech, and obtain the part of speech set P j ;

[0116] The structuring module is used to structure the event as a triple based on the entity set, part-of-speech set and dependency syntax analysis, and to construct the triple set of the event by analyzing the named entity and part-of-speech analysis;

[0117] The merging module is used to merge the triple sets corresponding to the description texts of all events to obtain a structured event list

[0118] First, preprocess the event description text. For the event description text set T, segment T into phrases, W = Tokenize (T). And filter out common words that do not carry important information. Use the NER model to identify named entities in the text, for each event description w j After named entity recognition, we get a set of entity sets E j Next, we use POS tagging technology to determine the part of speech of each word in the text in order to better understand the grammatical structure of the text. j After part-of-speech tagging, we get a set of part-of-speech sets P j .

[0119] Based on named entity E j , part-of-speech information P j The event is structured by using dependency parsing and represented as a triple (subject, predicate, object), where subject represents the subject of the event (a person or object in a named entity), predict represents the action or state of the event (a verb or adjective), and object represents the object of the event. j , by analyzing named entities and part-of-speech information, construct the event triple set d j ={(s j1 ,p j1 ,o j1 ),(s j2 ,p j2 ,o j2 ),...,(s jk ,p jk ,o jk )}, where (s jl ,p jl ,o jl ) indicates that in the text t j Finally, the event triples corresponding to all event description texts are merged to obtain a structured event list.

[0120] The conversion module is used to convert the triples into fixed-dimensional vectors using the pre-trained word vector model; the pre-trained word vector model Word2Vec is used to convert the triples in the event group into a fixed-dimensional vector. jl ,pjl ,o jl ), respectively mapping to obtain word vectors (s jl ',p jl ',o jl ')=f Word2Vec (s jl ,p jl ,o jl ), where s jl ' is the subject vector, p jl ' is the predicate vector, o jl ' is a vector of objects.

[0121] The seed word vector module is used to prepare a set of seed word vectors representing abnormal concepts in advance; prepare a set of seed word vectors representing abnormal concepts A = {a1, a2, ..., a m}, where a i is the i-th abnormal seed word vector, which are words related to abnormal events, such as "fire" and "explosion". The seed word vector can be obtained through the same pre-trained word vector model.

[0122] The calculation module is used to calculate the cosine similarity between the word vector in the event and the seed word vector; for the word vector in event E (s jl ',p jl ',o jl '), calculate it and each vector a in the seed word vector A respectively i Cosine similarity. Cosine similarity is an indicator that measures the similarity between two vectors in direction. Where v1·v2 is the dot product of two vectors, and |v1| and |v2| are the moduli of the two vectors. This method can be used to obtain sim(s jl ',a i ), sim(p jl ',a i ) and sim(o jl ',a i ).

[0123] The judgment module is used to judge whether there is an abnormal event based on the cosine similarity threshold θ. By conducting experiments and adjustments on the training data set, a suitable cosine similarity threshold θ is determined. jl ', predicate vector p jl ', object vector o jl ', if there exists i such that sim(s jl ',a i )>θorsim(p jl ',a i )>θorsim(o jl ',a i)>θ, then event t is determined j There is an exception in .

[0124] Embodiment 3

[0125] An electronic device, comprising:

[0126] one or more processors, memory, and displays;

[0127] The memory is used to store computer program codes, the computer program codes including computer instructions;

[0128] When one or more processors execute computer instructions, the electronic device performs the above-mentioned large model-based abnormal event detection method.

[0129] Embodiment 4

[0130] A computer scale storage medium is used to store a computer program. When the computer program is executed, it is used to implement the above-mentioned abnormal event detection method based on a large model.

[0131] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products, electronic devices. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0136] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0137] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0138] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0139] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0140] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting abnormal events based on a large model, characterized in that: include: The visual big model is used to process the video content and obtain the event description text T = {t1, t2, ..., t n }; Preprocess the event description text to obtain the entity set E j , and annotate the text in the entity set with parts of speech, and obtain the part of speech set P j ; The event is structured as a triple based on entity set, part-of-speech set and dependency syntax analysis, and the triple set of the event is constructed by analyzing named entities and part-of-speech analysis; Merge the triple sets corresponding to the description texts of all events to obtain a structured event list Use the pre-trained word vector model to convert the triples into vectors of fixed dimension; Prepare a set of seed word vectors representing abnormal concepts in advance; Calculate the cosine similarity between the word vector in the event and the seed word vector; Based on the cosine similarity threshold θ, it is determined whether there is an abnormal event.

2. The method for detecting abnormal events based on a large model as claimed in claim 1, characterized in that: The use of a large visual model to process video content includes: Use Gaussian filter to denoise video images; Adjust brightness and contrast, and resize to a uniform size; Image segmentation technology is used to remove irrelevant background and highlight the foreground.

3. The method for detecting abnormal events based on a large model as claimed in claim 1, characterized in that: The event description text preprocessing includes: Split the description text T into several phrases and filter out common words that are not important information; Use recognition models to identify named entities in text; After describing and identifying each event, we get the entity set E j .

4. The method for detecting abnormal events based on a large model according to claim 1, characterized in that: The triplet includes: The subject of the event, the person or object in the named entity; The action or state of an event predicte, using a verb or adjective; The object of the event object; Among them, the triple set is: d j ={(s j1 ,p j1 ,o j1 ),(s j2 ,p j2 ,o j2 ),...,(s jk ,p jk ,o jk )}; (s jl ,p jl ,o jl ) indicates that in the text t j The lth triplet constructed.

5. The method for detecting abnormal events based on a large model as claimed in claim 4, characterized in that: ； The method of converting a triple into a vector of fixed dimension by using a pre-trained word vector model includes: The pre-trained word vector model uses Word2Vec; For the input event triple (s jl ,p jl ,o jl ); Map the word vectors (s jl ',p jl ',o jl ')=f Word2Vec (s jl ,p jl ,o jl ); where s jl ' is the subject vector, p jl ' is the predicate vector, o jl ' is a vector of objects.

6. The method for detecting abnormal events based on a large model as claimed in claim 5, characterized in that: The seed word vector set representing abnormal concepts prepared in advance includes: Seed word vector set A = {a1, a2, ..., a m }, where a i is the i-th abnormal seed word vector; And the seed word vector is obtained through the pre-trained word vector model.

7. The method for detecting abnormal events based on a large model as claimed in claim 6, characterized in that: The cosine similarity is: Among them, v1·v2 is the dot product of two vectors, v1 and v2 are the moduli of the two vectors respectively; And use this to get sim(s jl ',a i )、sim(p jl ',a i )、sim(o jl ',a i ); The determining whether there is an abnormal event based on the cosine similarity threshold θ includes: If there exists i such that sim(s jl ',a i )>θorsim(p jl ',a i )>θorsim(o jl ',a i )>θ, then event t is determined j There is an exception in .

8. An abnormal event detection system based on a large model, characterized in that: include: The visual processing module is used to process the video content using a large visual model to obtain an event description text T = {t1, t2, ..., t n }; The text preprocessing module is used to preprocess the event description text to obtain the entity set E j , and annotate the text in the entity set with parts of speech, and obtain the part of speech set P j ; The structuring module is used to structure the event as a triple based on the entity set, part-of-speech set and dependency syntactic analysis, and to construct the triple set of the event by analyzing the named entity and part-of-speech analysis; The merging module is used to merge the triple sets corresponding to the description texts of all events to obtain a structured event list A conversion module, which is used to convert triples into vectors of fixed dimensions using a pre-trained word vector model; A seed word vector module is used to prepare a set of seed word vectors representing abnormal concepts in advance; The calculation module is used to calculate the cosine similarity between the word vector in the event and the seed word vector; The judgment module is used to judge whether there is an abnormal event based on the cosine similarity threshold θ.

9. An electronic device, characterized in that: include: one or more processors, memory, and displays; The memory is used to store computer program code, wherein the computer program code includes computer instructions; When one or more of the processors execute the computer instructions, the electronic device executes the large model-based abnormal event detection method described in any one of claims 1-7.

10. A computer scale storage medium, characterized in that: Used to store a computer program, which, when executed, is used to implement the large model-based abnormal event detection method as described in any one of claims 1 to 7.

Images