Abnormal behavior detection methods, devices and electronic equipment

Abstract

This disclosure provides a method, apparatus, and electronic device for abnormal behavior detection, relating to the field of behavior detection technology. The method includes: acquiring a monitoring dataset, which includes monitoring data collected using non-fixed cameras; detecting target features from the monitoring dataset using a pre-trained small target detection model, and labeling monitoring data containing target features as target images, where target features include any one or more of personnel, vehicles, and vessels; identifying key visual features from the target images using a pre-trained large multimodal model, and converting the key visual features into text data; comparing the text data with a pre-trained large language model against a preset scene knowledge base to determine whether the text data represents abnormal behavior data and whether to trigger an alarm. This disclosure can comprehensively extract key visual features, improve the recognition rate of key visual features, and accurately detect abnormal behavior.

Description

Technical Field

[0001] This disclosure relates to the field of behavior detection technology, specifically to an abnormal behavior detection method, device, and electronic equipment. Background Technology

[0002] In scenarios where non-fixed cameras are used to monitor abnormal behavior in important areas, in order to meet the needs of manual duty, non-fixed cameras are mainly PTZ cameras, whose viewing angle and focal length often change. The abnormal behaviors monitored in important areas are often sporadic, and the amount of sample data containing abnormal behaviors is small. It is difficult to obtain a large number of labeled samples for model training. The related technologies have insufficient generalization ability in the few-sample learning scenario, resulting in a low recognition rate of abnormal behaviors and an inability to accurately detect abnormal behaviors.

[0003] There is currently no effective technical solution to the problem that related technologies have insufficient generalization ability in few-sample learning scenarios, low recognition rate of abnormal behavior, and inability to accurately detect abnormal behavior. Summary of the Invention

[0004] The main purpose of this disclosure is to provide an abnormal behavior detection method, device, and electronic device to solve the problems of insufficient generalization ability of related technologies in few-sample learning scenarios, low recognition rate of abnormal behavior, and inability to accurately detect abnormal behavior.

[0005] To achieve the above objectives, a first aspect of this disclosure provides an abnormal behavior detection method, comprising:

[0006] Obtain a monitoring dataset, wherein the monitoring dataset includes monitoring data collected using non-fixed cameras;

[0007] The target features are detected from the monitoring dataset using a pre-trained target detection mini-model, and the monitoring data containing the target features are labeled as target images. The pre-trained target detection mini-model is a pre-trained target detection mini-model, and the target features include any one or more of people, vehicles, and ships.

[0008] Key visual features are identified from the target image using a pre-trained multimodal large model, and the key visual features are converted into text data. The pre-trained multimodal large model is a multimodal large model that has been pre-trained.

[0009] The text data is compared with a preset scenario knowledge base using a pre-trained large language model to determine whether the text data is abnormal behavior data and whether to trigger an alarm. The pre-trained large language model is a large language model that has been pre-trained.

[0010] Optionally, obtaining the monitoring dataset includes:

[0011] In a preset scenario, non-fixed cameras are used to collect monitoring data, and each image in the monitoring data is labeled.

[0012] Multi-source heterogeneous data is fused into the monitoring data, wherein the multi-source heterogeneous data includes data collected by radar, sound sensors or vibration sensors.

[0013] Optionally, the pre-trained target detection mini-model is a target detection algorithm based on convolutional neural networks, including any one or more of single-stage detectors and YOLO series models.

[0014] Optionally, the key visual features include the target features and environmental features;

[0015] The step of identifying key visual features from the target image using a pre-trained multimodal large model and converting the key visual features into text data includes:

[0016] Key visual features and environmental features are identified from the target image using a pre-trained multimodal large model. The pre-trained multimodal large model includes any one or more of CLIP, Visual-BERT, OFA model, RWKV series model, and VisualRWKV model. The environmental features include any one or more of barbed wire, river, fence, and sentry post.

[0017] By using pre-set questions, the spatiotemporal correlation between the target features and environmental features is determined;

[0018] The target features, environmental features, and spatiotemporal relationships are converted into text data.

[0019] Optionally, the step of comparing the text data with a preset scenario knowledge base using a pre-trained large language model to determine whether the text data is abnormal behavior data and whether to trigger an alarm includes:

[0020] In a pre-trained large language model, prompt words are optimized, and the text data is compared with a pre-set scene knowledge base to determine whether the text data conforms to the abnormal behavior of scene control. The pre-trained large language model includes any one or more of ChatGPT, BERT, and DeepSeek, and the scene knowledge base includes scene control rules and knowledge. The abnormal behavior includes any one or more of crossing the border, wading across the boundary, approaching the barbed wire fence, and climbing the barbed wire fence.

[0021] If the text data matches the abnormal behavior of the scenario control, then the text data is abnormal behavior data, triggering an alarm.

[0022] Furthermore, if the text data does not conform to the abnormal behavior of the scenario control, then the text data is not abnormal behavior data and no alarm is triggered.

[0023] Optionally, the method further includes:

[0024] Based on the pre-trained small object detection model, the pre-trained large multimodal model, and the pre-trained large language model, a three-level fusion architecture is established, wherein the pre-trained small object detection model is the bottom layer of the three-level fusion architecture, the pre-trained large multimodal model is the middle layer of the three-level fusion architecture, and the pre-trained large language model is the top layer of the three-level fusion architecture.

[0025] A second aspect of this disclosure provides an anomalous behavior detection device, comprising:

[0026] An acquisition unit is used to acquire a monitoring dataset, wherein the monitoring dataset includes monitoring data collected using non-fixed cameras;

[0027] The detection unit is used to detect target features from the monitoring dataset using a pre-trained target detection mini-model, and to mark the monitoring data containing the target features as target images. The pre-trained target detection mini-model is a pre-trained target detection mini-model, and the target features include any one or more of people, vehicles, and ships.

[0028] The recognition unit is used to identify key visual features from the target image using a pre-trained multimodal large model and convert the key visual features into text data, wherein the pre-trained multimodal large model is a pre-trained multimodal large model;

[0029] The comparison unit is used to compare the text data with a preset scenario knowledge base using a pre-trained large language model, determine whether the text data is abnormal behavior data, and determine whether to trigger an alarm. The pre-trained large language model is a large language model that has been pre-trained.

[0030] A third aspect of this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the abnormal behavior detection method provided in any of the first aspects.

[0031] A fourth aspect of this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the abnormal behavior detection method provided in any of the first aspects.

[0032] In the abnormal behavior detection method provided in this disclosure embodiment, a monitoring dataset is acquired, which includes monitoring data collected using non-fixed cameras; target features are detected from the monitoring dataset using a pre-trained target detection mini-model, and monitoring data containing the target features are marked as target images. The pre-trained target detection mini-model is a pre-trained target detection mini-model, and the target features include any one or more of personnel, vehicles, and vessels. Due to the fast processing capability of the pre-trained target detection mini-model, monitoring data containing target features can be quickly identified from massive amounts of data in a short time.

[0033] Key visual features are identified from the target image using a pre-trained multimodal large model, and these key visual features are converted into text data. The pre-trained multimodal large model is a pre-trained multimodal large model. The powerful generalization ability of the pre-trained multimodal large model reduces the dependence on target and abnormal behavior samples, thus solving the problem of limited samples in control scenarios. Furthermore, the use of multimodal fusion comprehensively extracts key visual features, improving the recognition rate of key visual features and solving the problem of difficulty in identifying special targets.

[0034] The text data is compared with a pre-trained large language model to determine whether it represents abnormal behavior and whether an alarm should be triggered. The pre-trained large language model is a pre-trained large language model. Leveraging its powerful semantic understanding and logical reasoning capabilities, the pre-trained large language model can accurately determine whether an alarm needs to be triggered, thus solving the problems of insufficient generalization ability, low recognition rate of abnormal behavior, and inability to accurately detect abnormal behavior in scenarios with few samples. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a schematic flowchart of the abnormal behavior detection method provided in the embodiments of this disclosure;

[0037] Figure 2 This is a block diagram of an abnormal behavior detection device provided in an embodiment of the present disclosure;

[0038] Figure 3A block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0039] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0042] In scenarios where non-fixed cameras are used to monitor abnormal behavior in important areas such as borders, in order to meet the needs of manual duty, non-fixed cameras are mainly PTZ cameras, whose viewing angle and focal length frequently change. Abnormal behaviors monitored in important areas are often sporadic, and the amount of sample data containing abnormal behaviors is small. It is difficult to obtain a large number of labeled samples for model training. Related technologies have insufficient generalization ability in few-sample learning scenarios, low recognition rate of abnormal behaviors, and inability to accurately detect abnormal behaviors. They face complex challenges such as dynamic scene adaptation, special target detection, and few-sample anomaly recognition.

[0043] To address the aforementioned problems, this disclosure provides an abnormal behavior detection method, such as... Figure 1 As shown, the method includes the following steps S11 to S14:

[0044] Step S11: Obtain the monitoring dataset, which includes monitoring data collected using non-fixed cameras; there are a large number of non-fixed cameras in important areas, which can continuously generate a large amount of monitoring data.

[0045] In one optional embodiment of this disclosure, step S11 includes:

[0046] The system collects monitoring data using non-fixed cameras in a preset scenario and labels each image in the monitoring data. The preset scenario can be a pre-set control scenario. For each image obtained by non-fixed cameras in the control scenario, the system can label the target category, precise location, and whether there is any abnormal behavior in each image. The target category can be people, vehicles, ships, etc.

[0047] Multi-source heterogeneous data is fused into monitoring data, including data collected by radar, sound sensors, or vibration sensors. This multi-source heterogeneous data fusion can be extended to the fusion processing of data from radar, sound sensors, vibration sensors, and other sources.

[0048] Step S12: Use a pre-trained target detection small model to detect target features from the monitoring dataset, and label the monitoring data containing target features as target images. The pre-trained target detection small model is a pre-trained target detection small model. Target features include any one or more of people, vehicles, and ships. The pre-trained target detection small model has already been pre-trained and does not need to undergo a training process again. The target image can be an alarm image or an alarm video.

[0049] This disclosure utilizes the rapid processing capabilities of a pre-trained target detection small model to quickly identify monitoring data containing target features from massive amounts of data in a short period of time.

[0050] In one optional embodiment of this disclosure, the pre-trained object detection mini-model in step S12 is an object detection algorithm based on a convolutional neural network, including any one or more of a single-stage detector and the YOLO series models.

[0051] Pre-trained small object detection models include Single Shot MultiBox Detector (SSD) and any one or more models from the YOLO series, such as the YOLO v8 model.

[0052] By using a pre-trained target detection mini-model, a preliminary and rapid screening of massive amounts of surveillance data can be performed. Due to the large number of non-fixed cameras in important areas, a large amount of surveillance data is continuously generated. The rapid processing capability of the pre-trained target detection mini-model can identify surveillance data containing target features from massive amounts of data in a short time, so as to narrow down the scope when processing image data later.

[0053] Step S13: Use a pre-trained multimodal large model to identify key visual features from the target image and convert these features into text data. The pre-trained multimodal large model is a pre-trained multimodal large model. Since it is already pre-trained, no further training is required. The pre-trained multimodal large model possesses powerful cross-modal information fusion capabilities, deep semantic understanding capabilities, and logical reasoning capabilities. It can simultaneously process multiple types of data, such as images, videos, and audio, fully exploring the complementary information between different modalities. This allows for a more comprehensive and accurate understanding of border surveillance scenarios. Through the pre-trained multimodal large model, real-time perception, precise analysis, and intelligent decision-making in complex scenarios in important areas can be achieved, effectively improving the efficiency and security of control.

[0054] This disclosure leverages the powerful generalization ability of pre-trained multimodal large models to reduce reliance on target and abnormal behavior samples. It eliminates the need for extensive sample annotation and enables abnormal behavior detection of special targets such as barbed wire in the presence of zero or few samples, thus solving the problem of limited and scarce samples in control scenarios. Furthermore, the use of multimodal fusion comprehensively extracts key visual features, improving the recognition rate of key visual features and addressing the difficulty in identifying special targets such as barbed wire.

[0055] In one optional embodiment of this disclosure, key visual features include target features and environmental features; key visual features may be the physical characteristics of a person, clothing characteristics, and items carried.

[0056] Step S13 includes:

[0057] This method utilizes pre-trained multimodal large models to identify target features and environmental features from target images. The pre-trained multimodal large models include any one or more of CLIP (Contrastive Language-Image Pretraining), Visual-BERT, OFA (One-For-All) models, RWKV (Recurrent Weighted Key-Value) series models, and Visual RWKV models. Target features include any one or more of people, vehicles, and ships, while environmental features include any one or more of barbed wire, rivers, fences, and sentry posts. For example, the pre-trained multimodal large model could be a 6B model of Visual RWKV 6.0.

[0058] By pre-setting questions, the spatiotemporal relationship between target features and environmental features can be determined; by pre-setting questions for the machine, using human-computer question-and-answer, human-computer interaction, and text-based question-and-answer methods, the spatiotemporal relationship between target features and environmental features can be determined; the spatiotemporal relationship can be such as people approaching barbed wire, people climbing barbed wire, vehicles approaching fences, and boats crossing rivers.

[0059] Convert target features, environmental features, and spatiotemporal relationships into text data.

[0060] Pre-trained multimodal large models can convert image data into text data and extract key visual features from image data. For example, for an image containing a suspicious person, a pre-trained multimodal large model can identify the person's appearance, clothing characteristics, and items being carried, and convert these key visual features into text data, providing a more easily understood and processed data format for subsequent analysis.

[0061] Step S14: Use a pre-trained Large Language Model (LLM) to compare the text data with a preset scenario knowledge base to determine whether the text data represents abnormal behavior and whether to trigger an alarm. The pre-trained LLM is a pre-trained large language model. Abnormal behavior data can include people wading through water, crossing borders, wading across boundaries, people lingering abnormally near barbed wire fences, vehicles approaching barbed wire fences, smuggling, etc. The pre-trained LLM is already pre-trained and does not require further training. The pre-trained LLM can be continuously optimized through prompts to become a scenario-fine-tuned LLM. Through fine-tuning of the controlled scenario, the pre-trained LLM possesses powerful semantic understanding and logical reasoning capabilities, enabling it to determine whether to trigger an alarm based on the input text data and the rules and knowledge of the controlled scenario.

[0062] This disclosure leverages the powerful semantic understanding and logical reasoning capabilities of a pre-trained large language model to accurately determine whether an alarm needs to be triggered. This addresses the issues of insufficient generalization ability, low recognition rate of abnormal behavior, and inability to accurately detect abnormal behavior in related technologies under few-sample learning scenarios.

[0063] In one optional embodiment of this disclosure, step S14 includes:

[0064] The prompt words are optimized in the pre-trained large language model. The text data is compared with the pre-set scene knowledge base to determine whether the text data conforms to the abnormal behavior of scene control. The pre-trained large language model includes any one or more of ChatGPT, BERT and DeepSeek. The scene knowledge base includes the rules and knowledge of scene control. Abnormal behavior includes any one or more of crossing the border, wading across the border, approaching the barbed wire fence and climbing the barbed wire fence. Abnormal behavior also includes people staying abnormally near the barbed wire fence, vehicles approaching the barbed wire fence, smuggling, etc.

[0065] If the text data matches the abnormal behavior of the scenario control, then the text data is abnormal behavior data and an alarm is triggered.

[0066] In one optional implementation of this disclosure, if the text data does not conform to the abnormal behavior of the scene control, then the text data is not abnormal behavior data and no alarm is triggered.

[0067] For example, when a pre-trained large language model receives text data described by a pre-trained multimodal large model, such as "a person not wearing a uniform is loitering near border protection facilities, carrying a suspected package," the pre-trained large language model, with continuous optimization based on manually provided prompts, can comprehensively analyze this information to determine whether this behavior conforms to abnormal behavior in border control, and thus decide whether to issue an alert.

[0068] In one optional embodiment of this disclosure, the method further includes:

[0069] A three-level fusion architecture is established based on a pre-trained small object detection model, a pre-trained large multimodal model, and a pre-trained large language model. The pre-trained small object detection model forms the bottom layer, the pre-trained large multimodal model forms the middle layer, and the pre-trained large language model forms the top layer. This three-level fusion architecture or framework is then used to accurately detect abnormal behaviors in controlled scenarios.

[0070] This disclosure overcomes related technical bottlenecks through cross-level technological collaboration. The first-level detection layer uses a lightweight pre-trained target detection model to quickly identify people, vehicles, and ships. The second-level multimodal understanding layer uses a pre-trained multimodal large model to perform semantic understanding of target images, supplementing information on surrounding facilities and the environment. The third-level inference layer constructs a scene-specific logical knowledge base and uses a pre-trained large language model to analyze the spatiotemporal relationship between targets and scenes, enabling intelligent decision-making in complex scenarios, such as boundary crossing risk assessment, analysis of abnormal behaviors like people wading in water, and climbing barbed wire. This three-level architecture is progressive, and the collaboration of different models at multiple levels improves the accuracy and reliability of alarms.

[0071] As can be seen from the above description, this disclosure achieves the following technical effects:

[0072] This disclosure utilizes the rapid processing capabilities of a pre-trained target detection small model to quickly identify surveillance data containing target features from massive amounts of data in a short period of time.

[0073] By leveraging the powerful generalization capabilities of pre-trained multimodal large models, the dependence on target and abnormal behavior samples is reduced, thereby solving the problem of insufficient samples in control scenarios. Furthermore, the use of multimodal fusion can comprehensively extract key visual features, improve the recognition rate of key visual features, and solve the problem of difficulty in identifying special targets.

[0074] By leveraging the powerful semantic understanding and logical reasoning capabilities of pre-trained large language models, it is possible to accurately determine whether an alarm needs to be triggered. This solves the problem that related technologies have insufficient generalization ability in few-sample learning scenarios, low recognition rate of abnormal behavior, and inability to accurately detect abnormal behavior.

[0075] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0076] This disclosure also provides an abnormal behavior detection device for implementing the above method embodiments, such as... Figure 2 As shown, the abnormal behavior detection device 20 includes:

[0077] Acquisition unit 21 is used to acquire a monitoring dataset, wherein the monitoring dataset includes monitoring data collected using non-fixed cameras;

[0078] The detection unit 22 is used to detect target features from the monitoring dataset using a pre-trained target detection small model, and to label the monitoring data containing target features as target images. The pre-trained target detection small model is a pre-trained target detection small model, and the target features include any one or more of people, vehicles, and ships.

[0079] The recognition unit 23 is used to identify key visual features from the target image using a pre-trained multimodal large model and convert the key visual features into text data. The pre-trained multimodal large model is a multimodal large model that has been pre-trained.

[0080] The comparison unit 24 is used to compare text data with a preset scene knowledge base using a pre-trained large language model to determine whether the text data is abnormal behavior data and whether to trigger an alarm. The pre-trained large language model is a large language model that has been pre-trained.

[0081] The specific methods of execution of each unit in the above device embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0082] This disclosure also provides an electronic device, such as... Figure 3 As shown, the electronic device includes one or more processors 31 and a memory 32. Figure 3 Take a processor 31 as an example.

[0083] The controller may also include an input device 33 and an output device 34.

[0084] The processor 31, memory 32, input device 33, and output device 34 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0085] Processor 31 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips. The general-purpose processor can be a microprocessor or any conventional processor.

[0086] The memory 32, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the control method in this embodiment. The processor 31 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 32, thereby implementing the abnormal behavior detection method of the above-described method embodiment.

[0087] The memory 32 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the processing device operated by the server. Furthermore, the memory 32 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 32 may optionally include memory remotely located relative to the processor 31, and these remote memories can be connected to a network connection device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0088] Input device 33 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the server's processing device. Output device 34 may include display devices such as a display screen.

[0089] One or more modules are stored in memory 32, and when executed by one or more processors 31, they perform actions such as... Figure 1 The method shown.

[0090] Those skilled in the art will understand that all or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes as described in the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory (FM), hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0091] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An abnormal behavior detection method characterized by, The method comprises the following steps: acquiring a monitoring data set, wherein the monitoring data set comprises monitoring data collected using a non-fixed camera; detecting target features from the monitoring data set using a pre-trained target detection small model, and marking monitoring data containing the target features as target images, wherein the pre-trained target detection small model is a pre-trained target detection small model, and the target features include any one or more of personnel, vehicles, and ships; identifying key visual features from the target images using a pre-trained multi-modal large model, and converting the key visual features into text data, wherein the pre-trained multi-modal large model is a pre-trained multi-modal large model; comparing the text data with a preset scene knowledge base using a pre-trained large language model to determine whether the text data is abnormal behavior data and whether an alarm should be triggered, wherein the pre-trained large language model is a pre-trained large language model; wherein the key visual features include the target features and environmental features; wherein the step of identifying key visual features from the target images using a pre-trained multi-modal large model and converting the key visual features into text data comprises: identifying target features and environmental features from the target images using a pre-trained multi-modal large model, wherein the pre-trained multi-modal large model includes any one or more of CLIP, Visual-BERT, OFA model, RWKV series model, and Visual RWKV model, and the environmental features include any one or more of barbed wire, river, fence, and sentry post; determining the spatio-temporal correlation between the target features and environmental features through a preset question; converting the target features, environmental features, and spatio-temporal correlation into text data.

2. The method of claim 1, wherein, The step of acquiring a monitoring data set comprises: collecting monitoring data using a non-fixed camera in a preset scene, and labeling each image in the monitoring data; fusing multi-source heterogeneous data in the monitoring data, wherein the multi-source heterogeneous data includes data collected by a radar, a sound sensor, or a vibration sensor.

3. The method of claim 1, wherein, The pre-trained target detection small model is a target detection algorithm based on a convolutional neural network, including any one or more of a single-stage detector and a YOLO series model.

4. The method of claim 1, wherein, The step of comparing the text data with a preset scene knowledge base using a pre-trained large language model to determine whether the text data is abnormal behavior data and whether an alarm should be triggered comprises: performing prompt word optimization in a pre-trained large language model to compare the text data with a pre-set scene knowledge base and determine whether the text data is an abnormal behavior that violates scene control, wherein the pre-trained large language model includes any one or more of ChatGPT, BERT, and DeepSeek, the scene knowledge base includes rules and knowledge for scene control, and the abnormal behavior includes any one or more of crossing a border, crossing a border by wading, approaching a barbed wire, and climbing a barbed wire. If the text data matches the abnormal behavior controlled by the scene, the text data is abnormal behavior data, and an alarm is triggered.

5. The method of claim 4, wherein, If the text data does not match the abnormal behavior controlled by the scene, the text data is not abnormal behavior data, and no alarm is triggered.

6. The method of claim 1, wherein, The method further comprises: According to the pre-training target detection small model, the pre-training multi-modal large model and the pre-training large language model, a three-level fusion architecture is established, wherein the pre-training target detection small model is the bottom layer of the three-level fusion architecture, the pre-training multi-modal large model is the middle layer of the three-level fusion architecture, and the pre-training large language model is the top layer of the three-level fusion architecture.

7. An abnormal behavior detection apparatus characterized by comprising: Comprise: An acquisition unit is configured to acquire a monitoring data set, wherein the monitoring data set comprises monitoring data collected using a non-fixed camera; A detection unit is configured to detect target features from the monitoring data set using a pre-trained target detection small model, and mark monitoring data containing the target features as target images, wherein the pre-trained target detection small model is a pre-trained target detection small model, and the target features include any one or more of personnel, vehicles and ships; An identification unit is configured to identify key visual features from the target images using a pre-trained multi-modal large model, and convert the key visual features into text data, wherein the pre-trained multi-modal large model is a pre-trained multi-modal large model; A comparison unit is configured to compare the text data with a preset scene knowledge base using a pre-trained large language model, determine whether the text data is abnormal behavior data, and determine whether to trigger an alarm, wherein the pre-trained large language model is a pre-trained large language model; The key visual features include the target features and environmental features. The pre-trained multi-modal large model is used to identify key visual features from the target images, and convert the key visual features into text data, comprising: The pre-trained multi-modal large model is used to identify target features and environmental features from the target images, wherein the pre-trained multi-modal large model includes any one or more of CLIP, Visual-BERT, OFA model, RWKV series model, Visual RWKV model, and the environmental features include any one or more of barbed wire, river, fence and sentry post; A preset question is used to determine the spatio-temporal correlation between the target features and the environmental features; The target features, environmental features and spatio-temporal correlation are converted into text data.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a computer to execute the abnormal behavior detection method of any one of claims 1 to 6.

9. An electronic device, comprising: The electronic device comprises at least one processor and a memory connected in communication with the at least one processor, wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to cause the at least one processor to execute the abnormal behavior detection method of any one of claims 1 to 6.

Images