Video monitoring method and system based on large model and related equipment

Abstract

The invention discloses a video monitoring method and system based on a large model and related equipment, and the method is applied to video monitoring equipment, and comprises the steps: obtaining video data of a monitoring region; sending the video data to a cloud server to enable a large model of the cloud server to perform target behavior analysis on the video data, extracting semantic information based on a target behavior analysis result, and generating corresponding response information; and receiving the returned response information, and executing corresponding operation according to the response information. According to the method, the video data is intelligently analyzed through the cloud large model, accurate behavior recognition and automatic response generation are realized, and the intelligence and real-time response capability of the monitoring system are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and in particular to a large model-based video surveillance method, system and related equipment. Background Art

[0002] With the rapid development of Internet of Things technology, video surveillance systems have been widely used in many fields such as home care, medical monitoring, and public safety, becoming an important tool for real-time monitoring and ensuring personnel safety. Traditional video surveillance systems usually collect video data through cameras, and users can remotely view the monitoring screen through terminal devices to achieve real-time or historical monitoring of specific areas. Although these systems have met users' needs for care and safety to a certain extent, existing monitoring systems still have great limitations in responding to real-time events and risk prediction. Summary of the invention

[0003] The embodiment of the present application provides a video monitoring method, system and related equipment based on a large model, which can realize accurate behavior recognition and automatic response generation through intelligent analysis of video data by a large cloud model, thus significantly improving the intelligence and real-time response capability of the monitoring system. The above technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a video monitoring method based on a large model, which is applied to a video monitoring device, and the method includes:

[0005] Obtain video data of the monitored area;

[0006] Send the video data to a cloud server so that the large model of the cloud server performs target behavior analysis on the video data, extracts semantic information based on the target behavior analysis results, and generates corresponding response information;

[0007] Receive the returned response information and perform corresponding operations according to the response information.

[0008] In a possible implementation, the target behavior analysis on the video data includes:

[0009] Performing target detection on the above video data to obtain information of multiple objects;

[0010] Determine the target object based on the above multiple object information, and perform behavior recognition to determine the behavior pattern of the above target object;

[0011] According to the behavior pattern matching of the target object, the corresponding behavior evaluation rules are evaluated to obtain the target behavior analysis results.

[0012] In a possible implementation, the above-mentioned extraction of semantic information based on the target behavior analysis result and generation of corresponding response information include:

[0013] Based on the target behavior analysis results, extracting semantic information related to the current behavior state of the target object;

[0014] A natural language response is generated according to the semantic information, and the natural language response is returned to the video surveillance device as response information.

[0015] In a possible implementation, the performing corresponding operations according to the response information includes:

[0016] When the response information is voice information, outputting the response information through the speaker of the video monitoring device;

[0017] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then output through a speaker of the video surveillance device.

[0018] In a possible implementation, the video surveillance device is associated with a terminal device, and the method further includes:

[0019] The response information is sent to the terminal device, and the response information is output through the speaker and / or display screen of the terminal device.

[0020] In a second aspect, an embodiment of the present application provides a video monitoring method based on a large model, which is applied to a cloud server, and the method includes:

[0021] Receiving video data of the monitoring area sent by the video monitoring equipment;

[0022] Performing target behavior analysis on the video data through the large model on the cloud server, extracting semantic information based on the target behavior analysis results, and generating corresponding response information;

[0023] The response information is returned to the video monitoring device so that the video monitoring device performs corresponding operations according to the response information.

[0024] In a possible implementation, the target behavior analysis on the video data includes:

[0025] Performing target detection on the above video data to obtain information of multiple objects;

[0026] Determine the target object based on the above multiple object information, and perform behavior recognition to determine the behavior pattern of the above target object;

[0027] According to the behavior pattern matching of the target object, the corresponding behavior evaluation rules are evaluated to obtain the target behavior analysis results.

[0028] In a possible implementation, the above-mentioned extraction of semantic information based on the target behavior analysis result and generation of corresponding response information include:

[0029] Based on the target behavior analysis results, extracting semantic information related to the current behavior state of the target object;

[0030] A natural language response is generated according to the semantic information, and the natural language response is used as response information returned to the video surveillance device.

[0031] In a possible implementation, the performing corresponding operations according to the response information includes:

[0032] When the response information is voice information, outputting the response information through the speaker of the video monitoring device;

[0033] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then output through a speaker of the video surveillance device.

[0034] In a possible implementation, the video surveillance device is associated with a terminal device, and the method further includes:

[0035] The response information is sent to the terminal device, and the response information is output through the speaker and / or display screen of the terminal device.

[0036] In a third aspect, an embodiment of the present application provides a video surveillance device based on a large model, which is applied to a video surveillance device, and the device includes:

[0037] An acquisition module is used to acquire video data of the monitoring area;

[0038] A sending module, used to send the video data to a cloud server, so that the big model of the cloud server performs target behavior analysis on the video data, extracts semantic information based on the target behavior analysis results, and generates corresponding response information;

[0039] The output module is used to receive the returned response information and perform corresponding operations according to the response information.

[0040] In a possible implementation, the target behavior analysis on the video data includes:

[0041] Performing target detection on the above video data to obtain information of multiple objects;

[0042] Determine the target object based on the above multiple object information, and perform behavior recognition to determine the behavior pattern of the above target object;

[0043] According to the behavior pattern matching of the target object, the corresponding behavior evaluation rules are evaluated to obtain the target behavior analysis results.

[0044] In a possible implementation, the above-mentioned extraction of semantic information based on the target behavior analysis result and generation of corresponding response information include:

[0045] Based on the target behavior analysis results, extracting semantic information related to the current behavior state of the target object;

[0046] A natural language response is generated according to the semantic information, and the natural language response is returned to the video surveillance device as response information.

[0047] In a possible implementation, the output module is specifically used to:

[0048] When the response information is voice information, outputting the response information through the speaker of the video monitoring device;

[0049] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then output through a speaker of the video surveillance device.

[0050] In a possible implementation, the video surveillance device is associated with a terminal device, and the output module is further used to:

[0051] The response information is sent to the terminal device, and the response information is output through the speaker and / or display screen of the terminal device.

[0052] In a fourth aspect, an embodiment of the present application provides a video monitoring device based on a large model, which is applied to a cloud server, and the device includes:

[0053] A receiving module, used for receiving video data of a monitoring area sent by a video monitoring device;

[0054] A processing module, used to perform target behavior analysis on the video data through the large model on the cloud server, extract semantic information based on the target behavior analysis results, and generate corresponding response information;

[0055] The response module is used to return the above response information to the above video monitoring device, so that the above video monitoring device performs corresponding operations according to the above response information.

[0056] In a possible implementation, the processing module is specifically configured to:

[0057] Performing target detection on the above video data to obtain information of multiple objects;

[0058] Determine the target object based on the above multiple object information, and perform behavior recognition to determine the behavior pattern of the above target object;

[0059] According to the behavior pattern matching of the target object, the corresponding behavior evaluation rules are evaluated to obtain the target behavior analysis results.

[0060] In a possible implementation, the processing module is specifically configured to:

[0061] Based on the target behavior analysis results, extracting semantic information related to the current behavior state of the target object;

[0062] A natural language response is generated according to the semantic information, and the natural language response is used as response information returned to the video surveillance device.

[0063] In a possible implementation, the performing corresponding operations according to the response information includes:

[0064] When the response information is voice information, outputting the response information through the speaker of the video monitoring device;

[0065] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then output through a speaker of the video surveillance device.

[0066] In a possible implementation, the video surveillance device is associated with a terminal device, and the response module is further configured to:

[0067] The response information is sent to the terminal device, and the response information is output through the speaker and / or display screen of the terminal device.

[0068] In a fifth aspect, an embodiment of the present application provides a video surveillance system based on a large model, including a video surveillance device and a cloud server;

[0069] The video surveillance device is used to execute the method steps provided by the first aspect of the embodiment of the present application or any possible implementation manner of the first aspect;

[0070] The above-mentioned cloud server is used to execute the method steps provided by the second aspect of the embodiment of the present application or any possible implementation method of the second aspect.

[0071] In a sixth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores instructions, and the instructions are suitable for being loaded by a processor and executing the method provided by the first aspect of the embodiment of the present application or any possible implementation of the first aspect or the second aspect or any possible implementation of the second aspect.

[0072] In a seventh aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0073] The processor is connected to the memory;

[0074] The above-mentioned memory is used to store executable program code;

[0075] The above-mentioned processor runs the program corresponding to the above-mentioned executable program code by reading the executable program code stored in the above-mentioned memory, so as to execute the method provided by the first aspect or any possible implementation of the first aspect or the second aspect or any possible implementation of the second aspect of the embodiment of the present application.

[0076] In one or more embodiments of the present application, by sending video data to a cloud server and using a large model to analyze target behavior, intelligent real-time monitoring of the monitoring area is achieved. This method can extract semantic information from the target behavior analysis results and generate accurate response information, thereby effectively improving the real-time response capability and intelligence level of the monitoring system, and achieving timely warning and effective intervention of potential risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0078] Figure 1 A schematic diagram of the architecture of a video surveillance system based on a large model provided by an exemplary embodiment of the present application;

[0079] Figure 2 A schematic diagram of a flow chart of a video monitoring method based on a large model provided as an exemplary embodiment of the present application;

[0080] Figure 3 A schematic flow chart of another large model-based video monitoring method provided as an exemplary embodiment of the present application;

[0081] Figure 4 A schematic structural diagram of a video monitoring device based on a large model provided as an exemplary embodiment of the present application;

[0082] Figure 5 A schematic structural diagram of another large-model-based video surveillance device provided as an exemplary embodiment of the present application;

[0083] Figure 6 A schematic structural diagram of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0084] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0085] In the description of the present application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship.

[0086] The present application is described in detail below with reference to specific embodiments.

[0087] Please refer to Figure 1 , Figure 1 The following is a schematic diagram of the architecture of a video surveillance system based on a large model provided by an exemplary embodiment of the present application. Figure 1 As shown, the video surveillance system may include a video surveillance device 110 and a cloud server 120 .

[0088] The video surveillance device 110 is responsible for data collection and transmission. The video surveillance device 110 is responsible for continuously collecting video data of the monitored area, which includes image information and may also include audio information. The camera of the video surveillance device 110 may adopt a high-definition camera, an infrared camera, etc. to support different shooting modes and shooting conditions. The network interface of the video surveillance device 110 is responsible for transmitting the video data to the cloud server 120 through the network. The network interface generally includes a wired network interface (such as Ethernet) or a wireless network interface (such as a Wi-Fi module or a 4G / 5G module). Optionally, the video surveillance device 110 may include a processing unit for preliminary processing of video data, such as implementing image preprocessing, data compression and other functions. Optionally, the video surveillance device 110 may include a storage module for temporarily storing video data to facilitate caching during data transmission. Optionally, the video surveillance device 110 may include an audio output module such as a speaker for playing audio information such as voice prompts and music.

[0089] The cloud server 120 is responsible for data analysis and response generation, and includes a computing unit, a storage unit, and a network interface. The computing unit includes a high-performance processor (CPU / GPU) for running large models and performing complex computing tasks, such as target behavior analysis, semantic extraction, etc. The storage system is used to store video data, analysis results, and training data for large models, and a high-speed solid-state drive (SSD) or a distributed storage system can be used. The network interface is used to receive data transmitted by the video surveillance device 110 and return the analysis results to the video surveillance device 110, and may include a high-speed network interface, such as Gigabit Ethernet or fiber optic connection. Optionally, the cloud server 120 also has a security module for protecting the security of data transmission, including security measures such as encryption and firewalls.

[0090] The video surveillance device 110 is connected to the cloud server 120 through a network interface and uploads the collected video data to the cloud server 120. After receiving the video data, the cloud server 120 uses the computing unit to run the large model for processing, and the generated response information is returned to the video surveillance device 110 through the network interface. Based on the response information returned by the cloud server 120, the video surveillance device 110 performs corresponding operations, such as issuing a prompt voice, adjusting the camera angle, etc.

[0091] Optionally, the video surveillance system can be associated with a terminal device, which can be used for user interaction and information viewing, such as viewing monitoring data, receiving response information, and controlling the system. The terminal device can be a personal computer, a tablet computer, a smart phone, a smart TV, a smart wearable device, etc. The terminal device may include an output unit, a network interface, and a processing unit. The output unit of the terminal device includes a display screen or a speaker, and the display screen can be used to display image content such as video surveillance images, response information, and system status, and the speaker can play corresponding audio content. The network interface of the terminal device is used to connect to the video surveillance device 110 or the cloud server 120, and may include a wired network interface (such as Ethernet) or a wireless network interface (such as a Wi-Fi module or a 4G / 5G module) to receive video data, analysis results, or response information. The processing unit of the terminal device is used to run a user interface program, process user instructions, and exchange data with the video surveillance device 110. Optionally, the terminal device may include an input device, such as a keyboard, a mouse, a touch screen, etc., for user operation and input instructions. For example, a user can control the functions of the video surveillance system by clicking a mouse, sliding a touch screen, or inputting a keyboard. Optionally, the terminal device may include a storage module for storing local data on the terminal device, such as user settings, historical operation records, historical video data, etc.

[0092] The terminal device can be connected to the video surveillance device 110 or the cloud server 120 through a network interface. The operation instructions issued by the user through the terminal device can be transmitted to the video surveillance device 110 or the cloud server 120 to execute the corresponding functions. The data, analysis results or response information returned by the video surveillance device 110 and the cloud server 120 can be transmitted to the terminal device for the user to view and further operate.

[0093] It should be noted that the above-mentioned association between the terminal device and the video surveillance system refers to the communication and interaction relationship established between the terminal device and the video surveillance device 110 and / or the cloud server 120. For example, the terminal device establishes a physical and network connection with the video surveillance device 110 or the cloud server 120 through a network interface. Optionally, during the connection process, the terminal device needs to configure network settings and perform identity authentication to ensure secure access. Once the connection is established, the terminal device can exchange data with the video surveillance device 110 and / or the cloud server 120 through standard protocols and application programming interfaces (APIs).

[0094] in, Figure 1 The number of video surveillance devices 110 is only an example. In actual applications, the number of video surveillance devices 110 and terminal devices in the video surveillance system can be one or more, which is determined based on the actual application environment and is not limited in this embodiment of the present application.

[0095] Next, combine Figure 1 , introduces the video monitoring method based on a large model provided by the embodiment of this application. For details, please refer to Figure 2 , which is a flow chart of a video monitoring method based on a large model provided by an exemplary embodiment of the present application, the video monitoring method includes the following steps:

[0096] S201, the video surveillance device obtains video data of the surveillance area.

[0097] Specifically, one or more video surveillance devices are arranged in the monitoring area to obtain video data of the monitoring area. The video surveillance device includes a camera that can collect clear image information. The video surveillance device may include an audio acquisition device that can collect corresponding audio information while collecting image information. It can be understood that the video data acquired by the video surveillance device may include only image information, or may include both image information and audio information.

[0098] Optionally, when collecting video data, the video surveillance device can perform preliminary processing on the captured image, such as noise reduction, sharpening and color correction, to improve image quality. Optionally, when collecting video data, the video surveillance device can use a compression algorithm to compress the video data (such as H.264, H.265 encoding) to reduce redundant data while maintaining image quality.

[0099] S202, the video surveillance device sends the video data to the cloud server.

[0100] Specifically, the video surveillance device can send video data to the cloud server through a network interface. The network interface can use a wired connection or a wireless connection. Exemplarily, the video surveillance device can be connected to the network through an Ethernet port to provide higher stability and transmission speed. The video surveillance device can use a wireless network to send video data to the cloud server through a Wi-Fi module or other wireless network interface to provide installation flexibility.

[0101] Among them, the video data can be sent to the cloud server through a standard network transmission protocol (such as Real-Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), Hypertext Transfer Protocol Secure (HTTPS)) to ensure that the data can be transmitted reliably. Optionally, to meet the needs of network transmission, the video data will be divided into multiple data packets during the transmission process, and the data packets will be reassembled into a complete video stream after arriving at the cloud server. Optionally, in order to ensure the security of the data during transmission, the video data will be encrypted (such as using TLS / SSL encryption protocol) to prevent the data from being stolen or tampered with during transmission. Optionally, during the transmission process, identity authentication will be performed between the video surveillance device and the cloud server to ensure that only authorized devices can send and receive data. Optionally, after the video data arrives at the cloud server, the video data will be stored in the cloud storage system for further analysis, playback or retrieval.

[0102] S203, the large model of the cloud server performs target behavior analysis on the video data, extracts semantic information based on the target behavior analysis results, and generates corresponding response information.

[0103] Specifically, the large model of the cloud server is a model based on deep learning, which has a large number of parameters and a complex network structure and can handle complex tasks. The large model can perform target behavior analysis on video data to identify target objects (such as people, animals or objects) in video data and evaluate the behavior of target objects in video data. The purpose of target behavior analysis is to understand the behavior of the target object and identify possible abnormal or meaningful behaviors. Semantic information is the content with practical significance extracted from the target behavior analysis. The specific behavior description of the target object can be extracted based on the analysis results, such as "the child is close to the window" or "the student is playing with the mobile phone." The description of potential risks or problems can be extracted based on the analysis results. For example, "the child may be in danger of falling when he is close to the window" or "the student may be distracted by playing with the mobile phone in class." Finally, the large model can convert the target behavior analysis results into natural language, prompts or alarms that are understandable and useful to humans based on the extracted semantic information, that is, generate response information based on the semantic information.

[0104] It should be noted that the large model of the above-mentioned cloud server can be a multimodal integrated model, so that it can simultaneously process image information, audio information, understand natural language, and generate natural language.

[0105] In order to achieve the above functions, the large model needs to be systematically trained and verified. Exemplarily, the large model training process may include the following steps:

[0106] 1) Data collection and preparation

[0107] First, collect a large amount of annotated video data. These video data should cover target objects, behaviors, and events in different scenes, and the video data should contain various behavior patterns (such as walking, sitting, approaching the window, etc.), and can carry relevant audio information. Next, annotate the video data, including object categories (such as people, objects), behavior patterns (such as standing, moving), and event contexts (such as approaching the window, using a mobile phone).

[0108] 2) Target behavior analysis model training

[0109] The image recognition sub-model can be trained to detect objects in video data, locate and classify these objects. Exemplary model algorithms that can be used include YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), and Faster R-CNN (Faster Region-based Convolutional Neural Network).

[0110] At the same time, the behavior recognition sub-model can be trained using the time series model to analyze the behavior patterns of the target object determined from the image recognition sub-model. The behavior recognition sub-model is able to recognize and classify various behaviors and can be implemented using algorithms such as LSTM (Long Short-Term Memory) and 3D-CNN (3D Convolutional Neural Network).

[0111] 3) Semantic information extraction model training

[0112] The target behavior analysis results output by the target behavior analysis model are used as input to train the semantic information extraction model to extract structured semantic information. For example, the model is trained to recognize and extract semantic information about the current state of the target object (such as "near the window" and "using a mobile phone"). In addition, a language generation model (such as a Transformer-based model) can be trained to convert the extracted semantic information into prompt words or keywords. The language generation model can be trained and fine-tuned on a large amount of annotated data to improve the accuracy of prompt word generation.

[0113] 4) Natural language generation model training

[0114] Train and fine-tune a natural language generation model (such as GPT, Generative Pre-trained Transformer, or similar language generation models) to generate contextual responses based on the extracted semantic information or prompt words. The natural language generation model can understand user needs and contextual information in different video scenes and generate corresponding natural language responses. For example, if the semantic information is "the child is sitting by the window playing with a mobile phone", the response "you are close to the window, please be safe" can be generated.

[0115] 5) Model integration and optimization

[0116] The target behavior analysis model, semantic information extraction model and natural language generation model are integrated into a multimodal large model system to ensure seamless collaboration and data transfer among modules. The integrated large model can also be optimized, including parameter adjustment, model structure improvement and training data enhancement, to further improve the accuracy and response speed of the model.

[0117] 6) Verification and testing

[0118] Validate large models on independent test sets to ensure they can accurately identify target behaviors, extract semantic information, and generate natural language responses. You can also conduct user testing in actual application scenarios to collect feedback and make iterative improvements.

[0119] It should be noted that the construction and training process of the above large model is an example. In the specific implementation, the construction and training process can be flexibly adjusted according to the needs, as long as it can realize the target behavior analysis of the video data, extract the semantic information based on the target analysis results, and generate the corresponding response information.

[0120] In some embodiments, the large model of the cloud server performs target behavior analysis on video data, including:

[0121] Perform target detection on video data to obtain information about multiple objects;

[0122] Determine the target object based on multiple object information, perform behavior recognition, and determine the behavior pattern of the target object;

[0123] According to the behavior pattern of the target object, the corresponding behavior evaluation rules are matched to perform evaluation and obtain the target behavior analysis results.

[0124] Specifically, the target detection module of the large model (such as YOLO, SSD, Faster R-CNN) is used to identify and locate all possible objects in the video data. For example, by scanning the video frame and generating a bounding box and category label containing each detected object. The target detection result can be a list containing all detected object information. The object information can include the location information, category, and confidence score of the object.

[0125] Among all detected objects, the object of interest is selected, i.e., the target object. The target object can be preset as a human object, or the target object selection can be performed according to the set priority, such as giving priority to human objects, animals, moving objects, etc., so that the large model can be applied to video surveillance in various scenarios.

[0126] After determining the target object, the behavior recognition module (such as LSTM, 3D-CNN) in the large model analyzes and identifies the behavior pattern of the target object. Behavior recognition will determine the behavior pattern of the target object, such as "sleeping", "studying", "watching TV", etc. Among them, the determination of the behavior pattern of the target object not only considers the position and state of the target object itself, but also the position and state of other objects around the target object. For example, a child sitting in front of a turned-off TV and eating has a behavior pattern of "eating"; a child sitting in front of a TV that is playing a picture and eating has a behavior pattern of "eating" and "watching TV". The behavior recognition result can include the behavior pattern of the target object, and a behavior pattern of a target object can include one or more.

[0127] Different behavior patterns correspond to different behavior evaluation rules. The behavior evaluation rules define the expected and abnormal behaviors under a specific behavior pattern. By matching the behavior pattern of the target object with the corresponding behavior evaluation rules for evaluation, it can be determined whether the behavior of the target object meets expectations or has abnormal behaviors. For example, when a user is sleeping, he may fall by the bed, and the quilt covering his mouth and nose may cause suffocation risks; the evaluation rules of the behavior pattern "sleeping" can include sleeping posture evaluation and suffocation evaluation; sleeping posture evaluation can include evaluating the distance of the target object from the bed. When it is identified that the target object is close to the bed at a certain distance, it can be determined that the target object is at risk of falling; suffocation evaluation can evaluate whether the mouth and nose of the target object are covered. When it is identified that the mouth and nose of the target object are covered, it can be determined that the target object is at risk of suffocation.

[0128] The purpose of performing the above target behavior analysis on video data is to extract and analyze the behavior of the target object from the video data and evaluate it according to predetermined rules in order to assess the current state and degree of abnormality of these behaviors. By performing target behavior analysis on video data, the big model can understand the dynamic behavior in the video data for real-time monitoring and risk warning. It should be noted that the above target behavior analysis results are the behavior patterns and state information of the target object identified by the big model after analyzing the target object in the video data.

[0129] In some embodiments, the large model of the cloud server extracts semantic information based on the target behavior analysis results and generates corresponding response information, including:

[0130] Based on the target behavior analysis results, semantic information related to the current behavior state of the target object is extracted;

[0131] A natural language response is generated according to the semantic information, and the natural language response is returned to the video monitoring device as response information.

[0132] Specifically, after analyzing the target object in the video data, the large model can identify the behavior pattern and state information of the target object. For example, the behavior of the target object may be identified as "by the window", "playing with a mobile phone" or "lying in bed", etc. Subsequently, the semantic information extraction module of the large model will extract useful semantic information related to the behavior state of the target object from the target behavior analysis results. This semantic information provides a clear explanation of the target behavior. For example, if a child is detected approaching an open window, the extracted semantic information may be "the child is close to the window, which is potentially dangerous." This semantic information provides the large model with a context for understanding the behavior of the target object and lays the foundation for generating response information.

[0133] The Natural Language Processing (NLP) module integrated in the large model can generate easy-to-understand natural language responses based on the extracted semantic information using natural language processing technology. For example, for the above extracted semantic information "the child is close to the window, there is a potential danger", the generated natural language response may be "please stay away from the window to avoid danger."

[0134] Optionally, in order to better guide the NLP module to generate appropriate response information, after extracting the semantic information, the extracted semantic information can be converted into targeted prompt words. These prompt words can be simplified and specific questions or instructions to guide the subsequent natural language generation process. For example, the semantic information "the child is close to the window and there may be a risk of falling" can be converted into the prompt word "how should I remind the child to pay attention to safety when he is near the window?"

[0135] Optionally, prompt words can be generated based on predefined rules and templates to convert semantic information into instructions to guide the NLP model. For example, a rule base is predefined, which contains the specific format of how to convert different behavior patterns or risk information into prompt words. For the situation of "children approaching the window", the template "When the target object [behaves], how should [behave]?" can be used to generate the corresponding prompt words.

[0136] Optionally, a natural language generation model (such as GPT, etc.) can be used to automatically generate prompt words. The natural language generation model can learn and generate appropriate prompt words based on the input semantic information. In the training stage, the natural language generation model uses training data with prompt words for learning, so as to master how to generate corresponding prompt words from specific semantic information.

[0137] Then, the generated prompt words will be input into the NLP module. Based on the prompt words, the NLP module (such as GPT, etc.) combines the training data and context information to generate detailed natural language responses that meet the needs of actual application scenarios. For example, when the prompt word is "When the child is near the window, how should he be reminded to pay attention to safety?", the NLP module will generate a response message of "Please pay attention to safety, stay away from the window, and be careful of falling." Finally, the generated natural language response will be transmitted to the video surveillance equipment through the network as response information to achieve more intelligent and timely monitoring and early warning functions.

[0138] S204, the cloud server returns the response information to the video surveillance device.

[0139] Specifically, after the cloud server completes the target behavior analysis, semantic information extraction and response information generation of the video data through the large model, it will transmit the generated response information back to the video surveillance device through the network, ensuring that the video surveillance device can receive intelligent feedback on the target behavior in the monitoring area in real time.

[0140] S205: The video surveillance device performs corresponding operations according to the response information.

[0141] Specifically, when the video monitoring device receives the response information returned by the cloud server, it will perform corresponding operations according to the response information. The video monitoring device includes an audio output module such as a speaker, which can play the audio corresponding to the response information.

[0142] In some embodiments, the video surveillance device performs corresponding operations according to the response information, including:

[0143] When the response information is voice information, outputting the response information through a speaker of the video monitoring device;

[0144] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then the response information is output through a speaker of the video monitoring device.

[0145] When the response information is a voice message, the response information generated by the cloud server is already in voice format, and the response information contains voice content directly used for reminders or notifications. In this case, the video surveillance device can develop the habit of directly using its own speakers to play the voice information. For example, if a child is detected approaching the window, the generated voice message may be "Please stay away from the window, pay attention to safety", and the device will immediately play this reminder through the speaker, so that the monitored object or the surrounding people will be informed in time.

[0146] When the response information is text information, the response information generated by the cloud server is in text format. At this time, the video surveillance device can convert the text information into voice format through its internal text-to-speech (TTS) module. The converted voice information is then played through the speaker. For example, if the text message is "Please stay away from the window and pay attention to safety", the TTS module will convert the text into corresponding voice and then output it through the speaker.

[0147] Through the above method, it can be ensured that the video surveillance equipment can flexibly process response information of different formats. Regardless of whether the information is directly generated voice or text, it can eventually be played through the speaker, which improves the efficiency of information transmission and ensures that the reminder information can be transmitted to the target object in a timely and accurate manner, effectively improving the real-time and response capabilities of the monitoring system.

[0148] Optionally, the above-mentioned corresponding operations performed according to the response information may also include but are not limited to the following:

[0149] Display prompt information: The video surveillance device can display prompt content related to the response information on the connected screen. For example, display the text message "Please stay away from the window to avoid danger" to ensure that the guardian or target object can see the reminder.

[0150] Triggering automatic control: Video surveillance equipment can be linked to other smart devices. For example, if a fire or flame is detected, the response information may contain relevant control instructions. The video surveillance equipment can automatically notify the smart home system to turn off the power, turn on the fire sprinkler system, or send an emergency notification to relevant personnel.

[0151] Push notification: Video surveillance equipment can send response information to the user's mobile phone or other terminal devices in the form of push notifications, ensuring that users can promptly understand abnormal situations in the monitored area.

[0152] In the embodiment of the present application, by sending video data to a cloud server and using a large model to analyze target behavior, intelligent real-time monitoring of the monitoring area is achieved. This method can extract semantic information from the target behavior analysis results and generate accurate response information, thereby realizing the intelligence and real-time nature of the video monitoring system, and greatly improving the system's capabilities in behavior recognition and risk warning.

[0153] The above-mentioned video monitoring method based on a large model will be introduced below in conjunction with a specific implementation scenario. In this exemplary embodiment, the video monitoring system is associated with a terminal device, and a communication and interaction relationship is established between the terminal device and the video monitoring device and / or the cloud server. Figure 3 , which is a flow chart of a video monitoring method provided by an exemplary embodiment of the present application. Figure 3 As shown, the video monitoring method includes the following steps:

[0154] S301, the video surveillance device obtains video data of the surveillance area.

[0155] Specifically, step S301 is consistent with step S201 and will not be described in detail here.

[0156] S302, the video surveillance device sends the video data to the cloud server.

[0157] Specifically, step S302 is consistent with step S202 and will not be described in detail here.

[0158] S303, the large model of the cloud server performs target detection on the video data to obtain information of multiple objects.

[0159] Specifically, the cloud server uses the big model to perform target detection on the received video data. The big model can automatically identify different objects in the video and obtain relevant information of each object, such as location, shape, size, category, etc. This object information includes people, animals, objects, etc., in order to provide a basis for subsequent target behavior analysis.

[0160] S304, the large model of the cloud server determines the target object based on multiple object information, and performs behavior recognition to determine the behavior pattern of the target object.

[0161] Specifically, after obtaining information about multiple objects, the big model will determine the target object that needs to be focused on based on preset rules or algorithms. For example, in a child monitoring scenario, the target object is the child. After determining the target object, the big model will identify its behavior, analyze the target object's movements, states, and activity trajectories, and determine its behavior patterns, such as "sleeping", "studying", "watching TV", "near the window", etc.

[0162] S305, the large model of the cloud server performs evaluation according to the behavior pattern matching corresponding behavior evaluation rules of the target object to obtain the target behavior analysis result.

[0163] Specifically, the cloud server's big model matches the target object's behavior pattern with pre-set behavior evaluation rules and evaluates the behavior. For example, the evaluation rules may include "Is there a potential danger when approaching a window", "Does being still for a long time require attention", or "Is he playing with his phone while studying?" By matching the target object's behavior with these rules, the target behavior analysis results of the target behavior can be obtained to determine whether it is abnormal, potentially dangerous, or requires reminders.

[0164] S306, the cloud server's large model extracts semantic information related to the current behavior state of the target object based on the target behavior analysis results.

[0165] Specifically, based on the target behavior analysis results, the large model will extract semantic information related to the current behavior state of the target object to form an understanding of the target object's behavior. For example, if the target behavior analysis results show that the child is using a mobile phone while studying, the extracted semantic information may be "the child is playing with the mobile phone while studying." This step converts the target behavior analysis results into understandable descriptive information, and can determine the prompt words required for natural language generation based on the semantic information.

[0166] S307, the big model of the cloud server generates a natural language response based on the semantic information.

[0167] Specifically, the large model uses natural language processing technology to generate a detailed natural language response that meets the needs of actual application scenarios based on the extracted semantic information. For example, for the semantic information "children play with mobile phones while studying", the generated natural language response may be "please stop playing with mobile phones and concentrate on studying."

[0168] S308, the cloud server sends the response information to the terminal device.

[0169] Specifically, the video surveillance system is associated with a terminal device, which may be a personal computer, tablet computer, smart phone, smart TV, smart wearable device, etc.; the cloud server may send the response information to the associated terminal device through the network. For example, if a child is using a mobile phone while studying, a natural language response of "Please do not play with your mobile phone, concentrate on studying" may be sent to the mobile phone the child is using or to a wearable watch the child is using; the response information may be sent to the parent's mobile phone to remind the parent to supervise. Optionally, the cloud server's large model may integrate a face recognition function, which can identify the identity of a person based on the image of the video data, so as to provide a target behavior analysis result containing the person's identity information.

[0170] S309, the terminal device outputs response information.

[0171] Specifically, when the response information is voice information, the response information is sent to the terminal device and output through the speaker of the terminal device; when the response information is text information, the response information is sent to the terminal device, and the text information is converted into voice format by the text-to-speech module inside the terminal device. The converted voice information is then played through the speaker of the terminal device, or the response information in text format is directly output using the display screen of the terminal device.

[0172] In the embodiment of the present application, video data of the monitored area is collected by video surveillance equipment, and a large cloud model is used to perform target detection, behavior recognition, risk assessment, semantic information extraction and natural language generation on the video data, and finally the generated response information is sent to the terminal device for output, thereby realizing real-time intelligent analysis and feedback of the target object by the monitoring system, greatly improving the accuracy and timeliness of the monitoring system in behavior recognition, risk prediction and information response, and effectively enhancing the early warning and handling capabilities of emergencies and potential risks in the monitored area.

[0173] Please refer to the following Figure 4 , Figure 4 A video monitoring device based on a large model is provided as an exemplary embodiment of the present application. The video monitoring device 400 is applied to a video monitoring device, such as Figure 4 As shown, the device includes an acquisition module 410 , a sending module 420 and an output module 430 .

[0174] An acquisition module 410 is used to acquire video data of a monitoring area;

[0175] The sending module 420 is used to send the video data to the cloud server, so that the large model of the cloud server performs target behavior analysis on the video data, extracts semantic information based on the target behavior analysis results, and generates corresponding response information;

[0176] The output module 430 is used to receive the returned response information and perform corresponding operations according to the response information.

[0177] In some possible embodiments, the target behavior analysis of the video data includes:

[0178] Performing target detection on the above video data to obtain information of multiple objects;

[0179] Determine the target object based on the above multiple object information, and perform behavior recognition to determine the behavior pattern of the above target object;

[0180] According to the behavior pattern matching of the target object, the corresponding behavior evaluation rules are evaluated to obtain the target behavior analysis results.

[0181] In some possible embodiments, the above-mentioned extraction of semantic information based on the target behavior analysis result and generation of corresponding response information include:

[0182] Based on the target behavior analysis results, extracting semantic information related to the current behavior state of the target object;

[0183] A natural language response is generated according to the semantic information, and the natural language response is returned to the video surveillance device as response information.

[0184] In some possible embodiments, the output module 430 is specifically used to:

[0185] When the response information is voice information, outputting the response information through the speaker of the video monitoring device;

[0186] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then output through a speaker of the video surveillance device.

[0187] In some possible embodiments, the video surveillance device is associated with a terminal device, and the output module 430 is further used to:

[0188] The response information is sent to the terminal device, and the response information is output through the speaker and / or display screen of the terminal device.

[0189] The division of each module in the above-mentioned video surveillance device based on the large model is only for illustration. In other embodiments, the video surveillance device based on the large model can be divided into different modules as needed to complete all or part of the functions of the above-mentioned video surveillance device based on the large model. The implementation of each module in the video surveillance device based on the large model provided in the embodiment of the present application can be in the form of a computer program. When the computer program is executed by the processor, all or part of the steps of the video surveillance method based on the large model described in the embodiment of the present application are implemented.

[0190] Please refer to the following Figure 5 , Figure 5 A video monitoring device based on a large model is provided as an exemplary embodiment of the present application. The video monitoring device 500 is applied to a cloud server, such as Figure 5 As shown, the device includes a receiving module 510 , a processing module 520 and a responding module 530 .

[0191] The receiving module 510 is used to receive the video data of the monitoring area sent by the video monitoring device;

[0192] The processing module 520 is used to perform target behavior analysis on the video data through the large model on the cloud server, extract semantic information based on the target behavior analysis result, and generate corresponding response information;

[0193] The response module 530 is used to return the response information to the video surveillance device, so that the video surveillance device performs corresponding operations according to the response information.

[0194] In some possible embodiments, the processing module 520 is specifically used to:

[0195] Performing target detection on the above video data to obtain information of multiple objects;

[0196] Determine the target object based on the above multiple object information, and perform behavior recognition to determine the behavior pattern of the above target object;

[0197] According to the behavior pattern matching of the target object, the corresponding behavior evaluation rules are evaluated to obtain the target behavior analysis results.

[0198] In some possible embodiments, the processing module 520 is specifically used to:

[0199] Based on the target behavior analysis results, extracting semantic information related to the current behavior state of the target object;

[0200] A natural language response is generated according to the semantic information, and the natural language response is used as response information returned to the video surveillance device.

[0201] In some possible embodiments, the performing corresponding operations according to the response information includes:

[0202] When the response information is voice information, outputting the response information through the speaker of the video monitoring device;

[0203] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then output through a speaker of the video surveillance device.

[0204] In some possible embodiments, the video surveillance device is associated with a terminal device, and the response module 530 is further used to:

[0205] The response information is sent to the terminal device, and the response information is output through the speaker and / or display screen of the terminal device.

[0206] The division of each module in the above-mentioned video surveillance device based on the large model is only for illustration. In other embodiments, the video surveillance device based on the large model can be divided into different modules as needed to complete all or part of the functions of the above-mentioned video surveillance device based on the large model. The implementation of each module in the video surveillance device based on the large model provided in the embodiment of the present application can be in the form of a computer program. When the computer program is executed by the processor, all or part of the steps of the video surveillance method based on the large model described in the embodiment of the present application are implemented.

[0207] An embodiment of the present application also provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the large model-based video surveillance method described in any of the above embodiments.

[0208] Please refer to Figure 6 , is a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device in this specification may include one or more of the following components: a processor 610, a memory 620, an input device 630, an output device 640, and a bus 650. The processor 610, the memory 620, the input device 630, and the output device 640 may be connected via a bus 650.

[0209] The processor 610 may include one or more processing cores. The processor 610 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 620, and calling data stored in the memory 620. Optionally, the processor 610 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 610 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 610, but may be implemented separately through a communication chip.

[0210] The memory 620 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 620 includes a non-transitory computer-readable storage medium. The memory 620 may be used to store instructions, programs, codes, code sets, or instruction sets.

[0211] The input device 630 is used to receive input commands or data, and includes but is not limited to a keyboard, a mouse, a camera, a microphone or a touch device. The output device 640 is used to output commands or data, and includes but is not limited to a display screen and a speaker.

[0212] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WIFI) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0213] Understandably, Figure 6 This is an optional structure of the electronic device shown for ease of understanding. The specific structure of the electronic device can be designed and changed according to actual conditions, and the embodiments of the present application do not impose specific limitations on this.

[0214] exist Figure 6 In the electronic device, the memory 620 may store any one of the above-mentioned programs of the video monitoring method based on the large model. The processor 610 may be used to call any one of the programs of the video monitoring method based on the large model stored in the memory 620.

[0215] For example, in some embodiments, the electronic device may be applied to a video surveillance device, and the processor 610 may be used to call a large model-based video surveillance method stored in the memory 620, and specifically perform the following operations:

[0216] Obtain video data of the monitored area;

[0217] Send the video data to a cloud server so that the large model of the cloud server performs target behavior analysis on the video data, extracts semantic information based on the target behavior analysis results, and generates corresponding response information;

[0218] Receive the returned response information and perform corresponding operations according to the response information.

[0219] In some embodiments, the target behavior analysis of the video data includes:

[0220] Performing target detection on the above video data to obtain information of multiple objects;

[0221] Determine the target object based on the above multiple object information, and perform behavior recognition to determine the behavior pattern of the above target object;

[0222] According to the behavior pattern matching of the target object, the corresponding behavior evaluation rules are evaluated to obtain the target behavior analysis results.

[0223] In some embodiments, the above-mentioned extraction of semantic information based on the target behavior analysis result and generation of corresponding response information include:

[0224] Based on the target behavior analysis results, extracting semantic information related to the current behavior state of the target object;

[0225] A natural language response is generated according to the semantic information, and the natural language response is returned to the video surveillance device as response information.

[0226] In some embodiments, when executing corresponding operations according to the response information, the processor 610 further specifically performs:

[0227] When the response information is voice information, outputting the response information through the speaker of the video monitoring device;

[0228] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then output through a speaker of the video surveillance device.

[0229] In some embodiments, the video surveillance device is associated with a terminal device, and the processor 610 further specifically performs:

[0230] The response information is sent to the terminal device, and the response information is output through the speaker and / or display screen of the terminal device.

[0231] In some embodiments, the electronic device may be applied to a cloud server, and the processor 610 may be used to call the large model-based video monitoring method stored in the memory 620, and specifically perform the following operations:

[0232] Receiving video data of the monitoring area sent by the video monitoring equipment;

[0233] Performing target behavior analysis on the video data through the large model on the cloud server, extracting semantic information based on the target behavior analysis results, and generating corresponding response information;

[0234] The response information is returned to the video monitoring device so that the video monitoring device performs corresponding operations according to the response information.

[0235] In some embodiments, when performing target behavior analysis on the video data, the processor 610 further specifically performs:

[0236] Performing target detection on the above video data to obtain information of multiple objects;

[0237] Determine the target object based on the above multiple object information, and perform behavior recognition to determine the behavior pattern of the above target object;

[0238] According to the behavior pattern matching of the target object, the corresponding behavior evaluation rules are evaluated to obtain the target behavior analysis results.

[0239] In some embodiments, when the processor 610 extracts semantic information based on the target behavior analysis result and generates corresponding response information, it further specifically executes:

[0240] Based on the target behavior analysis results, extracting semantic information related to the current behavior state of the target object;

[0241] A natural language response is generated according to the semantic information, and the natural language response is used as response information returned to the video surveillance device.

[0242] In some embodiments, the above-mentioned performing corresponding operations according to the above-mentioned response information includes:

[0243] When the response information is voice information, outputting the response information through the speaker of the video monitoring device;

[0244] When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then output through a speaker of the video surveillance device.

[0245] In some embodiments, the video surveillance device is associated with a terminal device, and the processor 610 further specifically performs:

[0246] The response information is sent to the terminal device, and the response information is output through the speaker and / or display screen of the terminal device.

[0247] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the large model-based video monitoring method as described in the above embodiments.

[0248] An embodiment of the present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the large model-based video surveillance method as described in the above embodiments.

[0249] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0250] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A video monitoring method based on a large model, characterized in that: Applied to video surveillance equipment, the method comprises: Obtain video data of the monitored area; Sending the video data to a cloud server so that the large model of the cloud server performs target behavior analysis on the video data, extracts semantic information based on the target behavior analysis results, and generates corresponding response information; Receive the returned response information, and perform corresponding operations according to the response information.

2. The method according to claim 1, characterized in that The performing target behavior analysis on the video data includes: Performing target detection on the video data to obtain information of multiple objects; Determine a target object based on the multiple object information, and perform behavior recognition to determine a behavior pattern of the target object; The target object's behavior pattern is matched with a corresponding behavior evaluation rule to perform an evaluation to obtain a target behavior analysis result.

3. The method according to claim 2, characterized in that The extracting of semantic information based on the target behavior analysis result and generating corresponding response information include: Based on the target behavior analysis result, extracting semantic information related to the current behavior state of the target object; A natural language response is generated according to the semantic information, and the natural language response is returned to the video monitoring device as response information.

4. The method according to claim 1, characterized in that: The performing corresponding operations according to the response information includes: When the response information is voice information, outputting the response information through a speaker of the video surveillance device; When the response information is text information, the response information is converted into a voice format through a text-to-speech module, and then the response information is output through a speaker of the video monitoring device.

5. The method according to claim 1 or 4, characterized in that: The video surveillance device is associated with a terminal device, and the method further includes: The response information is sent to the terminal device, and the response information is output through a speaker and / or a display screen of the terminal device.

6. A video monitoring method based on a large model, characterized in that: Applied to a cloud server, the method comprises: Receiving video data of the monitoring area sent by the video monitoring equipment; Performing target behavior analysis on the video data through the large model on the cloud server, extracting semantic information based on the target behavior analysis results, and generating corresponding response information; The response information is returned to the video monitoring device, so that the video monitoring device performs corresponding operations according to the response information.

7. A video monitoring device based on a large model, characterized in that: Applied to video surveillance equipment, the device comprises: An acquisition module is used to acquire video data of the monitoring area; A sending module, used for sending the video data to a cloud server, so that the big model of the cloud server performs target behavior analysis on the video data, extracts semantic information based on the target behavior analysis result, and generates corresponding response information; The output module is used to receive the returned response information and perform corresponding operations according to the response information.

8. A video monitoring device based on a large model, characterized in that: Applied to a cloud server, the device comprises: A receiving module, used for receiving video data of a monitoring area sent by a video monitoring device; A processing module, configured to perform target behavior analysis on the video data through the large model on the cloud server, extract semantic information based on the target behavior analysis result, and generate corresponding response information; The response module is used to return the response information to the video surveillance device, so that the video surveillance device performs corresponding operations according to the response information.

9. A video monitoring system based on a large model, characterized in that: Including video surveillance equipment and cloud servers; The video surveillance device is used to perform the method steps as claimed in any one of claims 1 to 5; The cloud server is used to execute the method steps as claimed in claim 6.

10. A computer storage medium, characterized in that: The computer storage medium stores instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 6.

Images