Intelligent emergency command and dispatch system and method

Abstract

The invention discloses an intelligent emergency command and dispatch system, and the system comprises a voice input device which is used for inputting to-be-analyzed voice data to the system; the data conversion equipment is used for converting the voice data to be analyzed into text data and transmitting the text data to a large language model for analysis; the large language model is used for analyzing the text data and generating an instruction; and the control equipment is used for converting the instruction generated by the large language model into an actual operation instruction so as to carry out emergency command and dispatch. According to the intelligent emergency command and dispatch system and method, a more intelligent and efficient technical solution is provided, the automation level of emergency management and dispatch is improved, and it is ensured that quick response and accurate command are achieved in a complex and dynamic emergency scene.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an intelligent emergency command and dispatch system and method. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the maturity of natural language processing (NLP), computer vision and multimodal technology, various industries have begun to widely use intelligent means to improve the efficiency of resource management and scheduling. In the field of emergency management, video monitoring and scheduling have become key links. It is particularly important to obtain and analyze on-site information in real time and make decisions quickly. However, existing technical means still have great limitations when dealing with complex and sudden incidents.

[0003] Existing technologies mainly rely on manual operation for video scheduling and command. These systems require operators to manually select monitoring sources and manually execute instructions to complete screen switching and resource calls. Although these systems can meet basic daily needs, in emergencies, manual operation often leads to delayed response speed, which in turn affects the efficiency of event processing. This highly manual operation mode limits its application in efficient and rapid response scenarios.

[0004] In the field of voice control, existing systems generally use keyword matching technology to identify user commands. When the user issues a command, the system extracts the keywords in the voice and matches them with the preset command words, thereby triggering the corresponding operation. This type of system has high requirements for the accuracy of user expression, and the user must strictly follow the preset command format to ensure the execution of the operation. This not only limits the flexibility of the operation, but also increases the probability of error. Especially in emergency situations, the flexible and accurate operation requirements are difficult to meet.

[0005] In addition, in terms of video content understanding, existing technologies are mostly based on object detection or specific scene recognition. Although these algorithms perform well in the recognition of a single target, their application scope is limited and they can usually only recognize preset target types. If more types of targets or complex scenes need to be recognized, additional training is usually required for each target, which is not only costly, but also has unsatisfactory recognition effects in multi-target scenarios. Existing technologies make it difficult to achieve intelligent analysis and real-time warning of multiple targets and multiple scenarios in emergency scenarios.

[0006] Therefore, although existing technologies can meet basic needs in some aspects, their lack of flexibility, intelligence and rapid response makes it difficult for them to play their maximum role in complex emergency management and resource scheduling. In order to meet these challenges, the existing system urgently needs to introduce more intelligent technical means to improve processing efficiency and emergency response capabilities. Summary of the invention

[0007] The present invention provides an intelligent emergency command and dispatch system and method, which are used to solve the problem that traditional technologies in the prior art are insufficient in flexibility, intelligence and rapid response, and cannot play the maximum role in complex emergency management and resource scheduling.

[0008] An intelligent emergency command and dispatch system in the present invention comprises:

[0009] A voice input device, used to input voice data to be analyzed into the system;

[0010] A data conversion device, used to convert the speech data to be analyzed into text data and transmit it to the large language model for analysis;

[0011] A large language model, used to analyze the text data and generate instructions;

[0012] A control device is used to convert the instructions generated by the large language model into actual operation instructions for emergency command and dispatch.

[0013] Optionally, the large language model further includes:

[0014] A word segmentation module, used for performing word segmentation processing on the text data to generate a plurality of words;

[0015] An encoding module, used for encoding the word to generate characteristic information of the word;

[0016] A calculation module, used for respectively calculating the attention of the word in multiple parallel subspaces to obtain the semantic output of the word in multiple subspaces, splicing the multiple semantic outputs and converting them into a final expression;

[0017] An intention understanding module, used for generating a relational expression of the words in the text data based on the final expression to obtain intention information;

[0018] An instruction generation module is used to convert the intention information into the instruction based on the feature information of the word.

[0019] Optionally, the control device further includes: a semantic similarity calculation module, configured to determine an operation scheme according to the instructions generated by the large language model to generate actual operation instructions.

[0020] Optionally, the system further includes:

[0021] Video acquisition equipment, used for acquiring image data;

[0022] a multimodal large model, configured to receive the image data and analyze a plurality of objects in the image data to generate an analysis result;

[0023] The early warning device is used to generate early warning information according to the analysis result.

[0024] Optionally, the multimodal large model includes:

[0025] A video segmentation unit, used for segmenting the image data to obtain a plurality of frame data;

[0026] A feature extraction unit, used for performing feature extraction on the plurality of frame data to obtain first high-dimensional feature data;

[0027] A target detection unit, used to detect a target in the first high-dimensional feature data and mark key information of the target;

[0028] A target conversion unit, used to convert the first high-dimensional feature data of the target into second high-dimensional feature data; the second high-dimensional feature data represents key information of the target;

[0029] A multimodal fusion unit is used to fuse the second high-dimensional feature data with the third high-dimensional feature data, and obtain the fused feature representation to generate an analysis result; the third high-dimensional feature data includes: visual features and text annotation information.

[0030] An intelligent emergency command and dispatch method in the present invention comprises:

[0031] Receiving voice data to be analyzed;

[0032] Convert the speech data to be analyzed into text data and pass it to the large language model for analysis;

[0033] The large language model analyzes the text data and generates instructions;

[0034] The instructions are converted into actual operation instructions for emergency command and dispatch.

[0035] Optionally, the large language model analyzing the text data and generating instructions includes:

[0036] Performing word segmentation processing on the text data to generate a plurality of words;

[0037] encoding the word to generate characteristic information of the word;

[0038] Calculating the attention of the word in multiple parallel subspaces respectively to obtain the semantic output of the word in multiple subspaces, splicing the multiple semantic outputs and converting them into a final expression;

[0039] generating a relational expression of the word in the text data based on the final expression to obtain intention information;

[0040] Based on the feature information of the word, the intention information is converted into the instruction.

[0041] Optionally, the method further includes:

[0042] Collecting image data and analyzing a plurality of objects in the image data to generate analysis results;

[0043] Generate warning information according to the analysis results.

[0044] Optionally, analyzing the multiple targets in the image data to generate analysis results includes:

[0045] Segmenting the image data to obtain a plurality of frame data;

[0046] Performing feature extraction on the plurality of frame data to obtain first high-dimensional feature data;

[0047] Detecting a target in the first high-dimensional feature data and marking key information of the target;

[0048] Converting the first high-dimensional feature data of the target into second high-dimensional feature data; wherein the second high-dimensional feature data represents key information of the target;

[0049] The second high-dimensional feature data is fused with the third high-dimensional feature data, and a fused feature representation is obtained to generate an analysis result; the third high-dimensional feature data includes: visual features and text annotation information.

[0050] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any of the methods described above.

[0051] The intelligent emergency command and dispatch system and method of the present invention provide a more intelligent and efficient technical solution to improve the automation level of emergency management and dispatch, and ensure rapid response and accurate command in complex and dynamic emergency scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a structural schematic diagram of an intelligent emergency command and dispatch system in an embodiment of the present invention;

[0053] Figure 2 It is a flow chart of an intelligent emergency command and dispatch method in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0055] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0056] The embodiment of the present invention provides an intelligent emergency command and dispatch system. Figure 1 As shown, the system comprises:

[0057] The voice input device 10 is used to input the voice data to be analyzed into the system. Specifically, the voice input device may include: a microphone and a voice acquisition terminal. The microphone receives and amplifies the user's voice and the voice acquisition terminal acquires the voice data to be analyzed and inputs it into the system. For example, the user inputs a voice command through the microphone, such as "contact the on-site personnel for a meeting", and the voice acquisition terminal acquires the voice command.

[0058] The data conversion device 20 is used to convert the speech data to be analyzed into text data and pass it to the large language model for analysis. Specifically, the speech data to be analyzed is converted into text data through automatic speech recognition ASR technology, and the text data is passed to the large language model.

[0059] The large language model 30 is used to analyze the text data and generate instructions. Specifically, the large language model 30 parses the semantics of the text data, understands the user's intention, and generates instructions for subsequent control of the device to perform instruction conversion.

[0060] The control device 40 is used to convert the instructions generated by the large language model into actual operation instructions for emergency command and dispatch. Specifically, the control device receives the instructions generated by the large language model and converts them into actual operation instructions, such as starting a video conference. The system will automatically generate the final actual operation instructions and execute them by the control device. For example, if the user enters "start a video conference", the system will schedule the corresponding video stream and establish a conference connection.

[0061] Through the above embodiments of the present invention, users can use natural language to directly operate the system, reducing the reliance on manual operations and greatly improving the operational efficiency and response speed in emergency processing. The solution is highly flexible and is no longer limited by the defects of traditional keyword matching.

[0062] In the intelligent emergency command and dispatch system described in the specific embodiment of the present invention, preferably, the large language model also includes:

[0063] A word segmentation module, used for performing word segmentation processing on the text data to generate a plurality of words;

[0064] An encoding module, used for encoding the word to generate characteristic information of the word;

[0065] A calculation module, used for respectively calculating the attention of the word in multiple parallel subspaces to obtain the semantic output of the word in multiple subspaces, splicing the multiple semantic outputs and converting them into a final expression;

[0066] An intention understanding module, used for generating a relational expression of the words in the text data based on the final expression to obtain intention information;

[0067] An instruction generation module is used to convert the intention information into the instruction based on the feature information of the word.

[0068] In a specific embodiment, the word segmentation module receives the converted text data, performs word segmentation on the text data through a Tokenizer, and then uses an encoding module to encode the words after the word segmentation to generate characteristic information of the words, wherein the characteristic information of the words is a feature vector of the words.

[0069] The calculation module calculates the attention of the word in multiple parallel subspaces to obtain the semantic output of the word in multiple subspaces, splices multiple semantic outputs and converts them into the final expression. Specifically, in a single subspace, the relationship between each word and all other words in the input sequence is calculated through the self-attention mechanism to capture contextual information, generate multiple vectors, such as query, key and value vectors, and calculate the self-attention score to weight the important content in the input. In multiple parallel subspaces, the attention of the word is calculated according to the above method based on the multi-head attention mechanism, so that the model can understand the input information from different semantic perspectives. The semantic outputs of all parallel subspaces will be spliced ​​and converted into the final expression through a linear layer.

[0070] After the multi-head attention mechanism, the context representation of each input word is generated, and the intent understanding module extracts the intent information through the Transformer encoder-decoder structure to obtain the user intent. The encoded information is passed to the decoder, which extracts the user intent through deep learning, such as "contact the on-site personnel for a meeting" or "start a video conference."

[0071] In a preferred embodiment, the intent understanding module also represents the generated final expression through features, and converts it into computer-understandable feature vectors through a linear layer. These feature vectors represent different parameters in the user's intent, such as location, time, object, etc.

[0072] The instruction generation module converts the intent information into executable instructions based on the feature information of each word. These feature information vectorizes the intent information and generates formatted data for retrieval or operation. The vectorized intent information is matched with the existing resources and operation modes in the system to determine what kind of instruction it is.

[0073] In the intelligent emergency command and dispatch system described in the specific embodiment of the present invention, preferably, the control device also includes: a semantic similarity calculation module, which is used to determine the operation plan based on the instructions generated by the large language model to generate actual operation instructions. Specifically, the most appropriate operation plan is determined by the semantic similarity calculation module, and the similarity calculation usually uses cosine similarity to measure the similarity between the query feature and the system resources and operation mode to determine the final actual operation instruction.

[0074] The intelligent emergency command and dispatch system described in the specific embodiment of the present invention preferably further includes:

[0075] The video acquisition device is used to acquire image data; specifically, the video acquisition device in the embodiment of the present invention can be a camera, a monitoring probe, a desktop monitoring, a fusion terminal, a single-soldier device, etc.

[0076] The multimodal large model is used to receive the image data and analyze multiple targets in the image data to generate analysis results; the multimodal large model can analyze multiple targets in the image data in real time.

[0077] The early warning device is used to generate early warning information according to the analysis results. Specifically, the early warning device generates real-time early warning information according to the analysis results, and automatically processes or notifies relevant business systems.

[0078] In the intelligent emergency command and dispatch system described in the specific embodiment of the present invention, preferably, the multi-modal large model includes:

[0079] A video segmentation unit, used for segmenting the image data to obtain a plurality of frame data;

[0080] A feature extraction unit, used for performing feature extraction on the plurality of frame data to obtain first high-dimensional feature data;

[0081] A target detection unit, used to detect a target in the first high-dimensional feature data and mark key information of the target;

[0082] A target conversion unit, used to convert the first high-dimensional feature data of the target into second high-dimensional feature data; the second high-dimensional feature data represents key information of the target;

[0083] The multimodal fusion unit is used to fuse the second high-dimensional feature data with the third high-dimensional feature data, and obtain the fused feature representation to generate an analysis result; the third high-dimensional feature data includes: visual features and text features.

[0084] Specifically, the video segmentation unit extracts frame data from the image data at a frequency of one second for analysis. Each frame of data represents time t i The scene at the moment is marked as F(t i ):

[0085] F(t i )=[p ij ],i,j∈[1,N]

[0086] Among them, F(t i ) indicates that at time t i The frame captured at time contains pixel information p ij .

[0087] The feature extraction unit uses a convolutional neural network (CNN) to collect frame data F(t i ) for feature extraction. CNN transforms frame data into the first high-dimensional feature vector C(t i ):

[0088] C(t i )=CNN(F(t i )).

[0089] The target detection unit uses the target detection algorithm to detect C(t i ) to detect different targets D(t i ) and annotate key information such as its category, such as "person", "car", "fire", etc.

[0090] D(t i )={(x k ,y k , w k ,h k , l k )}, k∈[1,M]

[0091] Among them, x k ,y k , w k ,h k , l k is the bounding box coordinates and size of target k, lk is the category label, and M is the number of targets detected in the current frame data.

[0092] The target conversion unit performs feature vectorization and converts the target D(t i ) is vectorized into the second highest dimensional feature vector V k (t i ) for subsequent analysis. Feature vectorization can contain information such as the location, size, and category of the target.

[0093] V k (t i )=E(x k ,y k , w k ,h k , l k )

[0094] Among them, V k (t i ) is time t i The second highest dimensional feature vector when , E(·) is the feature encoding function.

[0095] The multimodal fusion unit combines the second high-dimensional feature vector, as well as visual features and text features (such as scene descriptions or labels). The system maps this information into the same vector space to generate a fused feature representation F fusion (t i ).

[0096] F fusion (t i )=W v ·V k (t i )+W t ·T i

[0097] Among them, W v and W t are the weight matrices of visual and text features, respectively, T i is a text feature.

[0098] In a specific embodiment of the present invention, the system is connected to the real-time video streams of multiple surveillance cameras. The multimodal large model analyzes the video content and identifies possible dangerous situations in the scene, such as "fire", "crowd gathering", "traffic accident", etc. Based on the analysis results, the system automatically generates warning information and notifies the relevant business system for further processing.

[0099] The embodiment of the present invention also provides an intelligent emergency command and dispatch method, such as Figure 2 As shown, the method includes:

[0100] Step 501, receiving voice data to be analyzed;

[0101] Step 502, converting the speech data to be analyzed into text data and passing it to the large language model for analysis;

[0102] Step 503, the large language model analyzes the text data and generates instructions;

[0103] Step 504: convert the instructions into actual operation instructions for emergency command and dispatch.

[0104] In the intelligent emergency command and dispatch method described in the specific embodiment of the present invention, preferably, the large language model analyzes the text data and generates instructions, including:

[0105] Performing word segmentation processing on the text data to generate a plurality of words;

[0106] encoding the word to generate characteristic information of the word;

[0107] Calculating the attention of the word in multiple parallel subspaces respectively to obtain the semantic output of the word in multiple subspaces, splicing the multiple semantic outputs and converting them into a final expression;

[0108] generating a relational expression of the word in the text data based on the final expression to obtain intention information;

[0109] Based on the feature information of the word, the intention information is converted into the instruction.

[0110] The intelligent emergency command and dispatch method described in the specific embodiment of the present invention preferably further includes:

[0111] Collecting image data and analyzing a plurality of objects in the image data to generate analysis results;

[0112] Generate warning information according to the analysis results.

[0113] The intelligent emergency command and dispatch method described in the specific embodiment of the present invention preferably analyzes the multiple targets in the image data to generate the analysis results, including:

[0114] Segmenting the image data to obtain a plurality of frame data;

[0115] Performing feature extraction on the plurality of frame data to obtain first high-dimensional feature data;

[0116] Detecting a target in the first high-dimensional feature data and marking key information of the target;

[0117] Converting the first high-dimensional feature data of the target into second high-dimensional feature data; wherein the second high-dimensional feature data represents key information of the target;

[0118] The second high-dimensional feature data is fused with the third high-dimensional feature data, and a fused feature representation is obtained to generate an analysis result; the third high-dimensional feature data includes: visual features and text annotation information.

[0119] This embodiment achieves a comprehensive understanding of complex scenarios through multimodal technology, can proactively identify risks and issue warnings, effectively improves the system's ability to respond to emergencies, reduces the burden of manual monitoring, and improves the accuracy and timeliness of emergency response.

[0120] A specific embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any of the methods described above.

[0121] The intelligent emergency command and dispatch system and method described in the specific embodiments of the present invention provide a more intelligent and efficient technical solution to improve the level of automation of emergency management and dispatch, and ensure rapid response and precise command in complex and dynamic emergency scenarios.

[0122] 1. Improve operational efficiency: By introducing intelligent technology, simplify the resource scheduling process, and realize automated and intelligent command and dispatch, the emergency response speed can be significantly improved.

[0123] 2. Enhance the flexibility of voice control: To solve the problem of insufficient flexibility of voice control, through advanced semantic understanding technology, the system can accurately understand the intention and perform operations based on the user's natural language instructions without relying on specific keywords, thereby improving the flexibility and accuracy of operations.

[0124] 3. Improve the ability to understand video content: It solves the limitations of existing video content analysis capabilities, uses multimodal technology to intelligently identify and analyze multiple targets and plots in the video, proactively discovers potential risks, and issues real-time warnings to ensure the system's intelligent response capabilities in complex scenarios.

[0125] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. An intelligent emergency command and dispatch system, characterized in that: The system comprises: A voice input device, used to input voice data to be analyzed into the system; A data conversion device, used to convert the speech data to be analyzed into text data and transmit it to the large language model for analysis; A large language model, used to analyze the text data and generate instructions; A control device is used to convert the instructions generated by the large language model into actual operation instructions for emergency command and dispatch.

2. The intelligent emergency command and dispatch system according to claim 1 is characterized in that: The large language model also includes: A word segmentation module, used for performing word segmentation processing on the text data to generate a plurality of words; An encoding module, used for encoding the word to generate characteristic information of the word; A calculation module, used for respectively calculating the attention of the word in multiple parallel subspaces to obtain the semantic output of the word in multiple subspaces, splicing the multiple semantic outputs and converting them into a final expression; An intention understanding module, used for generating a relational expression of the words in the text data based on the final expression to obtain intention information; An instruction generation module is used to convert the intention information into the instruction based on the feature information of the word.

3. The intelligent emergency command and dispatch system according to claim 1 is characterized in that: The control device further includes: a semantic similarity calculation module, which is used to determine an operation scheme according to the instructions generated by the large language model to generate actual operation instructions.

4. The intelligent emergency command and dispatch system according to claim 1 is characterized in that: The system further comprises: Video acquisition equipment, used for acquiring image data; a multimodal large model, configured to receive the image data and analyze a plurality of objects in the image data to generate an analysis result; The early warning device is used to generate early warning information according to the analysis result.

5. The intelligent emergency command and dispatch system according to claim 4 is characterized in that: The multimodal large model includes: A video segmentation unit, used for segmenting the image data to obtain a plurality of frame data; A feature extraction unit, used for performing feature extraction on the plurality of frame data to obtain first high-dimensional feature data; A target detection unit, used to detect a target in the first high-dimensional feature data and mark key information of the target; A target conversion unit, used to convert the first high-dimensional feature data of the target into second high-dimensional feature data; the second high-dimensional feature data represents key information of the target; A multimodal fusion unit is used to fuse the second high-dimensional feature data with the third high-dimensional feature data, and obtain the fused feature representation to generate an analysis result; the third high-dimensional feature data includes: visual features and text annotation information.

6. An intelligent emergency command and dispatch method, characterized in that: The method comprises: Receiving voice data to be analyzed; Convert the speech data to be analyzed into text data and pass it to the large language model for analysis; The large language model analyzes the text data and generates instructions; The instructions are converted into actual operation instructions for emergency command and dispatch.

7. The intelligent emergency command and dispatch method according to claim 6 is characterized in that: The large language model analyzes the text data and generates instructions, including: Performing word segmentation processing on the text data to generate a plurality of words; encoding the word to generate characteristic information of the word; Calculating the attention of the word in multiple parallel subspaces respectively to obtain the semantic output of the word in multiple subspaces, splicing the multiple semantic outputs and converting them into a final expression; generating a relational expression of the word in the text data based on the final expression to obtain intention information; Based on the feature information of the word, the intention information is converted into the instruction.

8. The intelligent emergency command and dispatch method according to claim 6 is characterized in that: The method further comprises: Collecting image data and analyzing a plurality of objects in the image data to generate analysis results; Generate warning information according to the analysis results.

9. The intelligent emergency command and dispatch method according to claim 8, characterized in that: Analyzing the plurality of targets in the image data to generate analysis results includes: Segmenting the image data to obtain a plurality of frame data; Performing feature extraction on the plurality of frame data to obtain first high-dimensional feature data; Detecting a target in the first high-dimensional feature data and marking key information of the target; Converting the first high-dimensional feature data of the target into second high-dimensional feature data; wherein the second high-dimensional feature data represents key information of the target; The second high-dimensional feature data is fused with the third high-dimensional feature data, and a fused feature representation is obtained to generate an analysis result; the third high-dimensional feature data includes: visual features and text annotation information.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of claims 6 to 9.

Images