Human-computer interaction safety supervision method and device based on large model

Abstract

The invention provides a man-machine interaction safety supervision method and device based on a large model, and relates to the technical field of man-machine interaction safety supervision. The method comprises the steps of performing data processing through a Clip coding model according to an instruction text and an interaction scene image to obtain global image features and global text features; based on a security database, performing data retrieval according to the global image features and the global text features to obtain a matched image and a matched text; according to the global image features, the global text features, the matched images and the matched texts, interactive security check is carried out through a multi-level security check model, and security feedback information is obtained. The invention relates to a multi-level high-efficiency man-machine interaction safety supervision method based on a large model.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction safety supervision, and in particular to a human-computer interaction safety supervision method and device based on a large model. Background Art

[0002] In today's era, the rapid development of artificial intelligence technology is promoting the widespread application of large models in the field of human-computer interaction. In scenes such as manufacturing and home, the interaction and integration between humans and robots is the key to achieving technological progress. However, in the process of human-computer interaction, the risk of harm caused by robots to humans has become a key factor affecting safety.

[0003] In the previous human-computer interaction security supervision methods, identity authentication and access control-based technologies were widely used, combining multiple biometric features and traditional authentication methods such as passwords, facial recognition, and voice recognition to enhance the reliability of user identity authentication, and assign access rights according to the user's role to achieve logical separation of users and permissions, thereby limiting unsafe illegal access; the proposed security supervision method based on behavior analysis and intention judgment is also a hot topic in technical research, distinguishing normal operations from potential non-safe attacks through semantic analysis and intention recognition. However, in a dynamic and uncertain interactive environment, the current human-computer safety interaction system mainly relies on the robot's perception of the environment to achieve passive avoidance, and seriously lacks the adaptive decision-making ability based on mutual cognition between the two parties.

[0004] In the era of big models, the potential safety hazards on the big model side cannot be ignored. Some users can cleverly circumvent the security defenses of big models, thereby inducing artificial intelligence to generate illegal content, or even controlling robots to perform dangerous actions, which seriously threatens the safety of human-computer interaction. What is even more tricky is that in certain specific scenarios, even if the input and output instructions of the big model do not contain explicit dangerous keywords, it may still cause safety problems, which greatly increases the difficulty and complexity of security protection.

[0005] In the existing technology, there is a lack of a multi-level and efficient human-computer interaction safety supervision method based on a large model. Summary of the invention

[0006] In order to solve the technical problem that the human-computer interaction in the prior art lacks the adaptive decision-making ability based on mutual cognition between the two parties, the embodiment of the present invention provides a method and device for monitoring the safety of human-computer interaction based on a large model. The technical solution is as follows:

[0007] On the one hand, a method for human-computer interaction safety supervision based on a large model is provided, the method is implemented by a human-computer interaction safety supervision device, and the method includes:

[0008] The microphone is used to collect audio of voice commands, and OpenAI Whisper is used to convert data to obtain command text. The camera is used to capture images of the interactive scene to obtain interactive scene images.

[0009] According to the instruction text and the interactive scene image, the Clip coding model is used to process the data to obtain the global image features and global text features;

[0010] Based on the security database, data retrieval is performed according to global image features and global text features to obtain matching images and matching texts;

[0011] Based on global image features, global text features, matching images and matching texts, interactive security checks are performed through a multi-level security check model to obtain security feedback information;

[0012] Based on the safety feedback information, interactive safety confirmation is performed through a large language model to obtain a confirmation result; based on the confirmation result, the robot performs actions.

[0013] On the other hand, a large model-based human-computer interaction safety supervision device is provided, and the device is applied to a large model-based human-computer interaction safety supervision method, and the device includes:

[0014] The information acquisition module is used to collect audio of voice commands through a microphone and use OpenAIWhisper to convert data to obtain command text; the interactive scene is captured through a camera to obtain an interactive scene image;

[0015] The information encoding module is used to process the instruction text and the interactive scene image through the Clip encoding model to obtain the global image features and the global text features;

[0016] An information retrieval module is used to retrieve data based on a security database according to global image features and global text features to obtain matching images and matching texts;

[0017] A security feedback module is used to perform interactive security checks through a multi-level security check model based on global image features, global text features, matching images, and matching texts to obtain security feedback information;

[0018] The action execution module is used to perform interactive security confirmation through a large language model based on security feedback information to obtain a confirmation result; based on the confirmation result, the robot performs an action.

[0019] On the other hand, a human-computer interaction safety supervision device is provided, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned large-model-based human-computer interaction safety supervision methods is implemented.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned large model-based human-computer interaction safety supervision methods.

[0021] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0022] The present invention proposes a human-computer interaction security supervision method given a large model. Through multimodal input processing technology, it can accurately identify various potential risks including text, voice, images, etc.; combined with a deep user behavior analysis algorithm, it can effectively detect hidden security threats; equipped with a dynamic security database update system, it can learn new attack patterns in real time and update protection strategies. The present invention significantly improves the security protection capabilities of the human-computer interaction system. It can not only prevent explicit dangerous instructions, but also effectively identify and block implicit security threats, providing all-round and intelligent security protection for various human-computer interaction application scenarios. The present invention is a multi-level and efficient human-computer interaction security supervision method based on a large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a flow chart of a method for supervising the safety of human-computer interaction based on a large model provided by an embodiment of the present invention;

[0025] Figure 2 It is a block diagram of a human-computer interaction safety monitoring device based on a large model provided by an embodiment of the present invention;

[0026] Figure 3 It is a structural schematic diagram of a human-computer interaction safety monitoring device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0028] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0029] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0030] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] The embodiment of the present invention provides a method for monitoring the safety of human-computer interaction based on a large model. The method can be implemented by a human-computer interaction safety monitoring device, which can be a terminal or a server. Figure 1 The flowchart of the human-computer interaction safety supervision method based on a large model is shown. The processing flow of the method may include the following steps:

[0033] S1. Use the microphone to collect audio of voice commands, and use OpenAI Whisper to convert data to obtain command text; use the camera to capture images of the interactive scene to obtain interactive scene images.

[0034] In a feasible implementation, a microphone is used in the present invention to collect the voice commands issued by the user. The voice input uses OpenAI Whisper to convert the voice into text, which is implemented by calling the openai official application programming interface (Application Programming Interface, API).

[0035] The user initiates a request and provides an audio file, hoping to convert the speech content into text or translate it. The OpenAIWhisper API receives the user's request, processes the audio file, and returns the transcription or translation result. The API request parameters include the parameters specified by the user when calling the API, such as model name, response format, language, etc. The text content returned by the API indicates that the speech in the audio file has been converted to text.

[0036] Install the OpenAI and requests libraries. The sample code for calling the "whisper-1" model is as follows:

[0037] response = openai.Audio.transcriptions.create(

[0038] model="whisper-1", # Use Whisper model

[0039] file=audio_file, # uploaded audio file

[0040] response_format="text" # The format of the returned result (can be "text" or "json");

[0041] The image input is collected using a camera with a resolution of 1920×1080. The camera captures the image and performs operations such as resizing and center cropping. The 1080p original image captured by the camera is processed to obtain a 224×224 three-channel image.

[0042] In this process, the resized image is center cropped to ensure that the center part of the image is retained. It is cropped into a three-channel image of size 896×896. The image is averaged with a range of 4 and a step length of 4, and compressed into a three-channel image of 224×224. The image pixels are normalized to meet the input requirements of the model.

[0043] S2. According to the instruction text and the interactive scene image, data processing is performed through the Clip encoding model to obtain global image features and global text features.

[0044] Optionally, data processing is performed through a Clip coding model according to the instruction text and the interactive scene image to obtain global image features and global text features, including:

[0045] Perform pixel segmentation on the scene interaction image to obtain segmented small blocks; input the segmented small blocks into the linear projection layer for mapping to obtain image mapping vectors;

[0046] Based on the Clip.tokenize function, perform string conversion according to the instruction text to obtain a text tokenization tensor; input the text tokenization tensor into the embedding layer for high-dimensional space mapping to obtain a text mapping vector;

[0047] Add position encoding to the image mapping vector and the text mapping vector to obtain the image embedding vector and the text embedding vector;

[0048] Based on the multi-head attention mechanism, feature extraction is performed through a multi-layer Transformer encoder according to the image embedding vector and the text embedding vector to obtain image features and text features;

[0049] Based on the classification marks in the image features, the image features are aggregated to obtain the global image features;

[0050] Based on the end mark in the text feature, the text feature is aggregated to obtain the global text feature.

[0051] In a feasible implementation, in this step, a contrastive language-image pre-training (Clip) encoding model is used to encode text and images simultaneously.

[0052] The openai / clip-vit-base-patch32 model is used for encoding. The image is divided into 16×16 patches, each patch is 16×16 pixels in size, and the number of patches calculated is (224 / 16)×(224 / 16) = 14×14 = 196 patches.

[0053] Each image patch is linearly projected to a fixed-dimensional vector space. The feature dimension of each patch is 768. A positional encoding is added to each patch to preserve the spatial information of the image. The dimension of the positional encoding remains unchanged at 197×768.

[0054] The processed image embedding vector is input into the Transformer Encoder for feature extraction. The multi-head self-attention mechanism is used to capture the relationship between different regions of the image. In each Transformer block, the dimension remains unchanged at 197×768.

[0055] After multiple layers of processing by Transformer Encoder, a high-level feature representation of each patch is obtained. A special Classification Token (CLS) tag is used to aggregate the global features of the entire image. Finally, a 512-dimensional vector is output as the encoding representation of the entire image.

[0056] The [CLS] token is usually used to provide a fixed position in the input sequence of the model to summarize global information for classification tasks. In the Vision Transformer, the [CLS] token is added to the beginning of the input sequence to extract feature representations of the global image.

[0057] Use the clip.tokenize function to convert a list of text strings into a tokenized tensor that can be input into a CLIP model. This function handles the tokenization process, including adding special tokens, padding, and truncating the text to fit the model's expected input format.

[0058] Convert the input text tokens into embedding vectors. These embedding vectors are numerical representations that the model can process. token_embedding is an embedding layer that maps each token into a high-dimensional vector space. Add position encoding to each text token to preserve the order information of the text. The dimension of the position encoding remains unchanged at 77×512.

[0059] The processed text embedding vector is input into Transformer Encoder for feature extraction. The multi-head self-attention mechanism is used to capture the relationship between different parts of the text. After multi-layer processing of Transformer Encoder, a high-level feature representation of each text token is obtained. A special End of Sequence (EOS) marker is used to aggregate the global features of the entire text. Finally, a 512-dimensional vector is output as the encoded representation of the entire text.

[0060] The [EOS] token is used to indicate the end of a text sequence. In CLIP’s text encoder, the input text sequence is tokenized and the [EOS] token is added to the end of the sequence.

[0061] The sample code for image and text encoding using the openai / clip-vit-base-patch32 model is as follows:

[0062] import torch

[0063] from PIL import Image

[0064] from transformers import CLIPProcessor, CLIPModel

[0065] # Initialize the model and processor

[0066] model = CLIPModel.from_pretrained("openai / clip-vit-base-patch32")

[0067] processor = CLIPProcessor.from_pretrained("openai / clip-vit-base-patch32")

[0068] # Load the image

[0069] image = Image.open("example.jpg")

[0070] # Preprocess the image

[0071] inputs = processor(images=image, return_tensors="pt", padding=True)

[0072] # Encode the image

[0073] image_features = model.encode_image(inputs["pixel_values"])

[0074] # Encoding text

[0075] text = ["a photo of a cat", "a photo of a dog"]

[0076] text_inputs = processor(text=text, return_tensors="pt", padding=True)

[0077] text_features = model.encode_text(text_inputs["input_ids"])

[0078] # Calculate similarity

[0079] logits_per_image=model(image=inputs["pixel_values"], text=text_inputs["input_ids"]).logits_per_image

[0080] probs = logits_per_image.softmax(dim=1);

[0081] S3. Based on the security database, data retrieval is performed according to global image features and global text features to obtain matching images and matching texts.

[0082] Optionally, based on the security database, data retrieval is performed according to global image features and global text features to obtain matching images and matching texts, including:

[0083] Based on the database image feature set of the security database, cosine similarity calculation is performed according to the global image features to obtain an image similarity set;

[0084] Based on the database text feature set of the security database, cosine similarity calculation is performed according to the global text feature to obtain a text similarity set;

[0085] Based on a preset similarity threshold, the image similarity set and the text similarity set are screened to obtain matching image similarity and matching text similarity;

[0086] Based on the security database, an image corresponding to the matching image similarity is selected as the matching image;

[0087] Based on the security database, the text corresponding to the matching text similarity is selected as the matching text.

[0088] In a feasible implementation, since the input text and image are in the same format as the data stored in the database, both are encoded into 512-dimensional feature vectors, the information most relevant to the user's needs can be directly screened by calculating the cosine similarity.

[0089] The cosine similarity formula is as follows (1):

[0090] (1);

[0091] Where A is the embedding vector (text or image) input by the user; B is the embedding vector (text or image) stored in the database; ||A|| is the modulus of vector A; ||B||: the modulus of vector B; A·B is the dot product of vectors A and B; θ Represents the angle between the embedding vector A and the embedding vector B.

[0092] In the present invention, the user's text, image and the database's text, image are respectively formed into vector pairs for similarity calculation. First, vector pairs with similarity exceeding 0.9 are selected, and then a vector pair is randomly selected from them; if there is no vector pair exceeding 0.9, the vector pair with the highest similarity is selected.

[0093] The security database includes a database image feature set, a database image set, a database text feature set and a database text set.

[0094] In a feasible implementation, the safety database includes information such as safety of daily necessities, food safety, safety of the elderly, children, the sick and the disabled, and images of dangerous situations. This information is collected through the Internet or customized by the user. The text information is the filtered safety information, and the image information is the same as the image processed by the user input, which is a 224×224 three-channel image, and each image contains a dangerous situation. After collecting this data in advance, the supervision information is annotated for this data so that the large model can be supervised and fine-tuned later.

[0095] The image information in the database includes living room pictures, dining room pictures, bedroom pictures, etc., which are stored as 224×224 three-channel png format images, and also store feature vectors pre-encoded by clip. Specific image information, such as [standard information; there is danger in the picture; the fire source is easily accessible to children, the risk factor is 1]; [annotation information; there is danger in the picture; the medicine is easily accessible to children and ingested, the risk factor is 1].

[0096] The text information includes household life common sense, important living habits of users, weather and geographical information, etc., and stores feature vectors pre-encoded using clip. Specific text information, such as electric heaters such as heaters cannot be placed next to beds and sofas.

[0097] S4. Based on the global image features, global text features, matching images and matching texts, an interactive security check is performed through a multi-level security check model to obtain security feedback information.

[0098] Optionally, an interactive security check is performed through a multi-level security check model based on global image features, global text features, matching images, and matching texts to obtain security feedback information, including:

[0099] Based on the global image features and global text features, a lightweight multi-layer perceptron is used to perform feature fusion to obtain image-text fusion features;

[0100] According to the image-text fusion features, matching images and matching texts, the risk factor is assessed through the first-level safety inspection model to obtain the interactive risk factor;

[0101] Based on a preset risk factor threshold, the interactive risk factor is verified to obtain a verification result;

[0102] When the verification result shows that there is a safety hazard, the hazard description is generated through the second-level safety inspection model according to the image-text fusion features, matching images and matching texts to obtain an interactive hazard description.

[0103] In a feasible implementation, the present invention projects global image features (dimension = 768) into the embedding space of global text features (dimension = 4096) through a lightweight multi-layer perceptron. By inputting such a fusion of image and text into the Large Language and Vision Assistant (LLaVa) model, it can output a suitable multi-step task plan. In specific usage scenarios, a data set can be appropriately constructed for fine-tuning.

[0104] The first-level safety inspection model adds a sigmoid function after the last layer of the original LLaVa model as the output of the risk factor. In order for the large model to accurately judge the risk factor based on the input, the data set needs to be labeled in advance, with risk as 1 and safety as 0. Use such a data set to fine-tune LLaVa. When the risk factor output by the first-level safety inspection model is greater than the risk factor threshold, the second-level safety inspection model is entered. The risk factor threshold of the present invention uses 0.5.

[0105] The second-level safety inspection model needs to output specific risk factors in the image. For example, for the instruction "Put the fruit knife on the bed", the large model should output: Putting the knife on the bed is likely to cause accidental injury.

[0106] In order for the large model to accurately determine the risk factors based on the input, the data set needs to be annotated in advance, and the annotated information is written concisely and effectively by humans. Such a data set is used to fine-tune LLaVa.

[0107] Among them, the multi-level security inspection model is a pre-trained LLava large model.

[0108] In one possible implementation, LLaVA is a multimodal large model that combines language and vision capabilities, aims to understand and generate textual content associated with images or videos, and to perform comprehensive understanding and generation of images and text.

[0109] By combining language and visual information, smarter interactions and more accurate multimodal understanding can be achieved. For example, a user can upload a picture and ask related questions, and LLaVA can understand the content of the picture and generate accurate answers. LLaVA can be built on pre-trained visual and language models and fine-tuned with large-scale data to adapt to specific tasks.

[0110] LLaVA can be applied to a variety of multimodal tasks, including but not limited to image description generation, visual question answering (VQA), image-text retrieval, and multimodal dialogue.

[0111] S5. Based on the safety feedback information, interactive safety confirmation is performed through a large language model to obtain a confirmation result; based on the confirmation result, the robot performs actions.

[0112] Optionally, after the robot performs the action, the method further includes:

[0113] Based on a preset update threshold, the similarity of the matching images is verified to obtain an image update verification result;

[0114] When the image update verification result is updated, the matching image is replaced with the interactive scene image in the security database;

[0115] Based on a preset update threshold, the matching text similarity is verified to obtain a text update verification result;

[0116] When the text update verification result is an update, the instruction text is used to replace the matching text in the security database.

[0117] In a feasible implementation, the ChatGPT-3.5-turbo large model is used to determine whether the user's response to the safety feedback information is positive and affirmative. If the user indicates that the safety feedback information is correct, or the user directly raises specific requirements for safety, the large model is used to refine the safety issues in this feedback and save the image.

[0118] The text and image information are encoded into feature vectors through clip, and the cosine similarity is calculated with the feature vectors of all texts in the database. If there is a similarity above the update threshold, the latest information is retained and the old information is deleted from the database. The same action is performed on the image information. The online update optimization of the security database is realized. The update threshold of the present invention is 0.95.

[0119] The present invention proposes a human-computer interaction security supervision method given a large model. Through multimodal input processing technology, it can accurately identify various potential risks including text, voice, images, etc.; combined with a deep user behavior analysis algorithm, it can effectively detect hidden security threats; equipped with a dynamic security database update system, it can learn new attack patterns in real time and update protection strategies. The present invention significantly improves the security protection capabilities of the human-computer interaction system. It can not only prevent explicit dangerous instructions, but also effectively identify and block implicit security threats, providing all-round and intelligent security protection for various human-computer interaction application scenarios. The present invention is a multi-level and efficient human-computer interaction security supervision method based on a large model.

[0120] Figure 2 The block diagram of a human-computer interaction safety supervision device based on a large model according to an exemplary embodiment is shown. The device is used in a human-computer interaction safety supervision method based on a large model. Figure 2 The device includes an information acquisition module 210, an information encoding module 220, an information retrieval module 230, a security feedback module 240 and an action execution module 250. Among them:

[0121] The information acquisition module 210 is used to collect audio of the voice command through a microphone, and use OpenAIWhisper to convert the data to obtain the command text; and to capture the interactive scene through a camera to obtain an interactive scene image;

[0122] The information encoding module 220 is used to perform data processing through the Clip encoding model according to the instruction text and the interactive scene image to obtain global image features and global text features;

[0123] An information retrieval module 230 is used to perform data retrieval based on a security database according to global image features and global text features to obtain matching images and matching texts;

[0124] A security feedback module 240 is used to perform interactive security checks through a multi-level security check model based on global image features, global text features, matching images and matching texts to obtain security feedback information;

[0125] The action execution module 250 is used to perform interactive security confirmation through a large language model according to the security feedback information to obtain a confirmation result; according to the confirmation result, the robot performs an action.

[0126] Optionally, the information encoding module 220 is further configured to:

[0127] Perform pixel segmentation on the scene interaction image to obtain segmented small blocks; input the segmented small blocks into the linear projection layer for mapping to obtain image mapping vectors;

[0128] Based on the Clip.tokenize function, perform string conversion according to the instruction text to obtain a text tokenization tensor; input the text tokenization tensor into the embedding layer for high-dimensional space mapping to obtain a text mapping vector;

[0129] Add position encoding to the image mapping vector and the text mapping vector to obtain the image embedding vector and the text embedding vector;

[0130] Based on the multi-head attention mechanism, feature extraction is performed through a multi-layer Transformer encoder according to the image embedding vector and the text embedding vector to obtain image features and text features;

[0131] Based on the classification marks in the image features, the image features are aggregated to obtain the global image features;

[0132] Based on the end mark in the text feature, the text feature is aggregated to obtain the global text feature.

[0133] Optionally, the information retrieval module 230 is further configured to:

[0134] Based on the database image feature set of the security database, cosine similarity calculation is performed according to the global image features to obtain an image similarity set;

[0135] Based on the database text feature set of the security database, cosine similarity calculation is performed according to the global text feature to obtain a text similarity set;

[0136] Based on a preset similarity threshold, the image similarity set and the text similarity set are screened to obtain matching image similarity and matching text similarity;

[0137] Based on the security database, an image corresponding to the matching image similarity is selected as the matching image;

[0138] Based on the security database, the text corresponding to the matching text similarity is selected as the matching text.

[0139] The security database includes a database image feature set, a database image set, a database text feature set and a database text set.

[0140] Optionally, the safety feedback module 240 is further configured to:

[0141] Based on the global image features and global text features, a lightweight multi-layer perceptron is used to perform feature fusion to obtain image-text fusion features;

[0142] According to the image-text fusion features, matching images and matching texts, the risk factor is assessed through the first-level safety inspection model to obtain the interactive risk factor;

[0143] Based on a preset risk factor threshold, the interactive risk factor is verified to obtain a verification result;

[0144] When the verification result shows that there is a safety hazard, the hazard description is generated through the second-level safety inspection model according to the image-text fusion features, matching images and matching texts to obtain an interactive hazard description.

[0145] Among them, the multi-level security inspection model is a pre-trained LLava large model.

[0146] Optionally, the action execution module 250, the method further includes:

[0147] Based on a preset update threshold, the similarity of the matching images is verified to obtain an image update verification result;

[0148] When the image update verification result is updated, the matching image is replaced with the interactive scene image in the security database;

[0149] Based on a preset update threshold, the matching text similarity is verified to obtain a text update verification result;

[0150] When the text update verification result is an update, the instruction text is used to replace the matching text in the security database.

[0151] The present invention proposes a human-computer interaction security supervision method given a large model. Through multimodal input processing technology, it can accurately identify various potential risks including text, voice, images, etc.; combined with a deep user behavior analysis algorithm, it can effectively detect hidden security threats; equipped with a dynamic security database update system, it can learn new attack patterns in real time and update protection strategies. The present invention significantly improves the security protection capabilities of the human-computer interaction system. It can not only prevent explicit dangerous instructions, but also effectively identify and block implicit security threats, providing all-round and intelligent security protection for various human-computer interaction application scenarios. The present invention is a multi-level and efficient human-computer interaction security supervision method based on a large model.

[0152] Figure 3 is a schematic diagram of the structure of a human-computer interaction safety monitoring device provided by an embodiment of the present invention, such as Figure 3 As shown, the human-machine interaction safety supervision device may include the above Figure 2 The human-computer interaction safety supervision device based on the large model is shown. Optionally, the human-computer interaction safety supervision device 310 may include a first processor 2001 .

[0153] Optionally, the human-machine interaction safety supervision device 310 may further include a memory 2002 and a transceiver 2003 .

[0154] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0155] Combine the following Figure 3 The components of the human-machine interaction safety monitoring device 310 are specifically introduced as follows:

[0156] The first processor 2001 is the control center of the human-computer interaction safety supervision device 310, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0157] Optionally, the first processor 2001 may perform various functions of the human-computer interaction safety supervision device 310 by running or executing a software program stored in the memory 2002 , and calling data stored in the memory 2002 .

[0158] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in FIG.

[0159] In a specific implementation, as an embodiment, the human-computer interaction safety supervision device 310 may also include multiple processors, such as Figure 3 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0160] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0161] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and access the first processor 2001 through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0162] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0163] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 3 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0164] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0165] It should be noted that Figure 3 The structure of the human-computer interaction safety supervision device 310 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0166] In addition, the technical effects of the human-computer interaction safety supervision device 310 can refer to the technical effects of the human-computer interaction safety supervision method based on a large model described in the above method embodiment, and will not be repeated here.

[0167] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0168] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0169] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0170] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0171] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0172] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0173] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0175] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0176] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0178] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0179] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A human-computer interaction safety supervision method based on a large model, characterized in that: The method comprises: The microphone is used to collect audio of voice commands, and OpenAI Whisper is used to convert data to obtain command text. The camera is used to capture images of the interactive scene to obtain interactive scene images. According to the instruction text and the interactive scene image, the Clip coding model is used to process the data to obtain the global image features and global text features; Based on the security database, data retrieval is performed according to global image features and global text features to obtain matching images and matching texts; Based on global image features, global text features, matching images and matching texts, interactive security checks are performed through a multi-level security check model to obtain security feedback information; Based on the safety feedback information, interactive safety confirmation is performed through a large language model to obtain a confirmation result; based on the confirmation result, the robot performs actions.

2. The method for monitoring the safety of human-computer interaction based on a large model according to claim 1 is characterized in that: The method of performing data processing according to the instruction text and the interactive scene image through the Clip coding model to obtain global image features and global text features includes: Perform pixel segmentation on the scene interaction image to obtain segmented small blocks; input the segmented small blocks into the linear projection layer for mapping to obtain image mapping vectors; Based on the Clip.tokenize function, perform string conversion according to the instruction text to obtain a text tokenization tensor; input the text tokenization tensor into the embedding layer for high-dimensional space mapping to obtain a text mapping vector; Add position encoding to the image mapping vector and the text mapping vector to obtain the image embedding vector and the text embedding vector; Based on the multi-head attention mechanism, feature extraction is performed through a multi-layer Transformer encoder according to the image embedding vector and the text embedding vector to obtain image features and text features; Based on the classification marks in the image features, the image features are aggregated to obtain the global image features; Based on the end mark in the text feature, the text feature is aggregated to obtain the global text feature.

3. The human-computer interaction safety supervision method based on a large model according to claim 1 is characterized in that: The method of performing data retrieval based on the security database according to the global image features and the global text features to obtain the matching image and the matching text includes: Based on the database image feature set of the security database, cosine similarity calculation is performed according to the global image features to obtain an image similarity set; Based on the database text feature set of the security database, cosine similarity calculation is performed according to the global text feature to obtain a text similarity set; Based on a preset similarity threshold, the image similarity set and the text similarity set are screened to obtain matching image similarity and matching text similarity; Based on the security database, an image corresponding to the matching image similarity is selected as the matching image; Based on the security database, the text corresponding to the matching text similarity is selected as the matching text.

4. The method for monitoring the safety of human-computer interaction based on a large model according to claim 3 is characterized in that: The security database includes a database image feature set, a database image set, a database text feature set and a database text set.

5. The method for monitoring the safety of human-computer interaction based on a large model according to claim 1 is characterized in that: The interactive security check is performed through a multi-level security check model based on the global image features, global text features, matching images and matching texts to obtain security feedback information, including: Based on the global image features and global text features, a lightweight multi-layer perceptron is used to perform feature fusion to obtain image-text fusion features; According to the image-text fusion features, matching images and matching texts, the risk factor is assessed through the first-level safety inspection model to obtain the interactive risk factor; Based on a preset risk factor threshold, the interactive risk factor is verified to obtain a verification result; When the verification result shows that there is a safety hazard, the hazard description is generated through the second-level safety inspection model according to the image-text fusion features, matching images and matching texts to obtain an interactive hazard description.

6. The method for monitoring the safety of human-computer interaction based on a large model according to claim 5 is characterized in that: The multi-level security check model is a pre-trained LLava large model.

7. The method for monitoring the safety of human-computer interaction based on a large model according to claim 3 is characterized in that: After the robot performs the action, the method further includes: Based on a preset update threshold, the similarity of the matching images is verified to obtain an image update verification result; When the image update verification result is updated, the matching image is replaced with the interactive scene image in the security database; Based on a preset update threshold, the matching text similarity is verified to obtain a text update verification result; When the text update verification result is an update, the instruction text is used to replace the matching text in the security database.

8. A human-computer interaction safety supervision device based on a large model, the human-computer interaction safety supervision device based on a large model is used to implement the human-computer interaction safety supervision method based on a large model as claimed in any one of claims 1 to 7, characterized in that: The device comprises: The information acquisition module is used to collect audio of voice commands through a microphone and use OpenAI Whisper to convert data to obtain command text; the camera is used to capture images of the interactive scene to obtain interactive scene images; The information encoding module is used to process the instruction text and the interactive scene image through the Clip encoding model to obtain the global image features and the global text features; An information retrieval module is used to retrieve data based on a security database according to global image features and global text features to obtain matching images and matching texts; A security feedback module is used to perform interactive security checks through a multi-level security check model based on global image features, global text features, matching images, and matching texts to obtain security feedback information; The action execution module is used to perform interactive security confirmation through a large language model based on security feedback information to obtain a confirmation result; based on the confirmation result, the robot performs an action.

9. A human-computer interaction safety monitoring device, characterized in that: The human-computer interaction safety supervision device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images